Since missing IPI was the problem, next step is to implement sending of interrupts between cores. This too is not straightforward as Pi 2 does not have the Generic Interrupt Controller present with most ARM based processors. As an alternative to this, I have come up with the suggestion of using mailboxes of each core for this purpose.
The mailbox interrupt is generated as long as as a non-zero value is written to the mailbox register. Thus, the core wanting to send an interrupt can simply write a non-zero value to mailbox 3 of target core(s). For now, I just set the mailbox to value of 1. This has been added through _CPU_SMP_Send_interrupt() function in ~/raspberrypi/startup/bspsmp.c .
void _CPU_SMP_Send_interrupt( uint32_t target_processor_index )
{
/* Generates IPI */
uint32_t *target_mb_write = (uint32_t*)(BCM2836_MAILBOX_3_WRITE_SET_BASE + 0x10 * target_processor_index);
*target_mb_write = 0x1;
}
The IPI handler will basically be a handler for mailbox 3 interrupt. The handler resets contents of target mailbox 3 to zero. This clears the interrupt. Then the RTEMS IPI handler _SMP_Inter_processor_interrupt_handler() is called.
/* writing zero to mailbox clears the interrupt */
*mb_read_clr = 0x01;
_SMP_Inter_processor_interrupt_handler();
As part of initialization for IPI support, the mailbox 3 contents have to be reset to zero and mailbox 3 interrupt enabled in the interrupt control register for each core.
In order to be able to handle the interrupt appropriately, it has to be registered first. This requires an interrupt vector be associated with the mailbox 3 interrupt which will be used by all cores. This interrupt vector is not yet determined. Once determined, the handler can be installed for the interrupt vector.
How is the mailbox used?
Out of the four mailboxes of each core, one mailbox is reserved for this purpose. I have used mailbox 3. This might have to be changed to one of mailboxes 0,1,2 as mailbox 3 has a specific purpose for SMP start up.The mailbox interrupt is generated as long as as a non-zero value is written to the mailbox register. Thus, the core wanting to send an interrupt can simply write a non-zero value to mailbox 3 of target core(s). For now, I just set the mailbox to value of 1. This has been added through _CPU_SMP_Send_interrupt() function in ~/raspberrypi/startup/bspsmp.c .
void _CPU_SMP_Send_interrupt( uint32_t target_processor_index )
{
/* Generates IPI */
uint32_t *target_mb_write = (uint32_t*)(BCM2836_MAILBOX_3_WRITE_SET_BASE + 0x10 * target_processor_index);
*target_mb_write = 0x1;
}
The IPI handler will basically be a handler for mailbox 3 interrupt. The handler resets contents of target mailbox 3 to zero. This clears the interrupt. Then the RTEMS IPI handler _SMP_Inter_processor_interrupt_handler() is called.
/* writing zero to mailbox clears the interrupt */
*mb_read_clr = 0x01;
_SMP_Inter_processor_interrupt_handler();
As part of initialization for IPI support, the mailbox 3 contents have to be reset to zero and mailbox 3 interrupt enabled in the interrupt control register for each core.
In order to be able to handle the interrupt appropriately, it has to be registered first. This requires an interrupt vector be associated with the mailbox 3 interrupt which will be used by all cores. This interrupt vector is not yet determined. Once determined, the handler can be installed for the interrupt vector.
Raspberry Pi Interrupt Handling
Some part of the interrupt handling for Raspberry pi happens in ~/raspberrypi/irq/irq.c . How to merge IPI initalization and handling with existing RPi interrupt handling has to be considered.
Thanks for this nice writeup.
ReplyDeleteCould you please point me to source-code for RPI2 RTEMS ?