Monday 10 August 2015

Starting secondary cores with RTEMS

So I could find the missing jump address where the secondary cores should start executing from for RTEMS. We use the _start() function defined in the boot.S file. When starting from there, required initializations for all modes are done correctly for each core. So I have used the following instructions which will write the start address to specific memory locations for each core.

   "ldr r2 , =_start\n"
   "ldr r1 , =0x4000009C\n"
   "str r2 , [r1]\n"
   "ldr r1 , =0x400000AC\n"
   "str r2 , [r1]\n"
   "ldr r1 , =0x400000BC\n"
   "str r2 , [r1]\n"

That's it! Starting the secondary cores is as straightforward for bare metal development with the Pi 2. With RTEMS, this has to blend with the process RTEMS uses to bring up SMP for a variant.

So, next we need to decide where this process should be started. The primary core is the only working core at this time so the task of starting other cores has to be done by it. This should be done in an early stage of boot up. I have added a function raspberrypi_wake_secondary_processors() which will be called by core 0 from the bsp_start_hook_0().

When the secondary cores reach bsp_start_hook_0() further initializations like setting up translation table and MMU will be done by calling the function start_on_secondary_processor() there. This is the function which will eventually lead the secondary cores to SMP set up through a call to _SMP_Start_multitasking_on_secondary_processor().

 While the secondary cores execute their thread, the primary core proceeds from bootcard(), and then to rtems_initialize_data_structures() in ~/cpukit/sapi/src/exinit.c under which call to _SMP_Handler_Initialize() is the entry point for SMP initialization from its side. 

However starting SMP didn't turn out to be as easy as it seemed. There were issues and understanding the flow of execution in RTEMS for primary and secondary cores helped.

I used available debugging tools to understand the execution and identify problems. This post on debugging explains what I did.



 Code


As mentioned above, the RTEMS SMP start sequence has to be followed for proper start up. Part of this sequence are certain functions which must be defined. The definitions depend on BSP.
  • For a9mpcore BSPs, the  file ~/libbsp/arm/shared/arm-a9mpcore-smp.c is used in common. Only, the function _CPU_SMP_Start_processor() is specific to each variant. For Pi 2, I have defined these functions in ~/libbsp/arm/raspberrypi/startup/bspsmp.c. This has to be added to raspberry pi Makefile.am.
  • Also the variable bsp_processor_count has to be given a default value for BSPs supporting SMP.  This is 4 in case of Pi 2 since it has 4 cores.  For this we need to add the following line in linkcmds file bsp_processor_count=DEFINED(bsp_processor_count)?bsp_processor_count : 4;
  • When running an application, the maximum number of processors that it must use is specified using CONFIGURE_SMP_MAXIMUM_PROCESSORS which is defined at build time in system.h file for the application. 
  • The number of processors which will eventually be used is minimum of the number actually present in hardware and the number configured.  This can be found in ~/cpukit/score/src/smp.c in _SMP_Handler_initialize()  cpu_count = cpu_count < cpu_max ? cpu_count : cpu_max;
  • The config.ac for Raspberry Pi had to be modified a bit to enable SMP support in the configurations file configure. It adds support for the --enable-smp option for Raspberry Pi.

No comments:

Post a Comment