Page 1 of 2

Minimal ISR jitter possible?

Posted: Mon Sep 02, 2013 3:36 pm
by maximevince
Dear EE users, community,

My application needs the least ISR jitter possible,
as we are running a very time critical ISR. (just 2 time critical isrs, the rest is less time critical).

A certain ISR reacts on an ADC threshold being crossed, and I should switch output transistors within 10 to 15us.

Therefor, I'm looking for a way to make the critical section of the RTOS scheduler as small as possible,
such that my highest priority ISR can kick in as fast as possible. I'm running on an MPC5643L, with an e200z4 core.

Is there anything forseen in the kernel for this? Is the scheduler itsself pre-emptible by interrupts?

I have written an interrupt handler that saves and restores the complete context of the CPU, so that interrupting the scheduling process should not matter. (except of course the moment where it is saving and handfull of registers, such as SRR0 and SRR1, which is really a critical section that cannot be pre-empted - but that should only last a few clocks...)

Re: Minimal ISR jitter possible?

Posted: Mon Sep 02, 2013 4:40 pm
by maximevince
One possible path I'm thinking about is modifying the scheduler,
such that it will not disable interrupts, but elevates itselves to interrupt priority 14,
and thus allowing the high priority interrupts w/ prio 15 to be run anyway. (which will have to leave the whole context in a clean way).

Is something like this possible?

Re: Minimal ISR jitter possible?

Posted: Mon Sep 02, 2013 4:46 pm
by paolo.gai
Hi!

Yes I guess so... the idea is very similar also to what has been implemented in other CPUs (like the dsPIC where the DISI instruction does not disable priority 7 of the interrupts).

Note that you have to ensure that all the usages of the interrupts are done in the correct way, since there are a few functions working on the interrupts levels /enable/disable...

Ciao,

PJ

Re: Minimal ISR jitter possible?

Posted: Tue Sep 03, 2013 10:10 am
by e.guidieri
maximevince wrote:Dear EE users, community,

My application needs the least ISR jitter possible,
as we are running a very time critical ISR. (just 2 time critical isrs, the rest is less time critical).

A certain ISR reacts on an ADC threshold being crossed, and I should switch output transistors within 10 to 15us.

Therefor, I'm looking for a way to make the critical section of the RTOS scheduler as small as possible,
such that my highest priority ISR can kick in as fast as possible. I'm running on an MPC5643L, with an e200z4 core.

Is there anything forseen in the kernel for this? Is the scheduler itsself pre-emptible by interrupts?
For historical reasons (the most important of them is that until last year there wasn't a interrupt layer standard abstraction in ERIKA portings and that now is really difficult to backport it, because we don't have no more compilers and boads for all supported portings) Erika Kernel API lock all the interrupts, so ISR1 pre-emption of the kernel is not allowed,
this is completly accepted by OSEK specification and it's almost never a problem, because when you turn off checks ( STATUS = STANDARD; in Oil OS ) API are really, really short.

But if you really need preemption of this kind you can modify Kernel Critical Section HAL EE_hal_begin_nested_primitive and EE_hal_end_nested_primitive (these API are in $ee/pkg/cpu/e200zx/inc/ee_internal.h) in the following way:

Code: Select all

/* called as _first_ function of a primitive that can be called in
   an IRQ and in a task */
__INLINE__ EE_FREG __ALWAYS_INLINE__ EE_hal_begin_nested_primitive(void)
{
  EE_TYPEISR2PRIO prev_prio = EE_e200zx_get_int_prio();
  EE_e200zx_set_int_prio(14U);
  return (EE_FREG)prev_prio;
}


/* Called as _last_ function of a primitive that can be called in
   an IRQ and in a task.  Enable IRQs if they were enabled before entering. */
__INLINE__ void __ALWAYS_INLINE__ EE_hal_end_nested_primitive(EE_FREG f)
{
  EE_e200zx_set_int_prio( (EE_TYPEISR2PRIO)f );
}
This will leave you priority 15 to preempt the kernel (lowing the number passed to EE_e200zx_set_int_prio will leave more priorities to do same thing).

This is enough, but be aware SuspendOSInterrupts will still disable all the interrupts, if you are not using this API just don't bother you but if you are rely on that you have to modify its behaviour.

I hope this would be useful, give us a feedback or a patch if you make the things work.

I have written an interrupt handler that saves and restores the complete context of the CPU, so that interrupting the scheduling process should not matter. (except of course the moment where it is saving and handfull of registers, such as SRR0 and SRR1, which is really a critical section that cannot be pre-empted - but that should only last a few clocks...)
Mixing interrupt vector implementations is not really an easy task, for sure you need to do that to tweak a couple of asm files in Erika CPU layer...

With the modification suggested above try to use static ISR handling and declare your ISR as ISR1*, I'm sure you will satisfy your requirements in this way.

*ISR1 have to have higher priorities than all other ISR2 ( but don't worry RT-Druid will mutter if you make some mistakes :) ).

Re: Minimal ISR jitter possible?

Posted: Tue Sep 03, 2013 5:01 pm
by maximevince
Wow, that's just great support!
Fast, to-the-point answers!

I have changed my config into the following:
- FP kernel
- Nested IRQs
- Semaphores
- Therefor also Multi-Stack
- COM module
- Static ISR config
- Status = Standard (however I did not see this affect my ISR jitter?)

-Two tasks with full preemptive scheduling.
---One task waiting for the semaphore (high prio)
---One task posting a semaphore every x time (lower prio)
-A Periodic Interrupt Timer ISR @ prio 15 (max)

=> I'm getting an interrupt jitter of about 6 us with that. Which is awesome already.

Then, I tried the trick, overriding EE_hal_xx_nested_primitives.
This does work for a moment, however, after a random amount of time, I end up in an EE_e200zx_instruction_tlb_error.
I think that the PIT interrupt can happen when saving or restoring SRR-registers now, which can mess up the cpu. Is that possible?

Plus, using this technique, the ISR jitter is _still_ 6us! So there must be other places disabling global interrupts? (MSR.EE) Is that possible?
(I'm not calling the SuspendOSInterrupts function myself, btw)

Anyway, thanks for getting me this far already.
I really appreciate EE, it's so lean & mean!

Re: Minimal ISR jitter possible?

Posted: Tue Sep 03, 2013 6:44 pm
by e.guidieri
maximevince wrote:Wow, that's just great support!
Fast, to-the-point answers!

I have changed my config into the following:
- FP kernel
- Nested IRQs
- Semaphores
- Therefor also Multi-Stack
-Two tasks with full preemptive scheduling.
---One task waiting for the semaphore (high prio)
---One task posting a semaphore every x time (lower prio)
This configuration sounds a little bit weird to me. FP give its best if you are working with RTC (Run to completition, in other words functions that returns) TASKs. Since you receiving Task is at higher priority than posting just activate it from the other one, after populate the communication structures.

In ERIKA TASK are slightly different from usual thread and wen they terminate goes in SUSPENDED state and can be reactivated
- Status = Standard (however I did not see this affect my ISR jitter?)
FP API Critical sections are just too short to be the cause of the jitter.
Then, I tried the trick, overriding EE_hal_xx_nested_primitives.
This does work for a moment, however, after a random amount of time, I end up in an EE_e200zx_instruction_tlb_error.
I think that the PIT interrupt can happen when saving or restoring SRR-registers now, which can mess up the cpu. Is that possible?

Plus, using this technique, the ISR jitter is _still_ 6us! So there must be other places disabling global interrupts? (MSR.EE) Is that possible?
From the other thread I understood that you are using DECREMENTER to do "something", probably to drive a Counter...

The problem is that DECREMENTER is an exception and when interrupts are enabled it preempt every other external interrupt and the kernel too (with the new implementation of EE_hal_xx_nested_primitives), in other words actually decrementer has an higher priority respect any other external interrupt source.
In this case the kernel can be preempted by an ISR2 leaving it in an incoherent status.

Since you are using MPC5643L use the other supported source (STM) to drive your counter.

Bye

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 7:34 am
by maximevince
Hi e.guidieri,

You are right, the decrementer is an exception indeed,
therefor interrupting regular interrupts.
I will change to STM, since I have it available.

About the FP kernel: I there a comparison of the kernel types, and which should be used for what?
I'm using FP as I read that's fastest. Plus, I don't need to terminate tasks or change priorities dynamically.
I will explain the set-up I had in my previous OS, and what I'm trying to acheive in a new thread.

Thanks!

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 9:38 am
by paolo.gai
About the FP kernel: I there a comparison of the kernel types, and which should be used for what?
take a look at the wiki page

http://erika.tuxfamily.org/wiki/index.p ... -_Florence

and in particular to the file: "FLEX boards, difference with the various EE kernels"

- FP is a minimal implemenattion of a realtime scheduler "similar" to OSEK/VDX. tou can strip it down to 800-900 bytes :-)
- EDF implements EDF using a timer with wraparound
- FRSH implements the IRIS scheduling algorithm [*]
- there is another implementation of a reservation scheduler pending in the Hard Disk of Alessandro Biondi to be committed... let's hope that he will submit something anytime soon... (I hope he will read this message :-) )

Ciao,

PJ

[*] L. Marzario, G. Lipari, P. Balbastre, A. Crespo, IRIS: A new reclaiming algorithm for server-based real-time systems, Proceedings of the 10th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS’04)
downloadable at https://www.google.it/url?sa=t&rct=j&q= ... 5398,d.Yms

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 1:18 pm
by maximevince
Hey Paolo,

I have removed the decrementer exception.
That removes the jitter completely(!), using the EE_hal_xx_nested_primitives modifications you suggested.
However, it still crashes after some random time, into this EE_e200zx_instruction_tlb_error.

If I could get this right, the solution would just be awesome. Exactly what I need for my high prio ISRs.

Could you elaborate a little on where EE_hal_begin_nested_primitive and EE_hal_end_nested_primitive is called from?
Is it ever called from ISRs, where this ISR expects "protection" from this call?
Also, when could save_registers or restore_registers be harmfull?

Right now, there is a reason that entering an ISR after EE_hal_begin_nested_primitive() has been called, can result in a fatal exception in the cpu...

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 2:01 pm
by e.guidieri
To be honest this problem sounds like an Application problem to me, it seems that your code access a pointer dirty or nulled: are you sure if you restore the old behaviour of EE_hal_xx_nested_primitives you won't get it anymore?

How do you protect data shared between your ISR1 and TASKs ?

Could you post some code?

Bye,
Errico Guidieri

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 3:15 pm
by maximevince
Errico,

If I restore the old behviour of EE_hal_xx_nested_primitives , everything is fine indeed.
My application is _really_ basic at this moment. Just trying out basic OS features.

I can post the code below: (a little simplified for readibility)

Code: Select all

SemType s;

/* Let's declare the tasks identifiers */
DeclareTask(Task1);
DeclareTask(Task2);
DeclareTask(Task3);

/* some other prototypes */
void mydelay(long int end);
void led_blink(unsigned char theled);

void mydelay(long int end)
{
  	volatile long int i;
  	for (i=0; i<end; i++);
  	return;  
}

void led_blink(unsigned char theled)
{
  led_status |= theled;
  EE_leds(led_status);
  mydelay((long int)300000);
  led_status &= ~theled;
  EE_leds(led_status);
}

/* Task1: just call the ChristmasTree */
TASK(Task1)
{
	while (TRUE)
	{
	  task1_fired++;

	  PostSem(&s);

	  /* First half of the christmas tree */
	  led_blink(LED_0);
	  led_blink(LED_1);
	  led_blink(LED_2);
	  led_blink(LED_3);

	  Schedule(); //TerminateTask();
	}
}

/* Task2: Print the counters on the JTAG UART */
TASK(Task2)
{
	while (TRUE)
	{
	  WaitSem(&s);

	  /* count the number of Task2 activations */
	  task2_fired++;
	}
}

/* Task3: Increment counter */
TASK(Task3)
{
	task_3++;
}

ISR2(Counter_Interrupt)
{
	STM.CNT.R = 1u;
	STM.CIR0.B.CIF = 1u;

	t_uptime++;
}

static void setup_interrupts(void)
{
  EE_e200z7_enableIRQ();
}

void STMInit(void)
{
	STM.CR.B.FRZ = 1u;
	STM.CMP0.R = 0x0001D4C0u;
	STM.CCR0.B.CEN = 1u;
	STM.CR.B.TEN = 1u;
}

/* called as _first_ function of a primitive that can be called in
   an IRQ and in a task */
EE_FREG  EE_hal_begin_nested_primitive(void)
{
	/* vnz0r */
//	return EE_hal_suspendIRQ();

	EE_TYPEISR2PRIO prev_prio = EE_e200zx_get_int_prio();
	EE_e200zx_set_int_prio(14U);
	return (EE_FREG)prev_prio;
}


/* Called as _last_ function of a primitive that can be called in
   an IRQ and in a task.  Enable IRQs if they were enabled before entering. */
void  EE_hal_end_nested_primitive(EE_FREG f)
{
//	EE_hal_resumeIRQ(f);

	EE_e200zx_set_int_prio( (EE_TYPEISR2PRIO)f );
}


int main(void)
{
  WD_DisableSoftwareWatchdog();
  PLL_InitializeModesAndClock();
  PLL_initOutputClock();

  PIT_Init();
  PIT_SetState(ePIT_3, TRUE);

  STMInit();

  /* Init devices */
  EE_buttons_init();
  
  /* Init leds */
  EE_leds_init();
  
  setup_interrupts();
  
  /* let's start the multiprogramming environment...*/
//  StartOS(OSDEFAULTAPPMODE);

  InitSem(s,0);

  ActivateTask(Task2);
  ActivateTask(Task1);

  /* now the background activities... */
  for (;;);
  
  return 0;
}
The PIT3 ISR is an interrupt being fired @ 9,xx kHz, and that just toggles a GPIO pin, on which I measure the ISR jitter.

Re: Minimal ISR jitter possible?

Posted: Wed Sep 04, 2013 4:30 pm
by e.guidieri

Code: Select all

SemType s;

/* Let's declare the tasks identifiers */
DeclareTask(Task1);
DeclareTask(Task2);
DeclareTask(Task3);

/* some other prototypes */
void mydelay(long int end);
void led_blink(unsigned char theled);

void mydelay(long int end)
{
     volatile long int i;
     for (i=0; i<end; i++);
     return;  
}

void led_blink(unsigned char theled)
{
  led_status |= theled;
  EE_leds(led_status);
  mydelay((long int)300000);
  led_status &= ~theled;
  EE_leds(led_status);
}

/* Task1: just call the ChristmasTree */
TASK(Task1)
{
   while (TRUE)
   {
     task1_fired++;

     /* PostSem(&s); */ /* Use ActivateTask(Task2) */

     /* First half of the christmas tree */
     led_blink(LED_0);
     led_blink(LED_1);
     led_blink(LED_2);
     led_blink(LED_3);

     /* Schedule(); //TerminateTask(); */ /* EG: If you are not using resources this is useless */
     /* EG: Add some kind of delay or, better, return and reactive it with an alarm */
   }
}

/* Task2: Print the counters on the JTAG UART */
TASK(Task2)
{
   /* EG: Use Task Activation instead */
   /* while (TRUE) {
     WaitSem(&s); */

     /* count the number of Task2 activations */
     task2_fired++;
  /* } */
}

/* Task3: Increment counter */
TASK(Task3)
{
   task_3++;
}

ISR2(Counter_Interrupt)
{
   STM.CNT.R = 1u;
   STM.CIR0.B.CIF = 1u;

   t_uptime++;
}

static void setup_interrupts(void)
{
  EE_e200z7_enableIRQ();
}

void STMInit(void)
{
   STM.CR.B.FRZ = 1u;
   STM.CMP0.R = 0x0001D4C0u;
   STM.CCR0.B.CEN = 1u;
   STM.CR.B.TEN = 1u;
}

/* EG: Did you commented the code in .../e200zx/inc/ee_internal.h? Otherwise
   the following won't be used... */

/* called as _first_ function of a primitive that can be called in
   an IRQ and in a task */
EE_FREG  EE_hal_begin_nested_primitive(void)
{
   /* vnz0r */
//   return EE_hal_suspendIRQ();

   EE_TYPEISR2PRIO prev_prio = EE_e200zx_get_int_prio();
   EE_e200zx_set_int_prio(14U);
   return (EE_FREG)prev_prio;
}


/* Called as _last_ function of a primitive that can be called in
   an IRQ and in a task.  Enable IRQs if they were enabled before entering. */
void  EE_hal_end_nested_primitive(EE_FREG f)
{
//   EE_hal_resumeIRQ(f);

   EE_e200zx_set_int_prio( (EE_TYPEISR2PRIO)f );
}


int main(void)
{
  WD_DisableSoftwareWatchdog();
  PLL_InitializeModesAndClock();
  PLL_initOutputClock();

  PIT_Init();
  PIT_SetState(ePIT_3, TRUE);

  STMInit();

  /* Init devices */
  EE_buttons_init();
  
  /* Init leds */
  EE_leds_init();
  
  setup_interrupts();
  
  /* let's start the multiprogramming environment...*/
//  StartOS(OSDEFAULTAPPMODE);

  /* InitSem(s,0); EG: Try to no use semaphores with a fixed priority scheduler
     usually are useless */

  /* ActivateTask(Task2); EG: Active it from Task1 */
  ActivateTask(Task1);

  /* now the background activities... */
  for (;;);
  
  return 0;
}
I checked your code and made some comments (You can find them searching EG: string).

I found something missing in my EE_xx_nested_primitive approach, you need to modify $ee/pkg/cpu/common/src/ee_context.c file in the following way:

Code: Select all


EE_TID EE_std_run_task_code(EE_TID tid)
{
    /* EE_hal_enableIRQ(); EG: Removed */
    EE_e200zx_set_int_prio(0U); /* EG: added */
    EE_call_task_body(tid); /* Call the task body */
    EE_e200zx_set_int_prio(14U); /* EG: added */
    /* EE_hal_disableIRQ(); EG: removed */
    EE_thread_end_instance(); /* Call the scheduler */
    return EE_std_endcycle_next_tid;
}
To make me happy and more confident, could you try this after:

a) Removing Semaphores from your application.
2) Using monostack only.

Thank You,
Errico Guidieri.

Re: Minimal ISR jitter possible?

Posted: Thu Sep 05, 2013 8:05 am
by maximevince
Hey Errico,

Thanks again for the suggestion.
Unfortunately, I have no access to the hardware the coming few days, Monday I'll be able to experiment again.
I'll keep you posted!

Best regards,
Maxime

Re: Minimal ISR jitter possible?

Posted: Mon Sep 09, 2013 10:43 am
by maximevince
Errico,

I have:
- Reverted back to monostack mode
- Removed the semaphores, used ActivateTask() instead.
It looks like it works with the modifications to ee_context.c you gave me!
I have jitter almost never now. When it happens, it's about 3us.

I'm now going to set up our more complex task triggering algorithm inside this set-up.
However, I would like to have the multi-stack feature again, if possible.
I have tried running it, but then I got EE_e200zx_instruction_tlb_error exceptions.
They are caused when saving and restoring the stack. (I think)
I'm going to try by adding an interrupt disable around EE_std_change_context. See if that helps.

Thanks a lot!

Re: Minimal ISR jitter possible?

Posted: Mon Sep 09, 2013 12:59 pm
by e.guidieri
Hi maxevince,

happy to have been helpful.
maximevince wrote: - Removed the semaphores, used ActivateTask() instead.
I would suggest to keep this method in any case: understand how to work with RTC (Run To Completition), keeps the things faster, cleaner, smaller and easier.
It looks like it works with the modifications to ee_context.c you gave me!
I have jitter almost never now. When it happens, it's about 3us.
Probably this is tied to the small critical section needed by ISR2. Since you already have some understanding of ERIKA's internal I could suggest you a few thing to try to reduce this even more, but I suggest you to do that after understood what's happening in multistack environments.
I'm now going to set up our more complex task triggering algorithm inside this set-up.
However, I would like to have the multi-stack feature again, if possible. I have tried running it, but then I got EE_e200zx_instruction_tlb_error exceptions.
They are caused when saving and restoring the stack. (I think) I'm going to try by adding an interrupt disable around EE_std_change_context. See if that helps.
Obviously you are right, we should understand what's going on on that: could you trace someway code execution? At least can you crawl stack/resgisters to find witch instruction generate the exception?

Thanks a lot!
You are welcome