Friday, August 30, 2019

Which Linux Kernel timing or delay APIs to use for what?

I classify the waiting or timing API in the Linux kernel in two categories:
1. Which blocks the current thread of execution.
2. Something which has to be scheduled for later but we want the current thread to continue.
In most cases, the distinction between 1 and 2 is clear, but the techniques used to implement 2 can also be manipulated to behave like 1.


1. Blocking current thread of execution (Inline delays)
The API for 1. in the above case are:

//option A. Busy waiting - long delays
while (time_before(jiffies, j1))
    cpu_relax(  );

//option B. Completely yielding the processor
while (time_before(jiffies, j1))
    schedule(  );

// timeouts    
#include <linux/wait.h>
long wait_event_timeout(wait_queue_head_t q, condition, long timeout);
long wait_event_interruptible_timeout(wait_queue_head_t q,
                      condition, long timeout);
                          
// Shorter delays - busy waiting (worst case use)                          
#include <linux/delay.h>
void ndelay(unsigned long nsecs);
void udelay(unsigned long usecs);
void mdelay(unsigned long msecs);

// Short delays - without busy waiting
void msleep(unsigned int millisecs);
unsigned long msleep_interruptible(unsigned int millisecs);
void ssleep(unsigned int seconds)

2. Non-blocking deferred execution of code.
The API for 2, are only through the Linux kernel timers run on the same CPU that register them unless mentioned exclusively.
add_timer();
add_timer_on();

In my note I say that timers (2) can be used to implement logic similar to (1). But it is just a matter of convenience and ease in choosing a
certain technique.

Some other important factors that contribute to choosing one of the above techniques are:
1. SMP safeness - timers will usually run on the code they are scheduled. Waiting is in the thread so this is not an issue.
2. Locking - timer code runs in soft-irq context, so appropriate locking may be needed. This is usually not needed for just blocking the current thread.
3. Current context - Certain threads of execution cant be blocked. For e.g. if you are running in the kernel interrupt context, you cannot sleep or yield. It is better
to schedule a timer to do something later. If you see a crash because of your code that says Sleeping while atomic, it is probably because you used incorrect API which sleeps
when you are in interrupt context and you need to use a different mechanism to delay execution.