Thursday, June 4, 2020

Linux: Why is skb recycling done

Why is this done?
* Saves the cost of allocating and de-allocating memory repeatedly.
* Savings are significant because this is a very frequent operation (usually skb alloc and de-alloc is done on a per-packet basis).

Recent changes to SKB recycling:

"- Make skb recycling available to all drivers, without needing driver
  modifications.

- Allow recycling skbuffs in more cases, by having the recycle check
  in __kfree_skb() instead of in the ethernet driver transmit
  completion routine.  This also allows for example recycling locally
  destined skbuffs, instead of only recycling forwarded skbuffs as
  the transmit completion-time check does.

- Allow more consumers of skbuffs in the system use recycled skbuffs,
  and not just the rx refill process in the driver.
- Having a per-interface recycle list doesn't allow skb recycling when
  you're e.g. unidirectionally routing from eth0 to eth1, as eth1 will
  be producing a lot of recycled skbuffs but eth0 won't have any skbuffs
  to allocate from its recycle list."

Note: Generic skb recycling is slightly slower than doing it in the driver.

Custom implementation

If you need to implement SKB recycling for your kernel module you could use this approach.

SKB recycling can be implemented in a simple fashion. The way to do it would be through implementing and using wrapper functions for:
* dev_alloc_skb() and 
* kfree(skb)

Say for example, you can implement a custom_dev_alloc_skb() and custom_kfree() which is called by your programs. You can maintain a simple array of skb pointers and allocate and de-allocate skbs from this pool