Monday, May 3, 2021

Linux: Why do we need an executable stack with nested functions in GCC

 

  • The stack needs to be writable because the trampoline code is written on the stack which needs to be executed (to jump to the nested function). 
A nested function has no linkage by itself. So the trampoline code actually ensures two things:
  1. The outer () functions stack frame is available in the nested function
  2. Jump and execute code from the nested function.
Some other notes:
  • Nested functions are not a part of ANSI C, however, they are part of Gnu C.
  • The only reason nested functions are possibly useful because they use common stack variables.