Skip to content

Instantly share code, notes, and snippets.

@simonhf

simonhf/_libarchive-read-blocking-part-3.md

Last active November 7, 2019 22:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save simonhf/d38277d70f19a0ba92887e74486664b1 to your computer and use it in GitHub Desktop.
Save simonhf/d38277d70f19a0ba92887e74486664b1 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
_libarchive-read-blocking-part-3.md

Experiments with libarchive read blocking: Part 3

  • Note: This is a continuation from Part 2 [1].

[1] Experiments with libarchive read blocking: Part 2

What if we used coroutines to swap between different libarchive instances in the same thread?

  • Possible advantage: We might not need to modify libarchive itself!

How do coroutines work and which libraries implement them?

  • What is a coroutine? [1]
  • What are the implementations for C? [2]
  • So in C, to implement a coroutine there are 2 steps:
  • Step 1: Obtain a second call stack; "In order to implement general-purpose coroutines, a second call stack must be obtained, which is a feature not directly supported by the C language."
  • The problem with this step is "A reliable (albeit platform-specific) way to achieve this is to use a small amount of inline assembly to explicitly manipulate the stack pointer during initial creation of the coroutine."
  • This means all implementations are likely platform specific because they all use platform specific assembly language to manipulate the stack.
  • Step 2: Use setjmp() and longjmp() functions; "Once a second call stack has been obtained with one of the methods listed above, the setjmp and longjmp functions in the standard C library can then be used to implement the switches between coroutines."

[1] Coroutines [2] Coroutines: C Implementations

Examining corountine implementations for C

  • I'm interested in a portable coroutine implementation for C.
  • Preferrably -- for my purposes -- the implentation works on x86_64 and aarch64 Linux.
  • Ideally there should be no assembly language, i.e. no assembly language to support if it needs debugging later :-)
  • Ideally 'portable' does not mean a larger #if .. #else .. #endif block for each supported platform, i.e. less complicated debugging later :-)
  • I examine the implementations listed in [1].
  • In the process of examining implementations, I came acrosss this interesting paper [2] regarding 'The Signal Stack Trick' for creating new stacks via the POSIX signalstack() function.
  • Portable Coroutine Library does not look very portable in its source code [3].
  • libcoroutine contains assembly language source code [4].
  • libconcurrency works on Windows (not Linux?) via fibers [5].
  • libcoro contains assembly language source code for some implementations on the same platform, but also seems like an overwhelming melting pot of different implementations [6].
  • libdill has issues compiling on ARM and goes way beyond prividing an API for coroutines, perhaps being more of a 'Golang for C' [7]?
  • libaco has no support for ARM yet [8].
  • libco contains assembly language source code [9].
  • There seem to be no implementations for C which are obvious candidates to try with libarchive with my selection criteria :-(

[1] Implementations for C [2] PDF: Portable Multithreading: The Signal Stack Trick For User-Space Thread Creation [3] Portable Coroutine Library [4] libcoroutine [5] libconcurrency [6] libcoro [7] libdill [8] libaco [9] libco

How to create a platform agnostic coroutine in C?

  • One technique (proposed here [1]) is to not use different stacks, and therefore assembly language is not required to manipulate the stack.
  • This is achieved by pre-allocating all the stack space for coroutines using C (instead of assembly language) on the regular stack.
  • It works as follows:
  • The main coroutine / the regular thread of execution already has its own stack, and this can be configured to be larger if need be.
  • At run-time, the main coroutine uses alloca() to reserve an arbitrary amount of local stack space for each coroutine used.
  • Note: The space allocated by alloca() is never used by the main coroutine which created it.
  • The newly created corountine uses the space reserved by alloca() as its stack.
  • This means you must be very careful to reserve more stack space than the coroutine ever needs, otherwise stack collision / corruption will occur.
  • In theory this technique should work on any platform (because alloca() is on all platforms) regardless of whether the stack goes up or down.

[1] Coroutines in less than 20 lines of standard C

Experimenting with the proposed technique

  • It was very difficult to get this code [1] working.
  • The code likes to seg fault a lot, and it's not hardened to prevent and/or show why failures occur.
  • So I created a longer but more instrumented version of the code, which serves for both comprehension and sanity checking at run-time; cogo.c.
  • I learned the following by hacking the code:
  • Often the code runs to the end without a seg fault and appears to work, but it has subtly not worked.
  • For example, I had a case where there was no seg fault but piping stdout no longer worked, although there were no errors.
  • Another example, is passing control to another coroutine, but the wrong coroutine.
  • Another example, is passing control to another coroutine, but the stack is wrong, but not necessarily causing a seg fault.
  • The code is only consistently reliable if there are no statements between setjmp() and return, e.g. if (setjmp(here)) { /* unreliable if code here! */ return; }.
  • I could never get any gcc optimized build to work consistently and reliably over time. Which kind of makes sense.
  • Therefore, I made the compile fail if the gcc command line for cogo.c does not use -O0, i.e. no optimization.
  • Interestingly, another way to force -O0 is via #pragma GCC optimize ("O0") but for me this did not consistently avoid seg faults :-(

[1] Coroutines in less than 20 lines of standard C

Example output

  • The following output shows instrumentation which enables easier comprehension of what is happening at run-time.
  • The code is not tested on platforms with a stack in the opposite direction, but in theory it shouldn't take much to get it working.
  • We use Linux calls to get the size of the total size of the stack.
  • The code has a bunch of calls to `co_sanity_check()' which checks that the stack pointer is in the correct range for the current coroutine ID.
  • With the sanity checks, and compiling without optimization, then the code appears to 'just work' as expected :-)
  • Note: The code attempts to avoid outputting pointers as much as possible.
  • Note: The data of the coroutines is kept in a simple array which can be indexed by the corountine ID.
$ truncate --size=0 cogo.log ; gcc -O0 -fstack-protector-all -o cogo cogo.c && ./cogo | tee cogo.log ; ll cogo.log
- 0: Algorithm:
- 0: - Coroutine 0 launches and passes control to up to n new corountines, each of which has a fixed stack size.
- 0: - Each created coroutine enters an endless loop, which transfers control to a random other coroutine.
- 0: - If control is randomly passed to coroutine 0, a new coroutine is launched, or process exit if all launched
- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html
- 0: - Note: Stack space for each new coroutine is reserved using alloca().
- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.
- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.
- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().
- 0: sizeof(jmp_buf)=200
- 0: co_stack_direction_get(stack_addr_caller=0x7fffe47f5c78) {} = -1 AKA down // stack_addr_callee=0x7fffe47f5c28 diff=-80
- 0: co_stack_size_get() {} = 8,388,608 bytes downward @ stack_addr_main=0x7fffe47f5cac
- 0: launching 3 corountines in addition to coroutine 0
- 0: co_launch_internal(0 -> 1) // co_stack_corountine_partitions=1 size_of_all_partitions_so_far=1*10,000=10,000 co_stack_addr_main@-196 end1@-10,196 end2@-20,196
- 0: co_launch_internal(0 -> 1) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 1) // thread_main() AKA co_main()
- 1: co_sanity_check() // co_stack_addr_main@-10,256 end1@-10,196 end2@-20,196
- 1: co_main() // jumping to random corountine 1 of main+1 so far
- 1: co_switch_internal(1 -> 1) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 1 of main+1 so far
- 1: co_switch_internal(1 -> 1) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 0 of main+1 so far
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: co_launch_internal(0 -> 2) // co_stack_corountine_partitions=2 size_of_all_partitions_so_far=2*10,000=20,000 co_stack_addr_main@-196 end1@-20,196 end2@-30,196
- 0: co_launch_internal(0 -> 2) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 2) // thread_main() AKA co_main()
- 2: co_sanity_check() // co_stack_addr_main@-20,256 end1@-20,196 end2@-30,196
- 2: co_main() // jumping to random corountine 2 of main+2 so far
- 2: co_switch_internal(2 -> 2) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,336 end1@-20,196 end2@-30,196
- 2: co_main() // co_switch() returned!
- 2: co_main() // jumping to random corountine 1 of main+2 so far
- 2: co_switch_internal(2 -> 1) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 1 of main+2 so far
- 1: co_switch_internal(1 -> 1) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 0 of main+2 so far
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: co_launch_internal(0 -> 3) // co_stack_corountine_partitions=3 size_of_all_partitions_so_far=3*10,000=30,000 co_stack_addr_main@-196 end1@-30,196 end2@-40,196
- 0: co_launch_internal(0 -> 3) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 3) // thread_main() AKA co_main()
- 3: co_sanity_check() // co_stack_addr_main@-30,256 end1@-30,196 end2@-40,196
- 3: co_main() // jumping to random corountine 3 of main+3 so far
- 3: co_switch_internal(3 -> 3) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 3) // longjmp(3)
... voodoo ...
- 3: co_switch() continued via jump point b
- 3: co_sanity_check() // co_stack_addr_main@-30,336 end1@-30,196 end2@-40,196
- 3: co_main() // co_switch() returned!
- 3: co_main() // jumping to random corountine 0 of main+3 so far
- 3: co_switch_internal(3 -> 0) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: launched all corountines
-rw-rw-r-- 1 simon simon 5374 Nov  4 17:10 cogo.log

Modify example code to pre-launch coroutines

  • Modify the algorithm so that all coroutine stack space is initially pre-allocated.
  • We do this by pre-launching all coroutines, but not passing control to them.
  • After pre-launching, coroutine 0 continues in sub_main() and only returns when the process intends to exit, and all stack space allocated for pre-launched coroutines gets destroyed.
  • This has the advantage that coroutine 0 can continue to use the rest of the stack without 'bumping into' stack space allocated for pre-launched coroutines.
  • Hopefully in this way, a monolithic legacy process using libarchive can continue to run in coroutine 0 without worrying how much stack space it uses.
  • But it can also pre-launch n coroutines with fixed, pre-determined stack sizes for co-operative operation, such as, the libarchive read callback using co_switch() to produce a result similar to an async read?
  • The disadvantage of this method is that the maximum number of coroutines must be known at process start, since their 'stacks' are permanently allocated on the regular stack until the end of the process:
+ main() {       <- stack top
                 <- allocate stack bytes for coroutine 1
  ...
                 <- allocate stack bytes for coroutine 1+n
  + sub_main() { <- regular stack continues...
    ...          <- regular code  continues...
    }
  }              <- coroutine stacks destroyed here

$ cp cogo.c cogo-2-pre-launch.c
$ # hack code for pre-launching coroutines!
$ truncate --size=0 cogo-2-pre-launch.log ; gcc -O0 -fstack-protector-all -o cogo-2-pre-launch cogo-2-pre-launch.c && ./cogo-2-pre-launch | tee cogo-2-pre-launch.log ; ll cogo-2-pre-launch.log
- 0: Algorithm:
- 0: - Coroutine 0 pre-launches n new corountines, without passing control to them, each of which has a fixed stack size.
- 0: - After pre-launch coroutine 0 continues in sub_main() which becomes the new main()
- 0: - sub_main() switches control to coroutine 1.
- 0: - Each created coroutine enters an endless loop, which transfers control to a random other coroutine.
- 0: - If control is randomly passed to coroutine 0, the process exits
- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html
- 0: - Note: Stack space for each new coroutine is reserved using alloca().
- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.
- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.
- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().
- 0: sizeof(jmp_buf)=200
- 0: co_stack_direction_get(stack_addr_caller=0x7ffd6ece9eb8) {} = -1 AKA down // stack_addr_callee=0x7ffd6ece9e68 diff=-80
- 0: co_stack_size_get() {} = 8,388,608 bytes downward @ stack_addr_main=0x7ffd6ece9eec
- 0: launching 3 corountines in addition to coroutine 0
- 0: co_launch_internal(0 -> 1) // co_stack_corountine_partitions=1 size_of_all_partitions_so_far=1*10,000=10,000 co_stack_addr_main@-196 end1@-10,196 end2@-20,196
- 0: co_launch_internal(0 -> 1) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 1) // thread_main() AKA co_main()
- 1: co_sanity_check() // co_stack_addr_main@-10,256 end1@-10,196 end2@-20,196
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: co_launch_internal(0 -> 2) // co_stack_corountine_partitions=2 size_of_all_partitions_so_far=2*10,000=20,000 co_stack_addr_main@-196 end1@-20,196 end2@-30,196
- 0: co_launch_internal(0 -> 2) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 2) // thread_main() AKA co_main()
- 2: co_sanity_check() // co_stack_addr_main@-20,256 end1@-20,196 end2@-30,196
- 2: co_switch_internal(2 -> 0) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: co_launch_internal(0 -> 3) // co_stack_corountine_partitions=3 size_of_all_partitions_so_far=3*10,000=30,000 co_stack_addr_main@-196 end1@-30,196 end2@-40,196
- 0: co_launch_internal(0 -> 3) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 3) // thread_main() AKA co_main()
- 3: co_sanity_check() // co_stack_addr_main@-30,256 end1@-30,196 end2@-40,196
- 3: co_switch_internal(3 -> 0) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: launched all corountines
- 0: co_launch_internal(0 -> 0) // co_stack_corountine_partitions=4 size_of_all_partitions_so_far=4*10,000=40,000 co_stack_addr_main@-196 end1@-0 end2@-8,388,608 <-- coroutine 0 stack skips past all others!
- 0: co_launch_internal(0 -> 0) // thread_main() AKA sub_main()
- 0: co_sanity_check() // co_stack_addr_main@-40,256 end1@-0 end2@-8,388,608
- 0: sub_main() // continuing main() with stack position skipped past launched coroutine stacks
- 0: co_sanity_check() // co_stack_addr_main@-40,288 end1@-0 end2@-8,388,608
- 0: co_switch_internal(0 -> 1) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // jumping to random corountine 1 of main+3 so far
- 1: co_switch_internal(1 -> 1) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 3 of main+3 so far
- 1: co_switch_internal(1 -> 3) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 3) // longjmp(3)
... voodoo ...
- 3: co_switch() continued via jump point b
- 3: co_sanity_check() // co_stack_addr_main@-30,336 end1@-30,196 end2@-40,196
- 3: co_main() // jumping to random corountine 2 of main+3 so far
- 3: co_switch_internal(3 -> 2) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,336 end1@-20,196 end2@-30,196
- 2: co_main() // jumping to random corountine 2 of main+3 so far
- 2: co_switch_internal(2 -> 2) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,336 end1@-20,196 end2@-30,196
- 2: co_main() // co_switch() returned!
- 2: co_main() // jumping to random corountine 3 of main+3 so far
- 2: co_switch_internal(2 -> 3) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 3) // longjmp(3)
... voodoo ...
- 3: co_switch() continued via jump point b
- 3: co_sanity_check() // co_stack_addr_main@-30,336 end1@-30,196 end2@-40,196
- 3: co_main() // co_switch() returned!
- 3: co_main() // jumping to random corountine 1 of main+3 so far
- 3: co_switch_internal(3 -> 1) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 1 of main+3 so far
- 1: co_switch_internal(1 -> 1) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: co_main() // co_switch() returned!
- 1: co_main() // jumping to random corountine 3 of main+3 so far
- 1: co_switch_internal(1 -> 3) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 3) // longjmp(3)
... voodoo ...
- 3: co_switch() continued via jump point b
- 3: co_sanity_check() // co_stack_addr_main@-30,336 end1@-30,196 end2@-40,196
- 3: co_main() // co_switch() returned!
- 3: co_main() // jumping to random corountine 0 of main+3 so far
- 3: co_switch_internal(3 -> 0) // setjmp(3) for jump point b
- 3: co_switch_internal(3 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-40,336 end1@-0 end2@-8,388,608
- 0: sub_main() finishing
- 0: co_launch_internal() // coroutine 0 exiting; process shutting down shortly?
- 0: back from sub_main(); process exiting
-rw-rw-r-- 1 simon simon 7133 Nov  5 17:45 cogo-2-pre-launch.log

Modify example code to work with dummy libarchive with dummy read callback

  • Modify the code so that a dummy libarchive with dummy blocking read callback is called.
  • Make a dummy libarchive read function which has its own dummy read callback, and needs n dummy read callbacks to complete.
  • Call the dummy read functions from 2 coroutines and design the callbacks to only operate in a special certain order, simulating blocking reads.
$ cp cogo-2-pre-launch.c cogo-3-dummy-libarchive.c
$ # hack code for dummy libarchive!
$ truncate --size=0 cogo-3-dummy-libarchive.log ; gcc -O0 -fstack-protector-all -o cogo-3-dummy-libarchive cogo-3-dummy-libarchive.c && ./cogo-3-dummy-libarchive | tee cogo-3-dummy-libarchive.log ; ll cogo-3-dummy-libarchive.log ; cat cogo-3-dummy-libarchive.log | egrep "(instance_init|// order|callback.*instance.*leave)"
- 0: main() // enter
- 0: Algorithm:
- 0: - Coroutine 0 pre-launches n new corountines, without passing control to them, each of which has a fixed stack size.
- 0: - After pre-launch coroutine 0 continues in sub_main() which becomes the new main()
- 0: - sub_main() switches control to each new coroutine one by one, running until the first callback in invoked.
- 0: - Each callback is invoked after a random number of recursive function calls, simulating libarchive stack use.
- 0: - At this point all new coroutines have run until calling their blocking callback.
- 0: - sub_main() determines the order in which the blocking callbacks will continue.
- 0: - sub_main() switches control to each new coroutine in the predetermined order to unblock each callback
- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html modified for pre-launching.
- 0: - Note: Stack space for each new coroutine is reserved using alloca().
- 0: - Note: Stack space for exiting coroutine 0 can grow beyond co_stack_size_per_coroutine.
- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.
- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.
- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().
- 0: sizeof(jmp_buf)=200
- 0: co_stack_direction_get(stack_addr_caller=0x7ffe54615758) {} = -1 AKA down // stack_addr_callee=0x7ffe54615708 diff=-80
- 0: co_stack_size_get() {} = 8,388,608 bytes downward @ stack_addr_main=0x7ffe5461578c
- 0: launching 2 corountines in addition to coroutine 0                                 <-- milestone 1
- 0: co_launch_internal(0 -> 1) // co_stack_corountine_partitions=1 size_of_all_partitions_so_far=1*10,000=10,000 co_stack_addr_main@-196 end1@-10,196 end2@-20,196
- 0: co_launch_internal(0 -> 1) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 1) // thread_main() AKA co_main()
- 1: co_sanity_check() // co_stack_addr_main@-10,256 end1@-10,196 end2@-20,196
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: co_launch_internal(0 -> 2) // co_stack_corountine_partitions=2 size_of_all_partitions_so_far=2*10,000=20,000 co_stack_addr_main@-196 end1@-20,196 end2@-30,196
- 0: co_launch_internal(0 -> 2) // setjmp(0) for jump point a
- 0: co_launch_internal(0 -> 2) // thread_main() AKA co_main()
- 2: co_sanity_check() // co_stack_addr_main@-20,256 end1@-20,196 end2@-30,196
- 2: co_switch_internal(2 -> 0) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 0) // longjmp(0)
... voodoo ...
- 0: co_launch() continued via jump point a
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_launch() returned!
- 0: launched all corountines                                                           <-- milestone 2
- 0: co_launch_internal(0 -> 0) // co_stack_corountine_partitions=3 size_of_all_partitions_so_far=3*10,000=30,000 co_stack_addr_main@-196 end1@-0 end2@-8,388,608 <-- coroutine 0 stack skips past all others!
- 0: co_launch_internal(0 -> 0) // thread_main() AKA sub_main()
- 0: co_sanity_check() // co_stack_addr_main@-30,256 end1@-0 end2@-8,388,608
- 0: sub_main() // enter; continuing main() with stack position skipped past launched coroutine stacks
- 0: co_sanity_check() // co_stack_addr_main@-30,288 end1@-0 end2@-8,388,608
- 0: sub_main() // pass control to coroutines until first callback entered              <-- milestone 3
- 0: sub_main() // pass control to coroutine 1 until first callback entered
- 0: co_switch_internal(0 -> 1) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 1: dummy_libarchive_instance_init()
- 1: dummy_libarchive_random_stack_depth(2) // enter
- 1: dummy_libarchive_random_stack_depth(1) // enter
- 1: dummy_libarchive_random_stack_depth(0) // enter
- 1: dummy_libarchive_callback() // callback 1 for instance 1; enter
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: sub_main() // pass control to coroutine 2 until first callback entered
- 0: co_switch_internal(0 -> 2) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,336 end1@-20,196 end2@-30,196
- 2: dummy_libarchive_instance_init()
- 2: dummy_libarchive_random_stack_depth(1) // enter
- 2: dummy_libarchive_random_stack_depth(0) // enter
- 2: dummy_libarchive_callback() // callback 1 for instance 2; enter
- 2: co_switch_internal(2 -> 0) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: dummy_libarchive_callback_order_init() {} // order 2 1 1 2                         <-- milestone 4
- 0: sub_main() // pass control to coroutine callbacks in desired order                 <-- milestone 5
- 0: sub_main() // pass control to coroutine 2 to complete its callback
- 0: co_switch_internal(0 -> 2) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,464 end1@-20,196 end2@-30,196
- 2: dummy_libarchive_callback() // callback 1 for instance 2; leave
- 2: dummy_libarchive_random_stack_depth(0) // leave
- 2: dummy_libarchive_random_stack_depth(1) // leave
- 2: dummy_libarchive_random_stack_depth(4) // enter
- 2: dummy_libarchive_random_stack_depth(3) // enter
- 2: dummy_libarchive_random_stack_depth(2) // enter
- 2: dummy_libarchive_random_stack_depth(1) // enter
- 2: dummy_libarchive_random_stack_depth(0) // enter
- 2: dummy_libarchive_callback() // callback 2 for instance 2; enter
- 2: co_switch_internal(2 -> 0) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: sub_main() // pass control to coroutine 1 to complete its callback
- 0: co_switch_internal(0 -> 1) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,512 end1@-10,196 end2@-20,196
- 1: dummy_libarchive_callback() // callback 1 for instance 1; leave
- 1: dummy_libarchive_random_stack_depth(0) // leave
- 1: dummy_libarchive_random_stack_depth(1) // leave
- 1: dummy_libarchive_random_stack_depth(2) // leave
- 1: dummy_libarchive_random_stack_depth(3) // enter
- 1: dummy_libarchive_random_stack_depth(2) // enter
- 1: dummy_libarchive_random_stack_depth(1) // enter
- 1: dummy_libarchive_random_stack_depth(0) // enter
- 1: dummy_libarchive_callback() // callback 2 for instance 1; enter
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: sub_main() // pass control to coroutine 1 to complete its callback
- 0: co_switch_internal(0 -> 1) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 1) // longjmp(1)
... voodoo ...
- 1: co_switch() continued via jump point b
- 1: co_sanity_check() // co_stack_addr_main@-10,560 end1@-10,196 end2@-20,196
- 1: dummy_libarchive_callback() // callback 2 for instance 1; leave
- 1: dummy_libarchive_random_stack_depth(0) // leave
- 1: dummy_libarchive_random_stack_depth(1) // leave
- 1: dummy_libarchive_random_stack_depth(2) // leave
- 1: dummy_libarchive_random_stack_depth(3) // leave
- 1: co_switch_internal(1 -> 0) // setjmp(1) for jump point b
- 1: co_switch_internal(1 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: sub_main() // pass control to coroutine 2 to complete its callback
- 0: co_switch_internal(0 -> 2) // setjmp(0) for jump point b
- 0: co_switch_internal(0 -> 2) // longjmp(2)
... voodoo ...
- 2: co_switch() continued via jump point b
- 2: co_sanity_check() // co_stack_addr_main@-20,608 end1@-20,196 end2@-30,196
- 2: dummy_libarchive_callback() // callback 2 for instance 2; leave
- 2: dummy_libarchive_random_stack_depth(0) // leave
- 2: dummy_libarchive_random_stack_depth(1) // leave
- 2: dummy_libarchive_random_stack_depth(2) // leave
- 2: dummy_libarchive_random_stack_depth(3) // leave
- 2: dummy_libarchive_random_stack_depth(4) // leave
- 2: co_switch_internal(2 -> 0) // setjmp(2) for jump point b
- 2: co_switch_internal(2 -> 0) // longjmp(0)
... voodoo ...
- 0: co_switch() continued via jump point b
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 0: sub_main() // leave
- 0: co_launch_internal() // coroutine 0 exiting; process shutting down shortly?
- 0: main() // leave
-rw-rw-r-- 1 simon simon 9416 Nov  6 17:32 cogo-3-dummy-libarchive.log
  • Here we can see the in grep output:
  • The the dummy libarchive code is called in coroutine ID order until the first dummy blocking read callback.
  • The test program determines the random order that the dummy blocking read callbacks will be unblocked.
  • Control in passed to the dummy blocking read callbacks in the predetermined order.
$ cat cogo-3-dummy-libarchive.log | egrep "(instance_init|// order|callback.*instance.*leave)
- 1: dummy_libarchive_instance_init()
- 2: dummy_libarchive_instance_init()
- 0: dummy_libarchive_callback_order_init() {} // order 2 1 1 2                         <-- milestone 4
- 2: dummy_libarchive_callback() // callback 1 for instance 2; leave
- 1: dummy_libarchive_callback() // callback 1 for instance 1; leave
- 1: dummy_libarchive_callback() // callback 2 for instance 1; leave
- 2: dummy_libarchive_callback() // callback 2 for instance 2; leave

Testing cross platform prototype on ARM to ensure that it works

  • All seems to 'just work' according to the instrumentation output below.
  • Same number of bytes in output log.
  • Same order of callbacks and call depth due to predetermined random order due to rand() with same srand() each process start.
  • Same order of co_sanity_check() calls.
  • Note: Both platforms have 'down' stacks. I would not trust the current code with an 'up' stack platform and want to test it some more...
$ # find an ARM box ... :)
$ lscpu | egrep Arch
Architecture:        aarch64
$ truncate --size=0 cogo-3-dummy-libarchive.log ; gcc -O0 -fstack-protector-all -o cogo-3-dummy-libarchive cogo-3-dummy-libarchive.c && ./cogo-3-dummy-libarchive | tee cogo-3-dummy-libarchive.log ; ll cogo-3-dummy-libarchive.log ; cat cogo-3-dummy-libarchive.log | egrep "(instance_init|// order|callback.*instance.*leave)"
...
- 0: main() // leave
-rw-rw-r-- 1 simon simon 9416 Nov  7 22:06 cogo-3-dummy-libarchive.log
- 1: dummy_libarchive_instance_init()
- 2: dummy_libarchive_instance_init()
- 0: dummy_libarchive_callback_order_init() {} // order 2 1 1 2                         <-- milestone 4
- 2: dummy_libarchive_callback() // callback 1 for instance 2; leave
- 1: dummy_libarchive_callback() // callback 1 for instance 1; leave
- 1: dummy_libarchive_callback() // callback 2 for instance 1; leave
- 2: dummy_libarchive_callback() // callback 2 for instance 2; leave
  • We can see that the stack pointer addresses only vary slightly from platform to platform:
$ lscpu | egrep Arch ; cat cogo-3-dummy-libarchive.log | egrep co_sanity_check
Architecture:          x86_64
- 1: co_sanity_check() // co_stack_addr_main@-10,256 end1@-10,196 end2@-20,196
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,256 end1@-20,196 end2@-30,196
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_sanity_check() // co_stack_addr_main@-30,256 end1@-0 end2@-8,388,608
- 0: co_sanity_check() // co_stack_addr_main@-30,288 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,336 end1@-10,196 end2@-20,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,336 end1@-20,196 end2@-30,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,464 end1@-20,196 end2@-30,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,512 end1@-10,196 end2@-20,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,560 end1@-10,196 end2@-20,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,608 end1@-20,196 end2@-30,196
- 0: co_sanity_check() // co_stack_addr_main@-30,336 end1@-0 end2@-8,388,608

$ lscpu | egrep Arch ; cat cogo-3-dummy-libarchive.log | egrep co_sanity_check
Architecture:        aarch64
- 1: co_sanity_check() // co_stack_addr_main@-10,288 end1@-10,164 end2@-20,164
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,288 end1@-20,164 end2@-30,164
- 0: co_sanity_check() // co_stack_addr_main@-112 end1@-0 end2@-8,388,608
- 0: co_sanity_check() // co_stack_addr_main@-30,288 end1@-0 end2@-8,388,608
- 0: co_sanity_check() // co_stack_addr_main@-30,320 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,368 end1@-10,164 end2@-20,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,368 end1@-20,164 end2@-30,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,496 end1@-20,164 end2@-30,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,544 end1@-10,164 end2@-20,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608
- 1: co_sanity_check() // co_stack_addr_main@-10,592 end1@-10,164 end2@-20,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608
- 2: co_sanity_check() // co_stack_addr_main@-20,640 end1@-20,164 end2@-30,164
- 0: co_sanity_check() // co_stack_addr_main@-30,368 end1@-0 end2@-8,388,608

Try with real libarchive instead of dummy libarchive

  • Coming in part 4 !
Raw
cogo-2-pre-launch.c
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <bsd/stdlib.h> // sudo apt-get install libbsd-dev
#include <setjmp.h>
#include <sys/resource.h>
#include <alloca.h>
#include <locale.h>
// $ truncate --size=0 cogo-2-pre-launch.log ; gcc -O0 -fstack-protector-all -o cogo-2-pre-launch cogo-2-pre-launch.c && ./cogo-2-pre-launch | tee cogo-2-pre-launch.log ; ll cogo-2-pre-launch.log
//#pragma GCC optimize ("O0") <-- note: using this pragma to force -O0 should work but it does not... resulting in seg faults etc :-(
#if defined __OPTIMIZE__
#pragma message "Need to compile " __FILE__ " with -O0 otherwise stack manipulation for coroutines can silently fail even if it appears to work... you have been warned!"
#pragma GCC error "aborting compile!"
#endif
#define MAX_COROUTINES (1+3) /* usual / regular stack + n coroutine stacks */
#define MAX_COROUTINES_LAUNCHED (MAX_COROUTINES-1) /* n coroutine stacks */
static jmp_buf co_id_jmp_buf[MAX_COROUTINES];
static char * co_id_stack_lo[MAX_COROUTINES];
static char * co_id_stack_hi[MAX_COROUTINES];
static int co_id = 0;
static int co_id_hi = 0;
static int co_stack_direction = 0 ; // set via stack_direction_get(); + for upwards and - for downwards
static int co_stack_size = 0 ;
static int co_stack_size_per_coroutine = 10000 ; // todo: set this dynamically somehow
static int co_stack_corountine_partitions = 0 ;
static int co_routine_zero_exit = 0 ;
static char * co_stack_addr_main;
volatile void * co_stack_stick;
void co_sanity_check(int id_new) {
char * stack_addr_this = (char *)&id_new;
printf("- %d: co_sanity_check() // co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u\n",
co_id,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]));
if ((stack_addr_this > co_id_stack_lo[id_new])
|| (stack_addr_this < co_id_stack_hi[id_new])) { // todo: get this to work with "up" stacks
printf("- %d: co_sanity_check() // ERROR: current stack pointer is out of range!\n", co_id);
exit(1);
}
}
void co_switch_internal(int id_old, int id_new) {
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
void *there = &co_id_jmp_buf[id_new];
printf("- %d: co_switch_internal(%d -> %d) // setjmp(%d) for jump point b\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
printf("- %d: co_switch_internal(%d -> %d) // longjmp(%d)\n... voodoo ...\n", co_id, id_old, id_new, id_new);
co_id = id_new;
longjmp(there, 1);
}
void co_switch(int id_old, int id_new) {
co_switch_internal(id_old, id_new);
// note: return maybe from different coroutine, so cannot use function variables now!
printf("- %d: co_switch() continued via jump point b\n", co_id);
co_sanity_check(co_id);
}
int co_stack_direction_get(void * stack_addr_caller) {
int direction;
if (stack_addr_caller) {
// come here if 2nd call
void * stack_addr_callee = &stack_addr_caller;
int diff = stack_addr_callee - stack_addr_caller;
direction = (diff < 0) ? -1 : +1;
printf("- %d: co_stack_direction_get(stack_addr_caller=%p) {} = %+d AKA %s // stack_addr_callee=%p diff=%d\n", co_id, stack_addr_caller, direction, (-1 == direction) ? "down" : "up", stack_addr_callee, diff);
}
else {
// come here if 1st call
void * stack_addr_caller = &stack_addr_caller;
direction = co_stack_direction_get(stack_addr_caller);
}
return(direction);
}
int co_stack_size_get(void * argc_addr) {
co_stack_addr_main = argc_addr;
struct rlimit l;
getrlimit(RLIMIT_STACK, &l);
printf("- 0: co_stack_size_get() {} = %'ld bytes %s @ stack_addr_main=%p\n", l.rlim_cur, (co_stack_direction < 0) ? "downward": "upward", co_stack_addr_main);
char * co_id_stack_end_1 = co_stack_addr_main; // todo: get more precise stack addresses... from /proc ?
char * co_id_stack_end_2 = co_stack_addr_main + (co_stack_direction * l.rlim_cur);
co_id_stack_lo[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
return(l.rlim_cur);
}
void co_launch_internal(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
char * stack_addr_this = (char *)&thread_main_name;
// todo: assert that id_old must be coroutine 0
co_stack_corountine_partitions ++;
int size_of_all_partitions_so_far = (co_stack_corountine_partitions * co_stack_size_per_coroutine);
if (id_new) {
char * co_id_stack_end_1 = stack_addr_this + (co_stack_direction * ((0 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
char * co_id_stack_end_2 = stack_addr_this + (co_stack_direction * ((1 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
co_id_stack_lo[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
}
printf("- %d: co_launch_internal(%d -> %d) // co_stack_corountine_partitions=%d size_of_all_partitions_so_far=%d*%'d=%'d co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u%s\n",
co_id,
id_old,
id_new,
co_stack_corountine_partitions,
co_stack_corountine_partitions,
co_stack_size_per_coroutine,
size_of_all_partitions_so_far,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ),
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]),
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]),
id_new ? "" : " <-- coroutine 0 stack skips past all others!");
// todo: assert that stack for this process is big enough for alloca()
co_stack_stick = alloca(size_of_all_partitions_so_far); // ensure optimizer always keeps alloca() call
if (id_new) {
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
printf("- %d: co_launch_internal(%d -> %d) // setjmp(%d) for jump point a\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
}
printf("- %d: co_launch_internal(%d -> %d) // thread_main() AKA %s()\n", co_id, id_old, id_new, thread_main_name);
if (id_new) {
co_id = id_new;
}
co_sanity_check(co_id);
thread_main();
// note: return maybe from different coroutine, so cannot use function variables now!
if (co_id) {
printf("- %d: co_launch_internal() // INTERNAL ERROR: program flow for coroutines > 0 should never come here!\n", co_id);
abort(); // should never come here!
}
else {
printf("- %d: co_launch_internal() // coroutine 0 exiting; process shutting down shortly?\n", co_id);
co_routine_zero_exit = 1;
}
}
void co_launch(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
co_launch_internal(id_old, id_new, thread_main, thread_main_name);
// note: return maybe from different coroutine, so cannot use function variables now!
if (co_routine_zero_exit) {
// e.g. sub_main() returning to main() and soon process exit?
}
else {
printf("- %d: co_launch() continued via jump point a\n", co_id);
co_sanity_check(co_id);
}
}
static void co_main(void) {
co_switch(co_id, 0); // immediately give control back to coroutine 0
for (;;) {
int id_new = rand() % (1 + co_id_hi); // if co_id_hi is 5, then 'random' number between 0 and 5
printf("- %d: co_main() // jumping to random corountine %d of main+%d so far\n", co_id, id_new, co_id_hi);
co_switch(co_id, id_new);
printf("- %d: co_main() // co_switch() returned!\n", co_id);
}
}
static void sub_main(void) {
printf("- 0: sub_main() // continuing main() with stack position skipped past launched coroutine stacks\n");
co_sanity_check(0);
co_switch(0, 1);
printf("- 0: sub_main() finishing\n");
}
int main(int argc, char **argv) {
setlocale(LC_NUMERIC, "");
srand(4);
printf("- 0: Algorithm:\n");
printf("- 0: - Coroutine 0 pre-launches n new corountines, without passing control to them, each of which has a fixed stack size.\n");
printf("- 0: - After pre-launch coroutine 0 continues in sub_main() which becomes the new main()\n");
printf("- 0: - sub_main() switches control to coroutine 1.\n");
printf("- 0: - Each created coroutine enters an endless loop, which transfers control to a random other coroutine.\n");
printf("- 0: - If control is randomly passed to coroutine 0, the process exits\n");
printf("- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html modified for pre-launching.\n");
printf("- 0: - Note: Stack space for each new coroutine is reserved using alloca().\n");
printf("- 0: - Note: Stack space for exiting coroutine 0 can grow beyond co_stack_size_per_coroutine.\n");
printf("- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.\n");
printf("- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.\n");
printf("- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().\n");
printf("- 0: sizeof(jmp_buf)=%ld\n", sizeof(jmp_buf));
co_stack_direction = co_stack_direction_get(NULL);
co_stack_size = co_stack_size_get(&argc);
printf("- 0: launching %d corountines in addition to coroutine 0\n", MAX_COROUTINES_LAUNCHED);
while (co_id_hi < MAX_COROUTINES_LAUNCHED) {
co_id_hi ++;
co_launch(0, co_id_hi, co_main, "co_main");
printf("- %d: co_launch() returned!\n", co_id);
}
printf("- 0: launched all corountines\n");
co_launch(0, 0, sub_main, "sub_main");
printf("- 0: back from sub_main(); process exiting\n");
return 0;
}
Raw
cogo-3-dummy-libarchive.c
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <setjmp.h>
#include <sys/resource.h>
#include <alloca.h>
#include <locale.h>
// $ truncate --size=0 cogo-3-dummy-libarchive.log ; gcc -O0 -fstack-protector-all -o cogo-3-dummy-libarchive cogo-3-dummy-libarchive.c && ./cogo-3-dummy-libarchive | tee cogo-3-dummy-libarchive.log ; ll cogo-3-dummy-libarchive.log ; cat cogo-3-dummy-libarchive.log | egrep "(instance_init|// order|callback.*instance.*leave)"
//#pragma GCC optimize ("O0") <-- note: using this pragma to force -O0 should work but it does not... resulting in seg faults etc :-(
#if defined __OPTIMIZE__
#pragma message "Need to compile " __FILE__ " with -O0 otherwise stack manipulation for coroutines can silently fail even if it appears to work... you have been warned!"
#pragma GCC error "aborting compile!"
#endif
#define MAX_COROUTINES (1+2) /* usual / regular stack + n coroutine stacks */
#define MAX_COROUTINES_LAUNCHED (MAX_COROUTINES-1) /* n coroutine stacks */
static jmp_buf co_id_jmp_buf[MAX_COROUTINES];
static char * co_id_stack_lo[MAX_COROUTINES];
static char * co_id_stack_hi[MAX_COROUTINES];
static int co_id = 0;
static int co_id_hi = 0;
static int co_stack_direction = 0 ; // set via stack_direction_get(); + for upwards and - for downwards
static int co_stack_size = 0 ;
static int co_stack_size_per_coroutine = 10000 ; // todo: set this dynamically somehow
static int co_stack_corountine_partitions = 0 ;
static int co_routine_zero_exit = 0 ;
static char * co_stack_addr_main;
volatile void * co_stack_stick;
void co_sanity_check(int id_new) {
char * stack_addr_this = (char *)&id_new;
printf("- %d: co_sanity_check() // co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u\n",
co_id,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]));
if ((stack_addr_this > co_id_stack_lo[id_new])
|| (stack_addr_this < co_id_stack_hi[id_new])) { // todo: get this to work with "up" stacks
printf("- %d: co_sanity_check() // ERROR: current stack pointer is out of range!\n", co_id);
exit(1);
}
}
void co_switch_internal(int id_old, int id_new) {
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
void *there = &co_id_jmp_buf[id_new];
printf("- %d: co_switch_internal(%d -> %d) // setjmp(%d) for jump point b\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
printf("- %d: co_switch_internal(%d -> %d) // longjmp(%d)\n... voodoo ...\n", co_id, id_old, id_new, id_new);
co_id = id_new;
longjmp(there, 1);
}
void co_switch(int id_old, int id_new) {
co_switch_internal(id_old, id_new);
// note: return maybe from different coroutine, so cannot use function variables now!
printf("- %d: co_switch() continued via jump point b\n", co_id);
co_sanity_check(co_id);
}
int co_stack_direction_get(void * stack_addr_caller) {
int direction;
if (stack_addr_caller) {
// come here if 2nd call
void * stack_addr_callee = &stack_addr_caller;
int diff = stack_addr_callee - stack_addr_caller;
direction = (diff < 0) ? -1 : +1;
printf("- %d: co_stack_direction_get(stack_addr_caller=%p) {} = %+d AKA %s // stack_addr_callee=%p diff=%d\n", co_id, stack_addr_caller, direction, (-1 == direction) ? "down" : "up", stack_addr_callee, diff);
}
else {
// come here if 1st call
void * stack_addr_caller = &stack_addr_caller;
direction = co_stack_direction_get(stack_addr_caller);
}
return(direction);
}
int co_stack_size_get(void * argc_addr) {
co_stack_addr_main = argc_addr;
struct rlimit l;
getrlimit(RLIMIT_STACK, &l);
printf("- 0: co_stack_size_get() {} = %'ld bytes %s @ stack_addr_main=%p\n", l.rlim_cur, (co_stack_direction < 0) ? "downward": "upward", co_stack_addr_main);
char * co_id_stack_end_1 = co_stack_addr_main; // todo: get more precise stack addresses... from /proc ?
char * co_id_stack_end_2 = co_stack_addr_main + (co_stack_direction * l.rlim_cur);
co_id_stack_lo[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
return(l.rlim_cur);
}
void co_launch_internal(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
char * stack_addr_this = (char *)&thread_main_name;
// todo: assert that id_old must be coroutine 0
co_stack_corountine_partitions ++;
int size_of_all_partitions_so_far = (co_stack_corountine_partitions * co_stack_size_per_coroutine);
if (id_new) {
char * co_id_stack_end_1 = stack_addr_this + (co_stack_direction * ((0 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
char * co_id_stack_end_2 = stack_addr_this + (co_stack_direction * ((1 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
co_id_stack_lo[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
}
printf("- %d: co_launch_internal(%d -> %d) // co_stack_corountine_partitions=%d size_of_all_partitions_so_far=%d*%'d=%'d co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u%s\n",
co_id,
id_old,
id_new,
co_stack_corountine_partitions,
co_stack_corountine_partitions,
co_stack_size_per_coroutine,
size_of_all_partitions_so_far,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ),
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]),
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]),
id_new ? "" : " <-- coroutine 0 stack skips past all others!");
// todo: assert that stack for this process is big enough for alloca()
co_stack_stick = alloca(size_of_all_partitions_so_far); // ensure optimizer always keeps alloca() call
if (id_new) {
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
printf("- %d: co_launch_internal(%d -> %d) // setjmp(%d) for jump point a\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
}
printf("- %d: co_launch_internal(%d -> %d) // thread_main() AKA %s()\n", co_id, id_old, id_new, thread_main_name);
if (id_new) {
co_id = id_new;
}
co_sanity_check(co_id);
thread_main();
// note: return maybe from different coroutine, so cannot use function variables now!
if (co_id) {
printf("- %d: co_launch_internal() // INTERNAL ERROR: program flow for coroutines > 0 should never come here!\n", co_id);
abort(); // should never come here!
}
else {
printf("- %d: co_launch_internal() // coroutine 0 exiting; process shutting down shortly?\n", co_id);
co_routine_zero_exit = 1;
}
}
void co_launch(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
co_launch_internal(id_old, id_new, thread_main, thread_main_name);
// note: return maybe from different coroutine, so cannot use function variables now!
if (co_routine_zero_exit) {
// e.g. sub_main() returning to main() and soon process exit?
}
else {
printf("- %d: co_launch() continued via jump point a\n", co_id);
co_sanity_check(co_id);
}
}
#define DUMMY_LIBARCHIVE_CALLBACKS (2)
int dummy_libarchive_callback_calls[MAX_COROUTINES];
int dummy_libarchine_callback_order[MAX_COROUTINES_LAUNCHED];
void dummy_libarchive_instance_init(void) {
printf("- %d: dummy_libarchive_instance_init()\n", co_id);
dummy_libarchive_callback_calls[co_id] = 0;
}
void dummy_libarchive_callback(void) {
dummy_libarchive_callback_calls[co_id] ++;
printf("- %d: dummy_libarchive_callback() // callback %d for instance %d; enter\n", co_id, dummy_libarchive_callback_calls[co_id], co_id);
co_switch(co_id, 0);
printf("- %d: dummy_libarchive_callback() // callback %d for instance %d; leave\n", co_id, dummy_libarchive_callback_calls[co_id], co_id);
}
void dummy_libarchive_random_stack_depth(int depth) {
printf("- %d: dummy_libarchive_random_stack_depth(%d) // enter\n", co_id, depth);
if (depth) {
dummy_libarchive_random_stack_depth(depth - 1);
}
else {
dummy_libarchive_callback();
}
printf("- %d: dummy_libarchive_random_stack_depth(%d) // leave\n", co_id, depth);
}
void dummy_libarchive_callback_order_init(void) {
printf("- %d: dummy_libarchive_callback_order_init() {} // order", co_id);
for (int l = 1; l <= 4; l ++) {
int o = 0;
for (int id = 1; id <= MAX_COROUTINES_LAUNCHED; id ++) {
for (int num = 1; num <= DUMMY_LIBARCHIVE_CALLBACKS; num ++) {
if (1 == l) { dummy_libarchine_callback_order[o] = id; }
else if (4 == l) { printf(" %d", dummy_libarchine_callback_order[o]); }
else { int o2 = rand() % (MAX_COROUTINES_LAUNCHED * DUMMY_LIBARCHIVE_CALLBACKS); int t = dummy_libarchine_callback_order[o]; dummy_libarchine_callback_order[o] = dummy_libarchine_callback_order[o2]; dummy_libarchine_callback_order[o2] = t; }
o ++;
}
}
}
printf("\t\t\t\t<-- milestone 4\n");
}
static void co_main(void) {
for (;;) {
co_switch(co_id, 0); // immediately give control back to coroutine 0 AKA pre-launch
dummy_libarchive_instance_init();
for (int i = 0; i < DUMMY_LIBARCHIVE_CALLBACKS; i++) {
dummy_libarchive_random_stack_depth(1 + (rand() % 4));
}
}
}
static void sub_main(void) {
printf("- 0: sub_main() // enter; continuing main() with stack position skipped past launched coroutine stacks\n");
co_sanity_check(0);
printf("- 0: sub_main() // pass control to coroutines until first callback entered\t\t<-- milestone 3\n");
for (int id = 1; id <= MAX_COROUTINES_LAUNCHED; id ++) {
printf("- 0: sub_main() // pass control to coroutine %d until first callback entered\n", id);
co_switch(0, id);
}
dummy_libarchive_callback_order_init();
printf("- 0: sub_main() // pass control to coroutine callbacks in desired order\t\t\t<-- milestone 5\n");
for (int o = 0; o < (MAX_COROUTINES_LAUNCHED * DUMMY_LIBARCHIVE_CALLBACKS); o ++) {
printf("- 0: sub_main() // pass control to coroutine %d to complete its callback\n", dummy_libarchine_callback_order[o]);
co_switch(0, dummy_libarchine_callback_order[o]);
}
printf("- 0: sub_main() // leave\n");
}
int main(int argc, char **argv) {
printf("- 0: main() // enter\n");
setlocale(LC_NUMERIC, "");
srand(25);
printf("- 0: Algorithm:\n");
printf("- 0: - Coroutine 0 pre-launches n new corountines, without passing control to them, each of which has a fixed stack size.\n");
printf("- 0: - After pre-launch coroutine 0 continues in sub_main() which becomes the new main()\n");
printf("- 0: - sub_main() switches control to each new coroutine one by one, running until the first callback in invoked.\n");
printf("- 0: - Each callback is invoked after a random number of recursive function calls, simulating libarchive stack use.\n");
printf("- 0: - At this point all new coroutines have run until calling their blocking callback.\n");
printf("- 0: - sub_main() determines the order in which the blocking callbacks will continue.\n");
printf("- 0: - sub_main() switches control to each new coroutine in the predetermined order to unblock each callback\n");
printf("- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html modified for pre-launching.\n");
printf("- 0: - Note: Stack space for each new coroutine is reserved using alloca().\n");
printf("- 0: - Note: Stack space for exiting coroutine 0 can grow beyond co_stack_size_per_coroutine.\n");
printf("- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.\n");
printf("- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.\n");
printf("- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().\n");
printf("- 0: sizeof(jmp_buf)=%ld\n", sizeof(jmp_buf));
co_stack_direction = co_stack_direction_get(NULL);
co_stack_size = co_stack_size_get(&argc);
printf("- 0: launching %d corountines in addition to coroutine 0\t\t\t\t\t<-- milestone 1\n", MAX_COROUTINES_LAUNCHED);
while (co_id_hi < MAX_COROUTINES_LAUNCHED) {
co_id_hi ++;
co_launch(0, co_id_hi, co_main, "co_main");
printf("- %d: co_launch() returned!\n", co_id);
}
printf("- 0: launched all corountines\t\t\t\t\t\t\t\t<-- milestone 2\n");
co_launch(0, 0, sub_main, "sub_main");
printf("- 0: main() // leave\n");
return 0;
}
Raw
cogo.c
#include <stdio.h>
#include <stdlib.h>
#include <bsd/stdlib.h> // sudo apt-get install libbsd-dev
#include <setjmp.h>
#include <sys/resource.h>
#include <alloca.h>
#include <locale.h>
// $ truncate --size=0 cogo.log ; gcc -O0 -fstack-protector-all -o cogo cogo.c && ./cogo | tee cogo.log ; ll cogo.log
//#pragma GCC optimize ("O0") <-- note: using this pragma to force -O0 should work but it does not... resulting in seg faults etc :-(
#if defined __OPTIMIZE__
#pragma message "Need to compile " __FILE__ " with -O0 otherwise stack manipulation for coroutines can silently fail even if it appears to work... you have been warned!"
#pragma GCC error "aborting compile!"
#endif
#define MAX_COROUTINES (1+3) /* usual / regular stack + n coroutine stacks */
static jmp_buf co_id_jmp_buf[MAX_COROUTINES];
static char * co_id_stack_lo[MAX_COROUTINES];
static char * co_id_stack_hi[MAX_COROUTINES];
static int co_id = 0;
static int co_id_hi = 0;
static int co_stack_direction = 0 ; // set via stack_direction_get(); + for upwards and - for downwards
static int co_stack_size = 0 ;
static int co_stack_size_per_coroutine = 10000;
static int co_stack_corountine_partitions = 0 ;
static char * co_stack_addr_main;
volatile void * co_stack_stick;
void co_sanity_check(int id_new) {
char * stack_addr_this = (char *)&id_new;
printf("- %d: co_sanity_check() // co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u\n",
co_id,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]));
if ((stack_addr_this > co_id_stack_lo[id_new])
|| (stack_addr_this < co_id_stack_hi[id_new])) { // todo: get this to work with "up" stacks
printf("- %d: co_sanity_check() // ERROR: current stack pointer is out of range!\n", co_id);
exit(1);
}
}
void co_switch_internal(int id_old, int id_new) {
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
void *there = &co_id_jmp_buf[id_new];
printf("- %d: co_switch_internal(%d -> %d) // setjmp(%d) for jump point b\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
printf("- %d: co_switch_internal(%d -> %d) // longjmp(%d)\n... voodoo ...\n", co_id, id_old, id_new, id_new);
co_id = id_new;
longjmp(there, 1);
}
void co_switch(int id_old, int id_new) {
co_switch_internal(id_old, id_new);
// note: return from co_switch_internal() can be from different coroutine, so cannot use co_id_old and co_id_new now!
printf("- %d: co_switch() continued via jump point b\n", co_id);
co_sanity_check(co_id);
}
int co_stack_direction_get(void * stack_addr_caller) {
int direction;
if (stack_addr_caller) {
// come here if 2nd call
void * stack_addr_callee = &stack_addr_caller;
int diff = stack_addr_callee - stack_addr_caller;
direction = (diff < 0) ? -1 : +1;
printf("- %d: co_stack_direction_get(stack_addr_caller=%p) {} = %+d AKA %s // stack_addr_callee=%p diff=%d\n", co_id, stack_addr_caller, direction, (-1 == direction) ? "down" : "up", stack_addr_callee, diff);
}
else {
// come here if 1st call
void * stack_addr_caller = &stack_addr_caller;
direction = co_stack_direction_get(stack_addr_caller);
}
return(direction);
}
int co_stack_size_get(void * argc_addr) {
co_stack_addr_main = argc_addr;
struct rlimit l;
getrlimit(RLIMIT_STACK, &l);
printf("- 0: co_stack_size_get() {} = %'ld bytes %s @ stack_addr_main=%p\n", l.rlim_cur, (co_stack_direction < 0) ? "downward": "upward", co_stack_addr_main);
char * co_id_stack_end_1 = co_stack_addr_main; // todo: get more precise stack addresses... from /proc ?
char * co_id_stack_end_2 = co_stack_addr_main + (co_stack_direction * l.rlim_cur);
co_id_stack_lo[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[0] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
return(l.rlim_cur);
}
void co_launch_internal(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
char * stack_addr_this = (char *)&thread_main_name;
co_stack_corountine_partitions ++;
int size_of_all_partitions_so_far = (co_stack_corountine_partitions * co_stack_size_per_coroutine);
char * co_id_stack_end_1 = stack_addr_this + (co_stack_direction * ((0 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
char * co_id_stack_end_2 = stack_addr_this + (co_stack_direction * ((1 + co_stack_corountine_partitions) * co_stack_size_per_coroutine));
co_id_stack_lo[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_2 : co_id_stack_end_1;
co_id_stack_hi[id_new] = co_id_stack_end_2 > co_id_stack_end_1 ? co_id_stack_end_1 : co_id_stack_end_2;
printf("- %d: co_launch_internal(%d -> %d) // co_stack_corountine_partitions=%d size_of_all_partitions_so_far=%d*%'d=%'d co_stack_addr_main@%s%'u end1@%s%'u end2@%s%'u\n",
co_id,
id_old,
id_new,
co_stack_corountine_partitions,
co_stack_corountine_partitions,
co_stack_size_per_coroutine,
size_of_all_partitions_so_far,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - stack_addr_this ) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_lo[id_new]) ,
(co_stack_direction < 0) ? "-" : "+", abs(co_stack_addr_main - co_id_stack_hi[id_new]));
co_stack_stick = alloca(size_of_all_partitions_so_far); // ensure optimizer always keeps alloca() call
void *here = &co_id_jmp_buf[id_old]; // https://www.pcjs.org/pubs/pc/reference/microsoft/kb/Q34921/
printf("- %d: co_launch_internal(%d -> %d) // setjmp(%d) for jump point a\n", co_id, id_old, id_new, id_old);
if (setjmp(here)) {
// note: putting instrumentation here appears to mess up the stack (although no seg fault!)
return; // note: setjmp() man page: "The stack context will be invalidated if the function which called setjmp() returns."
}
printf("- %d: co_launch_internal(%d -> %d) // thread_main() AKA %s()\n", co_id, id_old, id_new, thread_main_name);
co_id = id_new;
co_sanity_check(co_id);
thread_main();
abort();
}
void co_launch(int id_old, int id_new, void (*thread_main)(void), const char * thread_main_name) {
co_launch_internal(id_old, id_new, thread_main, thread_main_name);
// note: return from co_launch_internal() can be from different coroutine, so cannot use co_id_old and co_id_new now!
printf("- %d: co_launch() continued via jump point a\n", co_id);
co_sanity_check(co_id);
}
static void co_main(void) {
for (;;) {
int id_new = rand() % (1 + co_id_hi); // if co_id_hi is 5, then 'random' number between 0 and 5
printf("- %d: co_main() // jumping to random corountine %d of main+%d so far\n", co_id, id_new, co_id_hi);
co_switch(co_id, id_new);
printf("- %d: co_main() // co_switch() returned!\n", co_id);
}
}
int main(int argc, char **argv) {
setlocale(LC_NUMERIC, "");
srand(4);
printf("- 0: Algorithm:\n");
printf("- 0: - Coroutine 0 launches and passes control to up to n new corountines, each of which has a fixed stack size.\n");
printf("- 0: - Each created coroutine enters an endless loop, which transfers control to a random other coroutine.\n");
printf("- 0: - If control is randomly passed to coroutine 0, a new coroutine is launched, or process exit if all launched\n");
printf("- 0: - Note: Instrumented version of https://fanf.livejournal.com/105413.html\n");
printf("- 0: - Note: Stack space for each new coroutine is reserved using alloca().\n");
printf("- 0: - Note: After switching coroutine control, the new stack pointer is sanity checked to be in expected range.\n");
printf("- 0: - Note: The first digit, after the dash on each output line, shows the coroutine ID with control.\n");
printf("- 0: - Note: The output lines with 'longjmp() ... voodoo' show when coroutine control is switched via longjmp().\n");
printf("- 0: sizeof(jmp_buf)=%ld\n", sizeof(jmp_buf));
co_stack_direction = co_stack_direction_get(NULL);
co_stack_size = co_stack_size_get(&argc);
printf("- 0: launching %d corountines in addition to coroutine 0\n", MAX_COROUTINES - 1);
while (++co_id_hi < MAX_COROUTINES) {
co_launch(0, co_id_hi, co_main, "co_main");
printf("- %d: co_launch() returned!\n", co_id);
}
printf("- 0: launched all corountines\n");
return 0;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment