forkProcess problems with GHC 7.6 and 7.8

error-ghc-7.6.3-linux-debug-segfault-gdb.txt

(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: /home/niklas/src/hs/forkProcess-problem +RTS -N8
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff640c700 (LWP 22269)]
[New Thread 0x7ffff5c0b700 (LWP 22270)]
[New Thread 0x7ffff540a700 (LWP 22271)]
[New Thread 0x7ffff4c09700 (LWP 22272)]
[New Thread 0x7fffe7fff700 (LWP 22273)]
[New Thread 0x7fffe77fe700 (LWP 22274)]
[New Thread 0x7fffe6ffd700 (LWP 22275)]
[New Thread 0x7fffe64ff700 (LWP 22276)]
[New Thread 0x7fffe5cfe700 (LWP 22277)]
[New Thread 0x7fffe4cfc700 (LWP 22279)]
[New Thread 0x7fffcf7fe700 (LWP 22281)]
[New Thread 0x7fffcffff700 (LWP 22278)]
[New Thread 0x7fffe54fd700 (LWP 22280)]
[New Thread 0x7fffceffd700 (LWP 22282)]
[New Thread 0x7fffce7fc700 (LWP 22283)]
[New Thread 0x7fffcdffb700 (LWP 22295)]
[Thread 0x7fffe7fff700 (LWP 22273) exited]
Response: Run
Response: Run
Response: Run

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffceffd700 (LWP 22282)]
0x000000000049affb in LOOKS_LIKE_CLOSURE_PTR ()
(gdb) bt
#0  0x000000000049affb in LOOKS_LIKE_CLOSURE_PTR ()
#1  0x000000000049b8b6 in evacuate ()
#2  0x000000000048e9dd in mark_root ()
#3  0x00000000004745d0 in markCapability ()
#4  0x000000000048dc15 in gcWorkerThread ()
#5  0x0000000000473abe in yieldCapability ()
#6  0x000000000047e940 in scheduleYield ()
#7  0x000000000047deed in schedule ()
#8  0x0000000000480dce in scheduleWorker ()
#9  0x00000000004854a1 in workerStart ()
#10 0x00007ffff7bc4182 in start_thread (arg=0x7fffceffd700) at pthread_create.c:312
#11 0x00007ffff6d6338d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

error-ghc-7.6.3-linux-glibc.txt

ghc --make -O forkProcess-problem-original.hs -threaded && ./forkProcess-problem-original +RTS -N8
Press enter to exit
*** glibc detected *** ./forkProcess-problem-original: corrupted double-linked list: 0x00007f3a68000900 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7ec66)
/lib/x86_64-linux-gnu/libc.so.6(+0x7ff7d)
./forkProcess-problem-original
./forkProcess-problem-original
./forkProcess-problem-original

error-ghc-7.6.3-linux-segfault-gdb.txt

(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: /home/niklas/src/hs/forkProcess-problem +RTS -N8
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff640c700 (LWP 4288)]
[New Thread 0x7ffff5c0b700 (LWP 4289)]
[New Thread 0x7ffff540a700 (LWP 4290)]
[New Thread 0x7ffff4c09700 (LWP 4291)]
[New Thread 0x7fffe7fff700 (LWP 4293)]
[New Thread 0x7fffe77fe700 (LWP 4294)]
[New Thread 0x7fffe6ffd700 (LWP 4295)]
[New Thread 0x7fffe64ff700 (LWP 4296)]
[New Thread 0x7fffe5cfe700 (LWP 4297)]
[New Thread 0x7fffe4cfc700 (LWP 4299)]
[New Thread 0x7fffce7fc700 (LWP 4303)]
[New Thread 0x7fffcf7fe700 (LWP 4301)]
[New Thread 0x7fffe54fd700 (LWP 4298)]
[New Thread 0x7fffcffff700 (LWP 4300)]
[New Thread 0x7fffceffd700 (LWP 4302)]
[Thread 0x7ffff640c700 (LWP 4288) exited]
[New Thread 0x7fffcdffb700 (LWP 4325)]
Response: Run
Response: Run
[New Thread 0x7ffff640c700 (LWP 4357)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff640c700 (LWP 4357)]
0x0000000000404b20 in evacuate ()
(gdb) bt
#0  0x0000000000404b20 in evacuate ()
#1  0x00000000004841df in mark_root ()
#2  0x0000000000478201 in markCapability ()
#3  0x0000000000485f91 in gcWorkerThread ()
#4  0x0000000000477cec in yieldCapability ()
#5  0x000000000047db8b in schedule ()
#6  0x000000000047f0ec in scheduleWorker ()
#7  0x00007ffff7bc4182 in start_thread (arg=0x7ffff640c700) at pthread_create.c:312
#8  0x00007ffff6d6338d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

error-ghc-7.6.3-linux-threaded-resurrectThreads.txt

% ./forkProcess-problem +RTS -N8
forkProcess-problem: internal error: resurrectThreads: thread blocked in a strange way: 10
    (GHC version 7.6.3 for x86_64_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

error-ghc-7.8.3-linux-threaded-resurrectThreads.txt

% ghc --make -O forkProcess-problem.hs -threaded && ./forkProcess-problem +RTS -N8
[1 of 1] Compiling Main             ( forkProcess-problem.hs, forkProcess-problem.o )
Linking forkProcess-problem ...
forkProcess-problem: internal error: resurrectThreads: thread blocked in a strange way: 10
    (GHC version 7.8.3 for x86_64_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

forkProcess-problem.hs

{-# LANGUAGE ScopedTypeVariables, LambdaCase #-}

import Prelude hiding (log)

import Control.Concurrent
import Control.Exception
import Control.Monad
import System.IO
import System.Posix.Process
import System.Posix.IO
import System.Posix.Signals (sigTERM, signalProcess)
import System.Posix.Types
import System.Timeout

-- Log a message to stderr
log :: String -> IO ()
log = hPutStrLn stderr

-- | If an error occurs within a given computation, it annotated
-- with a given message and logged and the error is re-thrown.
withErrorLog :: String -> IO a -> IO a
withErrorLog msg = handle $ \(e :: SomeException) -> do
  log (msg ++ ": " ++ show e)
  throwIO e

-- | Wrapper over System.Timeout.timeout that fails in the IO monad.
withTimeout :: Int -> String -> IO a -> IO (Either String a)
withTimeout secs descr action = do
  result <- timeout (secs * 1000000) action
  return $ case result of
    Nothing -> Left $ "Timeout in " ++ descr
    Just v -> Right v

-- Code that is executed in a @fork@-ed process and that the replaces iteself
-- with the actual job process.
child :: String -> Handle -> IO ()
child label hW = withErrorLog label $ do
    hPutStr hW "Run"
    hClose hW

-- | Forks a child POSIX process, creating a communication
-- channel between the master and the child processes.
-- Supplies the child action with its part of the pipe and returns
-- the master part of the pipe as its result.
forkWithPipe :: String -> IO (ProcessID, Handle)
forkWithPipe label = do
  (r, w) <- createPipe
  hR <- fdToHandle r
  hW <- fdToHandle w
  pid <- forkProcess (hClose hR >> child label hW)
  hClose hW
  return (pid, hR)

-- | Forks the job process and starts processing of the given job.
forkJobProcess :: String -> IO ()
forkJobProcess label = do
  (pid, hR) <- forkWithPipe label

  withTimeout 1 (label ++ ": reading response") (hGetLine hR) >>= \case
    Right resp -> do
      threadDelay 100000
      log $ "Response: " ++ resp

    Left err -> do
      log $ "Client FAILED: " ++ show err
      log "Closing the pipe"
      withErrorLog "Closing the communication pipe failed" (hClose hR)
      log $ "Getting the status of the process " ++ label
      status <- getProcessStatus False True pid
      case status of
        Just s -> do
          log $ "Child process (job " ++ show label ++ ") status: " ++ show s
        Nothing -> do
          log $ "Child process (job " ++ show label ++ ") running, killing by SIGTERM"
          signalProcess sigTERM pid


main :: IO ()
main = do

  let _FORKS = 80

  forM_ [1.._FORKS :: Int] $ \i -> forkIO $
    forever $ forkJobProcess ("Job" ++ show i)

  _ <- getLine
  return ()

ghc-irc-discussion-1.txt

(22:11:04) The topic for #ghc is: GHC Development | GHC 7.8.3 Released! | Trac: https://ghc.haskell.org/trac/ghc | Phabricator: https://phabricator.haskell.org | Please ask user questions in #haskell
(22:11:04) Topic for #ghc set by thoughtpolice!~a@unaffiliated/thoughtpolice at 01:00:41 on 13/07/14
328: http://hackage.haskell.org/trac/ghc
(22:13:06) nh2: hi, I have an odd problem. When I compile http://lpaste.net/raw/109569 using `ghc --make forkProcess-problem.hs -threaded -rtsopts` and run it with `./forkProcess-problem +RTS -N8` multiple times, ctrl-c-ing immeditately after each invocation, then at around the 10th run I get a segfault at startup and gdb says it's in evacutate()
(22:13:35) nh2: could anyone try to reproduce that for me with ghc 7.6.3 on linux (that's what I'm running)?
(22:25:14) dddfffd: slyfox: I could test it on mips64el if you want.  Is it necessary?
(22:25:48) carter: nh2: theres known issues with forkprocess on os x
(22:25:57) carter: @google ghc edsko forkprocess bug
(22:25:58) lambdabot: https://ghc.haskell.org/trac/ghc/ticket/9284
(22:25:59) nh2: carter: I'm on linux
(22:26:07) dddfffd: slyfox, ezyang: thanks for working on it!
(22:26:17) carter: nh2: could you test on 7.8.3?
(22:26:40) nh2: carter: tried that already, seems gone on 7.8.3, but I need to know what this is
(22:26:44) carter: ah
(22:26:50) carter: i'm on  mac and erranding this fatenroon
(22:26:51) slyfox: dddfffd: if it's not hard for you it would be great! Here is the exported patch: http://code.haskell.org/~slyfox/0001-UNREG-fix-PackageKey-emission-into-.hc-files.patch
(22:27:03) nh2: carter: and how on earth it can go across processes
(22:27:22) carter: ohhh
(22:27:27) carter: nh2: do you know how forkProcess works?
(22:27:36) carter: for ghc?
(22:27:43) dddfffd: slyfox: but it'll take a couple of days (my machine is quite slow).  should I try anyway?
(22:27:50) nh2: carter: not really
(22:27:57) nh2: carter: this is what I get after a few times:
(22:27:57) nh2: ./forkProcess-problem +RTS -N8
(22:27:57) nh2: Response: Run
(22:27:57) nh2: Response: Run
(22:27:57) nh2: [1]    15733 segmentation fault (core dumped)  ./forkProcess-problem +RTS -N8
(22:28:07) carter: nh2: think about it, any reachable data has to be copied to the new process
(22:29:27) nh2: carter: how do you mean, or how do you relate copying to the sometimes-segfault
(22:29:42) slyfox: dddfffd: as you like. i think a bit easier for you would be to wait when it hits master
(22:29:42) carter: evacuate ... i think is something related to GCing
(22:31:05) nh2: carter: yes, evacuate() is in the gc part
(22:31:16) dddfffd: slyfox: okay, I'll wait, then.  I'll send a message to ghc-devs if I encounter any issues.
(22:31:31) slyfox: thanks!
(22:33:17) nh2: carter: that's what I get in gdb: http://lpaste.net/raw/109576
(22:34:32) slyfox: nh2: you might try to add a -debug to linkflags
(22:34:46) carter: o/ slyfox
(22:34:51) slyfox: and also would be nice to see 'disassemble' output around SIGSEGV
(22:35:36) slyfox: hia carter :]
(22:37:54) nh2: carter: that got me more info: http://lpaste.net/raw/109577
(22:38:54) carter: nh2: honestly there are known bugs in how forkprocess works in even 7.8
(22:39:09) carter: forkprocess internally hooks into RTS stuff
(22:39:23) carter: 7.10 wil be the first "forkprocess is nice"
(22:39:29) carter: release in a few major versions I think
(22:39:48) ***carter wants to use fork process to fake having split up heaps
(22:40:04) nh2: carter: can you link me to some of them or a potential candidate for this one? I at least need to find a bug number to which I can contribute since forking in 7.6 is important for my stuff
(22:40:21) carter: nh2: https://ghc.haskell.org/trac/ghc/ticket/9284
(22:40:27) carter: nh2: do you need to fork process
(22:40:34) carter: or could you launch the binary several times?
(22:40:38) carter: and do it that way?
(22:40:47) carter: because thats another viable option
(22:40:59) carter: just means you have to serialize more of the initialization stuff for teh jobs
(22:42:01) nh2: I need to fork and have no alternatives (existing stuff that needs to be maintained for a few years - the ganeti project I once showed you)
(22:42:34) slyfox: yeah. stack trace is very scary. GC started marking stuff and stumbled on dangling pointer. must never happen. likely a race between wreezing existing haskell threads for GC and running new one
(22:43:07) carter: nh2: use shell
(22:43:09) carter: eg via shelly
(22:43:18) nh2: carter: ?? :)
(22:43:21) carter: or process api
(22:43:28) carter: nh2: you can run shell commands from a haskell program
(22:43:37) carter: so you can launch subprocesses
(22:43:38) carter: via that
(22:43:55) slyfox: nh2: are there many people working on ganeti in the Company? :]
(22:44:07) carter: nh2: look in System.Process
(22:44:19) slyfox: (just curious)
(22:44:22) carter: callProcess :: FilePath -> [String] -> IO ()
(22:44:23) nh2: carter: yes yes I know that, but the forking has no real alternatives; we have considered forkIO+createProcess but it doesn't work so well because we need a file lock to die when the process dies for reliability
(22:44:39) carter: and hows that differ?
(22:44:39) nh2: carter: yes we are around 6 atm
(22:44:47) carter: ?
(22:44:49) carter: g2g
(22:45:06) nh2: carter: with createProcess I can't make the file lock owned by the child process
(22:45:09) slyfox: it's likely for me
(22:45:17) carter: nh2: linux only?
(22:45:20) nh2: yes
(22:45:24) carter: use linux specific apis
(22:45:30) carter: you might ahve to wrap up some C :)
(22:45:45) slyfox: cgroups haxxz?
(22:45:45) nh2: carter: it works like `fork(); fileLock(); exec()`
(22:46:41) carter: nh2: no, look at the apis modern linux has
(22:46:44) carter: not just the posix crap
(22:47:48) nh2: carter: for the evacuate() business, do you know who is familiar with that?
(22:48:02) nh2: mr marlow?
(22:48:06) carter: nh2: you do NOT wanna muck around patching old ghc RTS stuff
(22:48:08) carter: not worth it
(22:48:18) slyfox: yeah, Simon
(22:48:31) carter: simon ++
(22:48:35) carter: well jaffacake
(22:48:36) carter: but yeah
(22:48:59) carter: but patching RTS stuff is not a casual thing
(22:49:06) nh2: I don't want to patch it, just find the bug and subscribe or make my own one so that I can link to that
(22:49:08) slyfox: might be easier to be caught in ghc-devs@ ML
(22:51:00) carter: yeah, email ghc-devs
(22:51:03) carter: with this stuff
(22:51:19) carter: maybe cc adreas v & edsko and simon marlow
(22:51:22) carter: or just emaiil devs
(22:52:30) nh2: ok, I'll see if I can hack a bash script that makes it appear more reliably and then do that
(22:53:08) slyfox: maybe haskell code itself sending kill() after delay will be enough
(22:53:42) slyfox: would be a nice addition to testsuite
(22:54:09) carter: nh2: oooo
(22:54:11) carter: i have an idea
(22:54:23) carter: so which step in the program triggers the crash?
(22:54:33) carter: some step of forkJobProcess :: String -> IO ()
(22:54:33) carter: forkJobProcess label = do  ?
(22:55:04) nh2: carter: I cannot really tell (or don't know how I could) since it's in the gc that can be triggered async any time, right?
(22:55:12) carter: no
(22:55:34) carter: this GC scan is the child process copying reachable values into the new process
(22:56:00) carter: nh2: idea: you're not giving enough time between calling fork process
(22:56:21) carter: add like threadDelay 1000   right after your forkProcess call
(22:56:41) carter: yeah... that'd do it
(22:56:42) carter: i think
(22:56:55) nh2: carter: what would I have to give time for?
(22:57:00) carter: copying the heap
(22:57:20) carter: because you're triggering a race condition of trying to scan the heap with mroe than one copying process
(22:57:21) nh2: carter: notice the forkIO around the forever: even with delay I cannot guarantee that they won't fall close together at some point
(22:57:42) carter: nh2: no
(22:57:46) carter: let me suggest it differently
(22:58:13) int-e: nh2: you could also take a close look at the 'process' package and replicate the cbits part carefully. oh and while you're at it marvel at the complexity of that code.
(22:58:40) carter: instead of  forM_ [1..8 :: Int] $ \i -> forkIO $
(22:58:40) carter:     forever $ forkJobProcess ("Job" ++ show i)
(22:58:40) carter: do
(22:58:40) carter:  forM_ [1..8 :: Int] $ \i -> do threadDelay 100 ; forkIO $
(22:58:40) carter:     forever $ forkJobProcess ("Job" ++ show i)
(22:58:40) carter: erm
(22:58:59) carter: 1000
(22:59:01) carter: or 10000
(22:59:01) carter: try it with threadDelay 10000
(22:59:02) carter: or something
(22:59:08) carter: that means it'd wait 10ms between each launch
(22:59:26) carter: but you def wanna give some time between the launches
(22:59:34) carter: nh2: try that please :)
(22:59:37) int-e: (hmm. not sure that's still the topic, but I was referring to forking a subprocess safely, with some extra functionality in the child)
(22:59:39) ion- is now known as ion
(22:59:55) carter: int-e: i know nutting
(23:00:04) carter: i'm just speculating on working aroudn the bug
(23:00:12) carter: nh2: can you run that experiment?
(23:01:06) carter: OR you could use a semaphor to make sure each launch only starts after another one finishes
(23:01:21) carter: http://hackage.haskell.org/package/base-4.7.0.1/docs/Control-Concurrent-QSem.html
(23:01:34) carter: only allow one launching of a child at a time
(23:01:41) carter: that'd probably be safer
(23:02:08) carter: yeah
(23:02:09) carter: try that
(23:02:50) carter: nh2: i bet you can make the bug happen more easily if you have a large boxed arrray on the heap that has to be copied over
(23:03:13) carter: mind you i'm just purely speculating about how you're tripping it
(23:06:09) nh2: carter: I just changed 1..8 to 1..80 and now seem to have it crash reliably at least on one of my computers (not so on the other one), and also get this new error from time to time: http://lpaste.net/raw/109581
(23:11:10) carter: nh2: did you try my suggestions or not?
(23:11:59) carter: i can write the code for you if you need
(23:12:24) carter: forkIO $ bracket_ waitQSem signalQSem action
(23:12:30) carter: or whatever
(23:12:35) carter: not quite that
(23:12:38) carter: but nearly that
(23:12:59) int-e: btw, "GHC note: forkProcess is not currently supported when using multiple processors (+RTS -N), although it is supported with -threaded as long as only one processor is being used."
(23:13:09) carter: HAH
(23:13:11) carter: :)
(23:13:17) carter: int-e: is that for 7.6 or generally?
(23:15:16) int-e: hmm I guess it's indeed supposed to work in 7.8
(23:15:45) carter: int-e: well, except for the ways its busted on mac
(23:15:50) carter: ok
(23:15:59) nh2: carter: does it matter if I use the QSem's or an mvar?
(23:16:00) carter: but eitehr way, the above forkIO codes aint supported
(23:16:08) carter: nh2: QSem will be simpler
(23:16:09) carter: use that
(23:16:47) carter: how many cores is it running on?
(23:16:56) carter: nh2: the bug might got away at -N1
(23:17:05) int-e: carter: of course that still leaves the other warning "forkProcess comes with a giant warning: since any other running threads are not copied into the child process, it's easy to go wrong: e.g. by accessing some shared resource that was held by another thread in the parent." which is still there.
(23:17:18) carter: yeah
(23:17:22) carter: like files and stuff?
(23:18:04) nh2: carter: yes, -N1 fixes the bug, I tried that earlier
int-e Intensity 
(23:18:14) carter: nh2: ooo
(23:18:17) nh2: int-e: but that particular warning is standard, same for C
(23:18:31) carter: if you use ForkOn and make sure all the launched threads use the same capability
(23:18:35) carter: the bug would go away
(23:18:43) carter: but thats the same as requireing -N1
(23:18:48) carter: just enforced in user land
(23:18:54) carter: well
(23:18:59) carter: i guess its a bit more general
(23:19:16) carter: like, yeah, forkON + make sure its using the same thread as main
(23:19:22) carter: plys maybe -qa and -qm
(23:19:30) nh2: carter: yes that would be better workaround
(23:19:47) nh2: the "forkProcess is not currently supported when using multiple processors" is gone for 7.8, right?
(23:19:56) carter: yeah, but other bugs remain
(23:20:03) carter: though they don't impact linux
(23:20:17) carter: so honestly, the above code gets really gnarly if 7.6 has to be it
(23:20:25) carter: or you get very Rigit RTS config space
(23:21:28) nh2: the other bug though, that my little file here tests when it's not segfaulting, seems to be present in 7.8 as well unfortunately
(23:21:39) carter: whats the other bug?
(23:21:55) carter: nh2: you neeed a way to check if a files still being used rihgt?
(23:22:12) carter: that all the processes can see right?
(23:22:34) nh2: the forkJobProcess creates a pipe and talks with the child trough the pipe
(23:22:45) carter: and teh bug is?
(23:23:11) int-e: carter: looking at the forkProcess() code it looks really nasty. all the worker threads are gone of course, and now threads are being killed forcefully, which might be in the middle of a  bracket  call. so MVars used as locked can become blocked. crazy. Which leaves the interesting question whether a mutator that is interrupted in the middle of execution can leave the heap in an inconsistent state. Great fun.
(23:23:21) nh2: the bug is that sometimes the child doesn't answer (probably it doesn't spawn right) - it happens when the "running, killing by" part appears in your terminal
(23:23:34) nh2: which is around every 100th line of output on my computer
(23:23:47) carter: nh2: i have an idea that lets you do System.Cmd based launching
(23:24:05) nh2: I'm all ears!
(23:24:36) carter: shared memory
(23:24:37) carter: https://hackage.haskell.org/package/mmap
(23:24:40) carter: https://hackage.haskell.org/package/vector-mmap
(23:24:58) carter: basically have a storable vector thats also a shared memory writable binary file
(23:25:04) carter: that anyone can read / write to
(23:25:09) carter: between a pair of process
(23:25:13) carter: OR just use unix sockets
(23:25:14) carter: actually
(23:25:21) carter: nh2: why are n't you trying out using unix sockets?
(23:25:34) carter: so it becomes a standard client server setup
(23:25:55) carter: AF_UNIX
(23:25:56) carter: http://hackage.haskell.org/package/network-2.5.0.0/docs/Network-Socket.html
(23:26:24) nh2: carter: I don't get it yet, how would that help me with the file lock?
(23:26:31) carter: you dont need a lock
(23:26:36) carter: you coordinate / track ownership
(23:26:38) carter: by talking
(23:26:52) int-e: carter: oh at least the latest worry is not an issue, forkProcess does an acquireAllCapabilities call on the threaded TRS. So the heap should(!) be consistent.
(23:27:18) carter: nh2: file locks make sense only if you dont control all the codes
(23:27:25) carter: otherwise just coordinate who "owns it"
(23:27:44) carter: so i'm suggesting a total redesign that reflects the specificity of what you actually want
(23:28:03) carter: i'm off for a while
(23:28:06) carter: god speed :)
(23:28:14) nh2: the file locking is an external thing though, I don't have control over that; the requirement is that my haskell process has to acquire the lock, hand it over to the forked child (which will exec() a python program), and an external program checks if the python program is alive by trying to get the file lock
(23:29:59) int-e: nh2: Btw I'm back to my original suggestion: if you feel that you must execute some code before exec() in the child, replicate the process package's code and add to the runProcess C code there, rather than hoping that forkProcess somehow leaves the IO manager and friends in a usable state.