flaky-repl-test-notes.md

after about 10 builds and playing with logging and so on, here are abridged talking points:

• failure is definitely intermittent
• initial attempts to add some println logging to repl-test and prepl-test seemed to make the failure stop entirely - this leads me to believe it could be a timing thing, e.g. maybe the bb process from repl-test isn't quite dead yet
• to see if I could get some info while not causing any extra delay between the tests, I removed the printlns from repl-test, but left them in prepl-test
• an interesting finding that (speculatively) supports the timing hypothesis is that on a passing run, there seems to be a wait time of ~100ms for the port to become available (as reported by wait-for-port); on a failing run, the wait time was reported as 0ms, so maybe the port is still open (open enough for wait-for-port to return) from the prior test

Proposals:

• add some sleep like the JVM branch of the test has - I don't really like this, because it feels like fudging stuff with timing, which could potentially still be flaky if the machine is especially slow to close the port
• use different ports for the tests - basically, increment the port numbers in each test, and update socket-command to take a port (or potentially have socket-command take a port and return a function with the existing interface with the port closured over)