from a user perspective, we want something that can run in the context of a cabal project.
it should have some idea of a hierarchy of tests (could be from tasty or hspec, i wouldn't bother trying to make it general). when it starts up, it ought to try to run all the tests, noting which ones fail (and also which ones fail to compile: ideally, you would be able to achieve local progress even if there are compile errors in other parts of the program, though it's not essential.)
when a test is failing, it should scope down to that file, and keep running that test on file change until it works. when it succeeds, the whole test suite one layer up should run, and so on up to the root of the test tree.
When a test file changes, that's easy - we just run it. When another file changes, we need to trace back all the test files that are affected by it, and add them to the list of tests to be run.
expected pain points:
- inotify equiv on mac - https://github.com/nkpart/fsevents-api maybe? would be nice to have a general layer.
- convincing cabal to give you the dependency tree of what needs to recompiled. would be nice to avoid a full "cabal build" on every run, as it can be slow.
OK, I found the only working part I wrote. It watches a filepath and runs "cabal build" on changes. I had started work on a small module to either whitelist or blacklist files to watch for (right now it's far too eager; if emacs saves a file in-progress everything recompiles), and then I moved to cabal integration and didn't get anywhere useful.
Integrating directly with cabal sounds like the smartest answer, but I haven't had the time to work through what that would even look like. An alternative I was working on was a basic config file that would let you specify what command to use for tests, and then a haskeline prompt that would let you either watch for changes and recompile or watch for changes and run tests. Being able to specify tests in a more granular format like @mwotton mentioned would be great too.