Figwheel light ideas

ideas.md

It would be interesting if Figwheel could essentially decorate any implementation of IJavaScriptEnv, taking on the responsibility of watching the filesystem (perhaps even delegating that to ClojureScript :watch, and expanded :watch-fn, and perhaps minimally calling IJavaScriptEnv -load when Figwheel determines it is needed.

Then, for those REPLs that can participate, if IFigwheelSupport can be extended to them, they can do further cool stuff. (Perhaps IFigwheelSupport is actually necessary while IJavaScriptEnv as described in the previous paragraph is insufficient, and a more fancy impl of -load is needed?)

That way, things like the existing Node REPL could be minimally supported by Figwheel, as well as Ambly, the Browser REPL, etc.

What does IFigwheelSupport do?

Perhaps somewhere there is a HUD interface and Figwheel can ask a REPL to display things on its HUD? An iOS or Android-specific REPL could delegate to Objective-C or Java that implement a HUD using platform-specific UI widgets.

Perhaps another thing is the "punch-through" where you add capabilities to talk to the underlying Clojure to do things.

In general though, that's all that is floating around in my mind right now: A way for Figwheel to decorate any REPL with minimal features, based on IJavaScriptEnv, and then additional interfaces that allow them to opt into more extensive features. (Or for other people to independently extends those REPLs with those featueres, given the flexibility of Clojure).

This is interesting.

Hmmmm maybe something general like INotificationSupport?? To allow the compiler/repl a chance to notify the user of compilation completion, failure, and warnings? This is strongly tied to watch.

Or maybe just IMessageSupport and document a set of messages that come out of the compilation process?

I'm wondering if we can just use the internal compile :watch? Seems all that we need if skipping CSS reloading.

Adding something like CSS is an interesting way to see how a core system would extend. Do you add or inject a client side css-loadier lib? Do you add another layer of middleware to watch for the change? And you send the message of the change over the IMessageSupport channel?

Should reload be handled as a message from ClojureLand i.e. -load? I could almost do this by simply tracking @compiler-env changes and hooking into the :watch-fn and then calling -load.

The reloading story is interesting. I we need is a list of files that changed hopefully in load order. I wonder if I can get this from the compile/watch process. Or should I plan on getting this from the compile-env?

It would be nice for the compiler to say hey this is what changed. The regular command line REPL could benefit from this info as well as the other notifications. We could stop relying on print to deliver such important information.

Does any of this make sense? :)

Yeah. Simplifying it a bit, it sounds like you are describing:

1. The need to properly handle asset changes.
2. Enriched compiler API support (:watch-fn enhancements or some such).

For #1, I hadn't even been considering assets (CSS, images, etc.). When these resources change, you want them to end up being reflected in your running program. While this whole asset business, on the surface, seems to be outside of what the ClojureScript compiler would care about (simply because they are not code), there is :asset-path at least. Maybe asset management is a big part of what Figwheel is? (In other words not only code reloading but resource reloading?)

For #2, in addition to richer callbacks, the load order stuff is very interesting. Reminds me of :reload-all. I think of :watch as simply compiling individual files as they change, with no regard to order, and :reload-all causing a traversal of the dependency tree. The need to know when reloading is needed and order, etc., seems to blur the line between what Figwheel is and what the compiler is doing.

Philosophically , my high-level thought on all of this: Fighweel has fundamentally shown that there is a different and very compelling style of dev. (Alex Redington told me he has been Lisping for a decade and thinks the REPL is overrated and prefers the Figwheel approach. Hearing him essentially reject the REPL has stuck with me for a while.) So, if you want that style of dev (which seems to be a very common expectation now), then the question in my mind is: How much of this works its way down into the core of Clojure/ClojureScript? Is it all too new and too experimental still, and thus way to early to talk about it going into core? I bet that is at least partially true. If you accept that, how much can be put into core to at least facilitate it, so that Figwheel can take on the responsibility of implementing the missing portions?

(I've even begun to think that a "Figwheel for Clojure" makes just as much sense, and the only prior art I'm aware on that side is Stuart Sierra's Component library which, while oversimplifying, seems to help address how you make your server-side application code be reloadable.)

Also, if you accept that IDEs are in use, a question that crops up is, how does the IDE work with this style of development. I bet that this could end up being interfaces that, say Cursive or Emacs modes use to interact with Figwheel, to make the entire thing be even more integrated and more awesome. Or, maybe IDEs need do nothing, since Figwheel works at a lower level. But, if we get to the point where things like Cursive are debugging code and it changes on the fly, then maybe IDEs begin to care about the compliation / reload model, and being able to hook into it.

The above is, of course, much broader in scope than the smaller problem we are trying to bite off here, but it reflects where my mind has been floating around.

I think for now, it would be cool to at least define some minimal interfaces that allow for a Figwheel light, and if we pull that off, a few months from now, everyone may see the next obvious direction that stuff should go. (I for one can't, unless we at least try by biting off one piece at a time.)

Really agreeing that this should have minimum impact for the compiler until this crazy way of doing things has more time in the world.

I have been thinking that the idea of a single ICompileWatcher (on-compile, on-warning, on-error) fits into to the idea of a watching compiler. It would be great if this could present warning and error information, its a bonus to have any sort of file change data.

The rest of the changes can be detected from the environment given this information. Maybe grabbing this info from the env can eventually reside in the notion of the CLJS build API.

It's kind of a bummer that the client of this ICompileWatcher api would have to asynchronously collect the warnings to associate them with a compile.

It seems with these ingredients we could decorate a REPL-Env to autoloading in a straightforward way.

As far as the bigger picture is concerned, I am thinking that along with IDEs, build scripts that can compose all this crazy stuff folks want to do with build tools are probably going to be a winning strategy. It would be nice if the API didn't "get in the way" of this happening, but rather supported and encouraged it.

Ahh, OK, so today you can :watch "src", and optionally supply a zero-argument fn to :watch-fn. Perhaps the change is to allow passing a :compile-watcher which is an implementation of ICompileWatcher.

One immediate benefit outside of Figwheel light would be that, say IDEs could also provide an ICompileWatcher impl and decorate lines in their code editors that have warnings and somesuch.

Even if you don't :watch "src", but supply a :compile-watcher, then if you go into a REPL and type (require 'foo.bar :reload) and there is, say a syntax error in that namespace, today, it prints that stuff right in your REPL. If you are in Cursive's REPL, it could do more, like highlight the problematic lines. :)

An additional thought: Would ICompileWatcher matter to Clojure? Perhaps it does for ClojureScript because it must be transpiled? But in Clojure does it only matter when AOT?

This sounds good.

This doesn't deliver the info to the REPL-Env. A smarter ReplEnv may want it. But that may just be a complected REPL.

What do you think?

Time is a good thing. ICompileWatcher seems like it would work really well.

I would add one more method to ICompileWatcher:
before-compile
after-compile
on-warning
on-error

before-compile would allow one to mark certain files for recompile before the compile kicks off
before-compile should be passed the files that changed