JS9 is a client server pair with both components implemented in JavaScript.
The client runs in the browser and is able to be used completely stand alone with no server side interaction at all.
The client provides all of the basic image manipulation: zoom, pan, color scale, stretch, regions, etc.
The server is a node.js server so the server side code is also JavaScript.
This is potentially interesting because code can slosh from client to server (or vice versa) with very little friction.
The server provides server side analysis plugins as well as functionality for viewing large files via caching.
Importantly, the server also provides the public API by which the python client interacts with the web client viewer.
Added functionality is implemented by a plugin system that can be mixed in with either the client or server. There are many examples of plugins including blinking frames, computing statistics, and an imexam clone. Plugins are, naturally, implemented in JavaScript.
JS9 is still under active development, though has essentially one contributor.
JS9 is not a feature for feature clone of DS9. It does, however, have a mode in which you can run it as a desktop app (I presume via electron or something similar). There was a conscious decision to minimize keyboard interaction since this was viewed as web native, so likely to be used on mobile devices. Pointer gestures are still used in many of the same was as in DS9.
Some aspects feel a little clunky.
E.g. scroll zoom doesn't seem to work for me in Chrome, however there are hot keys + and - to do that.
You can spawn multiple images in different windows that float around your browser, but they are hard for me to keep track of and can cover other menus.
As far as I can tell, the concept of frames is not the same in JS9 as in DS9.
Instead of having named frames, the primary id is the image file name.
I'm not sure what implications that has, but when trying to overlay a mask, I wasn't able to figure out trivially how to specify a specific image to put it on.
The images loaded in a single web client seem more connected than in DS9.
Specifically, one can load several images into a single pane as a stack, but then via a menu scatter them each to their own pane.
Once scattered, they can be syncd meaning the same operations apply to all sync'd images.
This can be reciprocal or one way and the actions that are sync'd are configurable via the UI or the API.
I was generally able to do all the things I expect to do in a single frame image viewer. Since the image manipulation is handled locally, everything feels very snappy.
As mentioned there is no scroll zoom.
It instead requires interaction with a menu or using the + or - hotkeys.
There is no hotkey for "zoom to fit".
These are selectable from a menu.
The scale parameters can be chosen from a menu. The standard pointer gestures work for fine tuning the parameters.
This was very responsive and easy to find in the UI.
The blinking is handled by a plugin. Blinking is controlled through a popup menu that allows one to pick which images to blink and at what rate. There is also a manual blink button that scrolls through the stack of images which I found surprisingly useful.
This was a little trickier than it is with DS9. First, all the images need to be put in their own panes. Next, one can reproject all the images to blink onto one of the WCS of one of the images. Then, the images need to be sync'd on pan and zoom. Pan and zoom to the desired location. Gather all the images into a single pane. Blink through the images. For some reason syncing doesn't work when the images are gathered.
I tried all the basic things that one can do via afwDisplay.
The functions in the public API are well documented.
I have no idea how this works and may be a problem.
I point my browser to the js9 test server end point.
Then, construct the JS9 object with the same end point, and somehow it links the python session with my browser session.
This means the browser and the python session must be in the same environment?
I'm frankly not sure how this works.
This was very straight forward.
I read one of the LSST exposures with astropy.io.fits.open and then called js9.SetFITS(hduList).
It took about 20 seconds to load a 4Kx4K exposure.
Calling this function again with a different loaded fits file adds the image to the image stack rather than replacing the previous one as suggested by the method name.
Once loaded, all the functions I tried worked fine from the UI.
I only tried loading an astropy FITS object, but there is also a SetNumpy method that seems like it should be able to take an array, though in that case I don't know how to send the metadata.
I was able to set the color map and scale the image from the python prompt. I also was able to pan and zoom. There were some quirks, like zooming reset the color map to gray scale from green, but maybe I did something wrong.
I was able to place regions, but they always showed up at the center of the image. I must be misinterpreting the API and will have to investigate further.
I think that JS9 is a very ready stand in for DS9 in the browser for most of the basic interactions people want to do.
Given the client is written in JavaScript I think it is probably easy to make it a JupyterLab extension.
It's less clear to me how to also support the node.js functionality (though Adam thinks it's probably not hard either).
All code is on GitHub and is active, but maintained by a skeleton crew to say the least.
JavaScript coding is required to contribute.
I'm not sure how much of a blocker that is.
Most people I know who code in JavaScript like it a lot.
The public API appears to have all the necessary tooling to implement a full featured afwDisplay back end.
However, it was not immediately obvious to me how to implement mask overlays.
That being said, they are mentioned several times in the documents and I have seen other apps using JS9 to overlay masks.
There is some general weirdness that I think we should report and see how it's handled.
- When I quit my python interpreter, I had to
ctrl-Cto get back to the command line. - I saw some JavaScript errors when some callbacks seemed to get unregistered
- The difficulty of locking two images to the same WCS and panning or blinking was disappointing (though all the functionality is there).
- Not being able to address individual planes in the image stack may be a problem.
@SimonKrughoff, I just came across this, so in case its still current, here are some very quick comments:
You can turn on scroll zoom via the JS9.globalOpts.mousetouchZoom parameter, either on a site-wide basis via js9Prefs.js or for a single user via the Preferences plugin. (I find scroll zoom to be a pain, so I turned it off by default).
Keyboard interaction is controlled either via the JS9.globalOpts.keyboardActions parameter, accessible (as usual) site-wide via js9Prefs.js and per user via the Preferences plugin. We have pre-defined a number of actions, but you can trivially add your own (e.g. zoom to fit).
Regions are always placed in the center to start. DS9's "click to make a region" default was a huge problem generating bogus regions when clicking a window to focus it. If you want to place a region in specific locations, make one in the center, resize it as needed, then select and copy it with 'r', then paste it repeatedly with 'p'.
I also think DS9 got it wrong on frames: so far as I know, if you naively load three images into DS9 and then want to RGB blend them, you cannot do so without reloading them into the RGB frame. Mosaics also have to go into special frames. These restrictions were removed in JS9: you just load images into a frame and then blend or mosaic them as you like, when you like. You also can load images into separate frames, like having multiple DS9 running but with more communication between them (e.g. you can move images between the displays, align the displays, etc.). Separate displays are identified by div id (which allows the API to target specific ids easily).
I'm not sure what problems you are having with masks (not even sure whether a mask is an image or a set of 2D regions) but if you still are having trouble loading them, please let me know the details. It has to be trivial, unless I am missing something. And if I am missing something, we'll add it.
WCS alignment needs work: I took it as far as I could without serious community feedback. I'll certainly improve it as needed, so let me know what you want.
The issue of my being the sole developer is a problem (though I note that DS9 has had the same problem for about 15 years). I'd love to have help from collaborators, which is why it is in Github. If LSST has JavaScript experts who are interested, we should talk more. In general, I would expect that a big project like LSST would want more involvement in JS9 core development and I agree to this idea.
cc: @frossie