Skip to content

Instantly share code, notes, and snippets.

@tomcrane
Last active September 17, 2018 16:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tomcrane/e03d5b0405cb23f937ef86aa8f2ae575 to your computer and use it in GitHub Desktop.
Save tomcrane/e03d5b0405cb23f937ef86aa8f2ae575 to your computer and use it in GitHub Desktop.

Updated 2018 - Please see http://canvas-panel.digirati.com/#/about for later developments. The conversation on this gist is kept here for its usefulness to posterity...


There is a resuable component that sits between tile renderers like OpenSeadragon (OSD) and manifest-level viewers. This component can be reused in very simple IIIF-powered applications - a few lines of JavaScript - but is sophisticated enough to form the basis of custom applications with complex layout and annotation rendering requirements.

I'm not quite sure what its external interface looks like yet - but it's the component I'd reach for to make an ad-hoc IIIF powered mini-application instead of OSD because it understands canvases directly. Under the hood it's wrapping OSD (or another tile renderer).

This is a component that provides a surface for rendering one or more sc:Canvas objects and the image annotations on them. So it's both a library (or combination of libraries) and a UI component that takes up screen space. It takes care of all the rendering, relative positioning, scaling, support for tiled and single images, oa:Choice, oa:SpecificResource etc within a canvas, but leaves the layout of multiple canvases under your control.

It's like OSD, and feels a lot like OSD for the developer - but it deals with sc:Canvas resources "natively" rather than tileSources. A likely first implementation would be a wrapper around OSD.

It could be instantiated in a similar way to OSD:

cp = canvasPanel({container: $('#cp')});
cp.addCanvas(myManifest.sequences[0].canvases[0]);

By default it would render the canvas to fill the world - but just as with OSD multi-tile source you can drive it for any kind of positioned or animated layouts - masonry-style, whatever. It helps you with layout in the same way OSD offers a world for multiple tileSources - but it's your decision (e.g., if you want to build things like http://iangilman.com/openseadragon/flickr/, you can).

It can also plot the bounding rectangles or svg shapes for any other non-image annotations you load (whether by following otherContent or loading arbitrary external lists), but leaves the rendering of the bodies of those annotations up to you. It gives you hooks for point annotations, hotspots for linking etc. It is the natural foundation for building a bespoke presentation of densely annotated material where the annotation bodies are rich content - it is not concerned with how you render those bodies (although makes it easy to overlay if you want to render on top), but it takes care of managing bounding rectangles or shapes and drawing and scaling them within a "world" of one or more rendered canvases.

It offers a capture surface for points, rectangles and more complex shapes, but leaves everything else up to the containing application. That is, it can be switched into a mode that a containing application could use to capture an annotation from a user. Not sure if it supplies its own drawing tools.

The visibility, appearance, animation effects etc of any annotations it displays are completely controllable externally. It is NOT concerned with how you capture the body of an annotation from a user (e.g., text boxes, forms etc). That's your UI to worry about.

It might be used as the rendering surface of a viewer that understands things at the manifest level - e.g., it could be the centre panel of UV or Mirador), but on its own it doesn't know anything about ranges, sequences or other presentation API resources. You would need to build an application and bring in other libraries for that.

While the containing application is responsible for understanding how annotationLists are grouped into layers, the interface exposed by canvasPanel makes it easy to show and hide individual annotations, whole lists, or sets of lists. That is, show the bounding shape and offer affordance for selection. How they are shown and hidden (fades, animations etc) needs to be externally controllable.

// not really sure what this looks like yet... 
cv = myManifest.sequences[0].canvases[0];
cp.addCanvas(cv);

// a wrapped annotation ref?
cpAnno = cp.show(cv.annotationLists[0].findById("@..."));
cpAnno.hide();

// and a list
cpAnnoList = cp.show(cv.annotationLists[2]);
cpAnnoList.hide();

// I have my own annos on this canvas as well
myAnnos = loadExternalAnnos("http://myannos.com/list");
cp.show(myAnnos);

Relationship to other libraries; manifesto(r)

It doesn't need to know about manifests and collections, it's a surface for rendering canvases. Complex layouts are driven externally. It clearly intersects significantly with iiifManifestLayouts. In fact you could take iiifManifestLayouts and reshuffle it - the code in canvasObject.js, canvasUtils.js, imageResource.js, imageResourceFactory.js etc becomes the canvasPanel wrapper round OSD and the manifest(or)-level code then lays out the canvases on the panel via the canvasPanel abstraction rather than directly.

canvasPanel is an interface. It fires events, it exposes functions. You can make an OSD version, a leaflet version, whatever. But the abstraction level of its interface is at the canvas level, not the tilesource level. This is what makes it powerful for quick IIIF-powered user interfaces.

@aeschylus
Copy link

This is meant to be IIIFManifestLayouts. The layout functions themselves are pure functions, and have been broken out into this library, which needs a better name. We certainly need some refactoring to separate the bits of OSD-specific code that have crept into the canvasObject/canvasUtils/imageResource files, with a focus on events. I would also be very happy if the single-canvas components of this were separated into yet another separate component with its own tests, demo pages and self-contained API.

Please see these briefs [1] (the thumbnail generation part of that needs to be rewritten, see #116, [2] about these components of IIIFManifestLayouts.

@aeschylus
Copy link

aeschylus commented May 26, 2016

Just as we should delete the layout component from manifestLayouts and then include it from that other repository where it is separately managed, we should also delete the single-canvas element and separately include it. This will encourage us to be very deliberate about what state is accessible by which components at what times, and about the ultimate API each component exposes.

One concern I've had on my mind for awhile with regard to these components is the tension between keeping all state completely outside of the rendering components, while at the same time making some complete level of functionality possible when a component is used in isolation. It is enormously useful and powerful to completely separate state and rendering, but there is some finesse necessary when working with self-contained components. See: http://staltz.com/unidirectional-user-interface-architectures.html

@tomcrane
Copy link
Author

Thanks Drew. I think we're talking about the same thing here, and sul-dlss-deprecated/iiifManifestLayouts#40 (comment) certainly sounds like it. Especially the notion of semantic zoom which helps with densely annotated material in the style of Google Maps - with complete control of behaviour from outside the component.

canvasPanel + bespoke script = ad-hoc viewing experience - go create!
canvasPanel + manifesto + bespoke script = a custom viewer when you need to do a bit more with the Presentation API
canvasPanel + manifesto/r + navigation + thumbs + x + y + z... = UV, Mirador etc
canvasPanel + bespoke script + anno protocol client = ad-hoc annotation environment - "tag my painting"

I can take OSD as a component and incorporate it into lots of radically different user interfaces quite quickly. I feed it tileSources, position them, and draw styled overlays on top.

I can take canvasPanel as a component and incorporate it into lots of radically different user interfaces quite quickly. I feed it canvases, position them, and feed it annotations to render. Its API allows me to offer control of image choice, anno visibility etc., to my users as I see fit. I can control the visibility of annotations and animate fade ins, fade outs etc in response to changes in zoom level.

If you can drive canvasPanel through its API, what happens when it is used by another component such as the UV? I also want to be able to drive the UV externally - load this manifest, go to the third canvas, show image choice 4 and draw these 3 annotations on top. The first two of those are jobs for the UV's API, the second two are jobs for the panel under the control of UV. Does the UV offer the panel's API directly? Does it wrap its API in a facade?

You can imagine where I'm coming from with this. We want to consume IIIF resources in lots of different bespoke discovery environments, sometimes a lot more free-form than a "reader" application. My motivation is to build such applications quickly, and not reinvent image, canvas and annotation processing code every time. I'll spend the time (and therefore budget) on UX around the panel instead.

@aeschylus
Copy link

I've been freed up to work on IIIFManifestLayouts for the next 2 weeks, so this will be a central part of that work.

We are definitely on the same page. ManifestLayouts/[this-as-yet-named-single-canvas-component] are to the Presentation API what OpenSeadragon is to the Image API. The work we (Digerati, Mirador community, Bodelian, etc.) do over the next 6 months should make it possible to build a bespoke Mirador/UV in a few days, and something slightly simpler in a matter of hours, without locking people into 50 other unwanted features. There are going to be applications we can't anticipate, and those will be much easier to build once we release the base libraries people need to focus on their particular domains.

@tomcrane
Copy link
Author

This is the kind of thing that a canvas renderer with annotation support should enable:

https://tuinderlusten-jheronimusbosch.ntr.nl/en

(not IIIF, but everything it does could be accomplished in the model).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment