IxIF-Notes.md

General

sequence -> canvases

OK for video. A canvas with a time dimension. "duration"... (dc?)

Is "canvas" general enough for audio, born digital, 3D?

example sc:Sequence - a sequence of sound clips, video, 3D etc.

Argument for "canvas": it's a viewer. It takes up pixels in the browser. There's always something to paint - a progress bar.

I think this is problematic for audio. Providing a canvas to "paint" a scrubber seems artificial; the audio being represented has no spatial dimensions, only a temporal one. A scrubber is client-chosen UI, like Seadragon's navigation window. There might be a poster image for the audio track, but putting this centre stage in the model to have something for the canvas to do doesn't feel right. The poster image is an annotation but there's no canvas to paint it on, a viewer (or audio player) can do what it likes with it. A player doesn't have to take up pixels in the browser, and if it does, those pixels are not representative in the same way image canvas pixels are.

Example - a manifest consisting only of audio could be played on the radio...

• In the Image API, the canvas is the abstract space, a coordinate space for us to paint our annotations.
• For sound, the ??? is an abstract duration, a timespan within which we ??? our annotations.
• For video, the canvas? is an abstract space with a duration, an x,y coordinate space + t. Video is a combination of the reqts of image and audio (time-based with spatial extent)
• For Born Digital, ??? ... (e.g., PDF, probably well understood from OA; hypothes.is, pdf.js, etc... look at that)
• For 3D, the stage? is an abstract x,y,z space for us to paint our annotations
• For time-based 3D, the stage? is an abstract x,y,z space with a duration t

media	type
Audio	?*
Image	Canvas
Video	Canvas, Screen
3D model	Space, Stage
3D animation	Stage

*I'm sure there's an elegant word for this

Either we overload canvas too much, or we pick nouns that people in the audio, video and 3D communities recognise from their own tools and models.

"stage" could work for audio too...

canvases: {
  @type: "@id",
  @id: "sc:hasCanvases",
  @container: "@list"
}

canvas -> images

images: {
  @type: "@id",
  @id: "sc:hasImageAnnotations",
  @container: "@list"
}

how about...

• Canvas is a subClass of "Xxx"

• sc:hasCanvases is a subProperty ixif:hasXxxes

• sc:hasImageAnnotations is a subProperty of ixif:hasZzzAnnotations

• dctypes:StillImage subClassOf ixif:Zzz

and...

• dctypes:Sound subClassOf ixif:Zzz
• dctypes:MovingImage subClassOf ixif:Zzz
• dctypes:PhysicalObject subClassOf ixif:Zzz ?? OK if 3D model is of a real thing - what about a dragon?
• moving 3D...
• Born Digital (text???)

Media Frags

http://www.w3.org/TR/media-frags/#fragment-dimensions

Is it reasonable to simply extend xywh to xyzwhd?

In the Image API, the canvas has width and height properties. But these are NOT pixels. The canvas width and height properties serve 2 purposes:

1. They give the client the aspect ratio
2. They provide an integer coordinate system for friendly annotation targets (no floating point arithmetic)

By convention (a SHOULD) the canvas dimensions match the pixel dimensions of the largest image (which for most people is the only image); this makes it easy for humans.

This seems equally applicable to video. In fact it even helps with anamorphic etc - if a video annotates the whole canvas, the client must scale it. You could force a correcting distortion on an incorrectly described anamorphic clip this way. We still want an integer coordinate system for painting annotations, even if the w and h of a VIDEO aren't as clear cut as for an image. It may be more typical for a manifest author to simply pick a width and height that clearly express the aspect ratio AND give plenty of room for painting annos without straying from integer arithmetic, e.g.,

width: 4000
height: 3000

...even though none of the annotating videos have these pixel dimensions.

Does it work for 3D and moving 3D? Is it still desirable to have an integer coordinate space for easily readable anno targets? It does if the anno targets are geometric shapes (cuboids mostly), but what if the anno target is a portion of a surface texture? It may be very laborious to describe the patch of surface in the coordinate system of the stage.

My stage is set up (for example) with the coordinate space thus:

width: 10000
height: 8000
depth: 5000
duration: "1042"

width, height and depth are dimensionless properties. As in the Image API, they are NOT pixels It is more obvious that they are not pixels in a 3D stage. They give the stage its proportions and set up a coordinate system for annotations to target. Duration, however, has a time dimension and should be given with the syntax used in media frags. The above would be interpreted as 1042 seconds, and so would

duration: "npt:0:17:22.0"

Video

Use cases:

• (put Clipper use cases here)

• user collects clips

• The videos that annotate the canvas are Youtube, Vimeo, etc

• User adds anno

• Subtitles!

• A subtitle is an annotation; DURATION only, it's up to the viewer what it does with it.

• But close-caption subtitles DO have an xywh too

• See http://www.opencast.org/ - subtitling...

• Subtitles as JSON-LD value objects... lang...

Audio

• BL Save our Sounds
• Talk to Andrew Hankinson
• Talk to BL