Modified `<picture>` syntax to address concerns

alternate.md

Proposal

What if media queries and breakpoints could stay in the .css file?

What if the image source was chosen based on the min-width, max-width, min-height, and max-height of the <picture> element instead?

Media queries would apply size restrictions to images based on css classes.

The element itself would only be responsible for providing a collection of URLs, described by their size, in pixels, and their mime-type.

The UA would be able to choose the best match based on the pixel density muliplier and available bandwidth.

This means more work for RICG, as we need to specify a reccomended selection algorithm, but I believe that's something we need to do anyway.

Old syntax

<picture alt="Description of image subject."> 
  <source srcset="small.jpg 1x, small-highres.jpg 2x"> 
  <source media="(min-width: 18em)" srcset="med.jpg 1x, med-highres.jpg 2x"> 
  <source media="(min-width: 45em)" srcset="large.jpg 1x, large-highres.jpg 2x"> 
  <img src="small.jpg" alt="Description of image subject."> 
</picture>

New syntax (css)

@media (min-width: 18em){
    picture{
        max-width:320px;
    }
}
@media (min-width: 45em){
    picture{
        max-width:800px;
    }
}

New syntax (html)

<picture alt="Description of image subject."> 
  <source src="small.jpg" width="300" height="200" />
  <source src="med.jpg" width="600" height="400" />
  <source src="large.webp" width="1200" height="800" type="image/webp" />
  <source src="large.jpg" width="1200" height="800" />
  <img src="small.jpg" alt="Description of image subject."> 
</picture>

Benefits

• DRY and separation of concern - breakpoints do not have to be specified in-markup
• Fewer image variations have to be described - pixel density is handled intrinsically (large is typically a hi-res version of medium, etc)
• No complicated parsing. One value per attribute - simple!
• Highly forward-flexible - CSS properties such as 'picture-select: favor-large` can be added to provide browsers with suggestions or overrides (logos, for example, we might prefer to always be highres, while content pictures can degrade on slow connections, etc). Browser vendors have shown CSS is much easier to innovate with than markup.

Prefetch considerations

(Nearly) as efficient image prefetching should still be possible under the following conditions:

1. The CSS breakpoint rules do not use percentages values for min-width, min-height, max-width, or max-height.
2. Those CSS rules are placed at the beginning of the CSS file, with the fonts.
3. (optional, for small gain) The last breakpoint rule includes a CSS flag precluding additional rules, such as picture-select:now;.

These preconditions seem easily achieved, and in an optimal scenario would only delay prefetching by ~15ms. Since absolute dimensions are provided, layout (and even render, for some UAs) can occur earlier than would be possible with the original design. The time till fully loaded could still be affected, but seems unlikely, considering the average responsive site (in my experience) uses over 35 assets and the pipeline would be full anyway.

In a worst-case scenario, image prefetching would be delayed until CSS was downloaded, but it would not be delayed until after layout, unless percentage units were used. Performance in an average scenario should be identical to CSS background images.

I think the original proposal is guilty of premature optimization. Reaching a wider user base with a simpler solution (which can be optimized) seems more in-line with the goals of HTML.

Addressed use-cases

See the Use Cases document

• 3.1 Resolution-switching - UA knows the multiplier, and can evaluate whether to apply it or not
• 3.2 Art-direction based - Full control based on element max-width and max-height from CSS media queries
• 3.3 Design breakpoints - Much better support than existing proposal, as they go where they belong - in CSS
• 3.5 Relative units - inherently addressed
• 3.6 API to manipulate sources - Vastly simplified vs. original proposal.
• 3.8 User control over sources - not affected

Nearly addressed use-cases

• 3.4 Matching Media Features and Media Types
• 4.1 The solution MUST afford developers the ability to match image sources with particular media features and/or media types - and have the user agent update the source of an image as the media features and media types of the browser environment change dynamically.

Solutions to 3.4 and related 4.1

This solution proposes resolution-based selection as the primary mechanism instead (or in combination with - see below) of a more generic 'media feature' query.

Look at the media query spec.

All the media features are essentially resolution indicators. Combining them into sane rules is not simple, and duplicating those rules for every element (versus in a CSS file) does not make sense.

The only media features that don't end up as resolution indicators are:

• 4.8. color
• 4.9. color-index
• 4.10. monochrome
• 4.12. scan
• 4.13. grid

I cannot invent any use-cases for scan, grid, or color-index.

If, in the future, 48-bit color becomes the norm, then I could see a use case for color

monochrome is the only non-resolution media query with a current, valid, use-case, AFAIK.

As mentioned in the use-case specification, many monochrome images require completely alernate imaging (and as such, will likely require alternate alternate text and perhaps alternate associated captions). I don't anticipate monochrome versions being provided frequently for other scenarios, as current UAs (Kindle, Nook, even 1996 eBookwise) are quite capable of converting color to grayscale.

I wonder if this single use-case deserves such an inflated and over-generic <picture> element?

Solution A (css)

    @media all and (monochrome) picture.monochrome{
        display:inline;
    }
    
    @media all and (monochrome) picture.non-monochrome{
        display:none;
    }

Solution A (html)

<picture class="monochrome" alt="Description of monochrome alternative ."> 
  <source src="monochrome-list.jpg" width="300" height="200" />
  <img src="small.jpg" alt="Description of image subject."> 
</picture>
<picture class="non-monochrome" alt="Description of image subject."> 
  <source src="color-pie-chart.jpg" width="300" height="200" />
  <img src="small.jpg" alt="Description of image subject."> 
</picture>

Solution B

We restore the media attribute to the the source element, but suggest that it only be used for non-resolution related queries. We will still have to define the interaction of rules in this case - so this could delay the spec.

Summary

From the use cases I have seen, this solution solves 95% of use cases with much simpler markup and better presentation/markup separation. It is also much more accessible to markup authors and content editing tools, as it is just a "list of images and description". The CSS class of the picture element permits pre-designed layout rules to be used, so that content authors can provide responsive images without a full understanding of media queries and layout rules.

It also does not preclude the remaining 5% of use cases, as shown by Solution A & B.