I need to write this up in more detail soon, but in chatting with Patrick, it is clear that 140 chars isn't cutting it.
The way I've been thinking about the web on TVs and consoles is something like this:
- What are the design patterns and interaction models that would make for a successful experience?
- Can we build those experiences now? If not, what would need to change to make it possible?
In my mind, this starts with the basics of designing an interface that works well on a TV and can be interacted with using a remote control. Once we have the basics out of the way, we can proceed to multiscreen interactions.
There is a pattern for a basic TV interface layout. It is a grid of some sort. The Netflix interface is a good example.
On many TVs, you navigate around the grid using a d-pad. When you hit the end of a row and then select right on the row, additional items come onto the screen. Same with the bottom of the screen.
This is very similar to the way interfaces in the Smart TV's own menus work. And it makes intuitive sense to people.
Building an interface like that using web technology that would work across devices is currently very difficult. Here are some of the challenges I saw when I was testing TVs this year. These are in no particular order:
- Other than UA, you don't know if it is a TV or not.
- You don't know if the remote is using a pointer or a d-pad.
- If it is a pointer and your web page is built to respond to someone reaching the end of the row in order to push in new items, the page will never work for someone with a pointer.
- Oh, and a lot of TVs allow users to switch between pointers and d-pad in the browser which means you may or may not have an intuitive interface.
- To replicate that Netflix interface, you'd like to get rid of the Chrome. 2013 Samsung TVs support the full screen API, but it doesn't get rid of the chrome. Others you can turn on full screen in a menu that is buried. On the Sharp TV, you switch the setting to enable fullscreen and the next time you open the browser, the chrome is back.
- Samsung's browser supports the d-pad, but you cannot capture its events. It will only go from link to link. In fact, they call that Link Browsing mode and it behaves differently than any other TV browser that supports d-pad that I tested.
- The Panasonic TV browser has a persistent left panel with a picture in picture and some bookmarks that you cannot get rid of.
- The LG 60LA7400 allowed you to switch between pointer and d-pad and acted much like a Wii remote in your hand. However, the LG55LN5700 had a more traditional remote that only had a d-pad. Ironically, this TV only supported the pointer in the browser and did not support switching to a d-pad mode at all.
And these are just a handful of the things I saw when testing on TVs.
Really, they do.
The LG TVs I tested had better HTML5 and CSS3 test scores than my iPhone 4S running iOS 6 does.
Patrick wrote:
so you meant "we need tv browsers to support standards" rather than "we need new standards for tv browsers"? if so, agree!
I think it might be both. I'm not sure yet.
- At minimum, it seems that TV browsers could benefit from a recipe of standards that are important on TVs and as well as feedback on where the pain points are. "Hey, fullscreen API is a big deal. Stop screwing it up with your implementations." This is similar to how position:fixed to viewport became more important as mobile took off.
- It seems like there are defacto standards that it might be nice to make sure everyone agrees with and explicitly say we expect. For example, d-pad controls trigger the keycodes for arrow keys.
- There are clearly gaps such as the way to identity that a 10ft UI is approach for a given screen that beg for either a new standard or common implementations such as lying about device width that would allow developers to be successful.
- There are other gaps like being able to know what type of input device is being used and being able to define how the d-pad works that it is unclear to me whether they are new standards, existing standards, better browsers, or what.
And that's just to support the basic interface scenario. That's not beginning to look at the really interesting stuff that can be done with multiple screens or new forms of input:
All of which is why I said:
I don’t believe you can build something using existing standards that would provide a good exp across TVs & consoles.
Great stuff, this is indeed better than 140 chars ;)
A hopefully not too long response to your points, from my perspective.
Agreed. Many of the points actually seem to echo the other current discussion I'm trying to stay involved in, which is the "how can I detect if the user is using a mouse or a touchscreen", where my holistic answer is "design for both, it's not either/or anymore". And sometimes yes, having to still have a completely separate TV (or leanback, big screen, etc) experience may currently be the only option.
Yup, that's been my major bug-bear too. As I said, I still believe that a smooth solution would be for TVs to respond to viewport meta / @Viewport CSS directives, AND to effectively lie about their device-width/device-height and pixel density. From a user experience, the perceived size of a big TV at 10' is on par with that of a smartphone at 2', so regardless of the real-world pixel density, TV should indicate that it has a high density (2 or 2.5) to account for the perceived density at normal viewing distance. Immediately, that would make mobile-optimised layouts work on TV, including not just the layout, but affordances like larger buttons, larger text, etc.
And currently on desktop/laptop you don't know if a user is using a keyboard, or a mouse, or a touchscreen, and certainly can't say if they're using either one exclusively (see Chromebook Pixel, Win8 laptops with touchscreen - and there's no concept here of "primary" input either, as they're equipotent).
How do you currently account for things like carousels for users with mouse and/or keyboard and/or touch? Same principles apply here. This is not a new problem (the keyboard accessibility issue has been a constant one forever), just one that designers seem to just find out about in a roundabout way ("what, there's no hover on touch? what now?" when there's not been hover - but rather focus - for keyboard users forever too).
In this specific case, make affordances like tiny arrow areas to move back/forward in the carousel? Make them only work with mouse/pointer, if you must. If the user is firing key events, hide them...but bring them back in view when a pointer is moving? Options are there.
See above. Either design an interface that has elements for both modes, or interactively (depending on last used input mode) show/hide them gracefully.
Not so dissimilar from what keyboard users have been doing on desktop forever. Or how blind users with VoiceOver on iOS, or TalkBack on Android, experience most sites. Linearly tabbing back/forward through focusable elements in source order. In theory, we should already be accounting for that in our sites.
Now this is getting to a more fundamental area. You, as a designer, have decided that your app would be best experienced without any distracting chrome (this is along the same lines as that advice from the comedy Google TV UI app to not use anchor elements because they'll show a "distracting" tooltip/hint about the link's target), and that it needs a certain input modality. But truth is: your site won't live in a vacuum. You may decide that it would look best without any chrome or work best with d-pad controls, but this is not necessarily what users on that TV will know from any other site they visit. You can argue that they're getting a crappy experience on all other sites and that YOUR site will make it better, and lament that TVs don't give you that control...but this really reminds me of the old days when sites launched chromeless giant new windows (ideally self-centering on your desktop and then expanding to take over the screen.width and screen.height ... remember those?) to control the whole user experience. It's a shade arrogant to say "I know that platform conventions are different, but I want my site to look like this on your TV".