Skip to content

Instantly share code, notes, and snippets.

@Sakurina

Sakurina/wwdc-2019-session-notes.markdown

Last active December 3, 2019 10:20
Show Gist options
  • Save Sakurina/f96a33f26b8aeaa965e3298c4aefb7f6 to your computer and use it in GitHub Desktop.
Save Sakurina/f96a33f26b8aeaa965e3298c4aefb7f6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
wwdc-2019-session-notes.markdown

Introducing Desktop Class Browsing on iPad

  • What does desktop class mean?
    • Getting more done in Safari
    • New download manager
      • Download and upload files in the background while working on other tabs or other apps
    • Full-page zoom, hiding toolbar, per-site preferences
    • All the keyboard shortcuts you expect from a desktop browser
    • At the heart of it all is browsing desktop websites
  • Apple has been working on Web browsers for a long time
    • When iPhone launched, all websites were designed for desktops and the browser was made to make those sites work on a mobile device
    • Then new Web APIs happened to take advantage of mobile devices' unique capabilities
      • Responsive design techniques look great on iPad today
      • Also fairly common to have two separate versions of the site, one for mobile devices, and one for desktop devices
      • iPad Safari now presents itself as a Mac via its user agent string
  • Some desktop websites are designed with high information density but the iPad's default viewport made those websites appear zoomed in
    • And this viewport was shared amongst all sizes of iPad, which could make sites look fine on iPad or iPad mini but goofy on the iPad Pro
    • Viewports will now match iPad screen sizes so web sites can make better use of screen real estate
  • Some desktop websites are designed for mouse input
    • New in iOS 13: better emulation of mouse input via touches
    • So websites that are overly reliant on hover states now work out of the box
    • If you want to design for touch specifically, the pointer events API is now supported in Safari
  • How your apps and websites can take full advantage
    • For app developers: how to make use of full desktop browsing in your app
    • For web developers: how to make websites work even better on iPad
  • Starting with apps
    • 4 common ways apps use web views
      • Following links
      • Web browser
      • Hybrid apps
      • Authentication
    • Following links
      • Best way to follow links is to use Safari View Controller
      • Brings the best of Safari's features to your app like Reader and AutoFill
      • Safari View Controller gets desktop-class browsing for free
      • Chooses the most appropriate browsing mode for the size class (so mobile views are used in narrow multitasking views or on iPad mini)
    • Web browser
      • First, build your app with WKWebView and compile using the iOS 13 SDK
      • Specify an application name using applicationNameForUserAgent on WKWebViewConfiguration, this will get fed into the user agent that will dynamically adapt to the appropriate content mode
        • Use this instead of the customUserAgent property
      • WKWebpagePreferences has a preferredContentMode property you can set to "recommended" for auto detection, or "mobile" or "desktop" for a full override
      • To allow user switching or have per-site controls, you can return a custom instance of WKWebpagePreferences via a new method on WKNavigationDelegate
    • Hybrid apps
      • Use WKWebView, build with iOS 13 SDK
      • Test your app, and if needed, force the mobile mode in WKWebpagePreferences
    • Authentication
      • ASWebAuthenticationSession is the best way to authenticate
      • New OAuth-friendly SafariViewController derivative that presents in a form sheet
      • Just like Safari View Controller, all the work is done for you
  • Demo
  • Designing websites for iPad
    • If you have a responsive Web site, very little should change but we have some new tools to maintain and improve responsive sites
    • New best practices for iPad Web development, regardless of whether you are a responsive site or a desktop site
    • Six new features to discuss: some are developer features, and some are end-user features with developer implications
    • Pointer events
      • Reconciling mouse and touch input is messy
      • On iPad, Safari tries to keep mouse functionality intact for websites that are primarily reliant on mouse events
      • When a user taps, mousedown/mouseup/click events are dispatched as they were before, and hover does as well
      • However, mousemove makes no sense on iPad
      • We tried to emulate mousemove events at the same time as touchmove for compatibility but it interfered too much with scrolling to be viable
      • If you want something like mousemove, you should use the pointer events API
      • A Web standard that offers abstraction between listening to input and the input method used, making it easy to react to both mouse, touch, or pen events
      • Instead of using mouse or touch-specific event names, substitute the name of that input method with "pointer"
      • Use feature detection because not all browsers currently use this!
      • PointerEvent is literally a subclass of MouseEvent, so unless you want to use new pointer event-specific features, there's nothing else to do
      • Use either PointerEvent or MouseEvent on a given action, but do not register both as both will be called!
      • Use the pointerType property on the event to determine if it is a "mouse", "touch" or "pen" pointer
      • On Mac, use preventDefault() in the body of the event listener to prevent interference from browser native behavior like text selection
      • On iOS, set the touch-action CSS property to none on the div responding to the pointer events
        • Easier to use than writing JavaScript to handle the default prevention, and allows a more granular range of options (like disabling scrolling, but keeping zoom intact)
        • Declarative, so it is less performance intensive than the JavaScript equivalent
    • Hover
      • WebKit also sends hover events for compatibility but they are tricky
      • Hover detection was changed in iOS 13 to improve compatibility
      • Hover is sent when you tap something and if that event has caused what WebKit considers to be a meaningful change on the page, it will stop at the hover event instead of sending along mousedown/mouseup/click events like it used to do on previous OSes
      • The second tap will trigger mousedown/mouseup/click
      • This should greatly improve compatibility with websites that rely on navigation where categories display subnavigation on hover, but category names are also clickable links to landing pages of their own, which were problematic on older versions
      • Heuristic-based and tries to determine design intent, so it is imperfect
      • Hover best practices:
        • Provide another way to access hover content: good for accessibility and is a nice fallback for when changes are too minor for Safari to consider them meaninguful
        • Avoid using hover for common interactions as it forces that action to take 2 taps and slows down the user
        • Don't register timers in your hover event, because Safari will wait for that timer to end in order to detect if there was a meaningful change in page contents before resuming event handling
    • Accelerated scrolling
      • WebKit always had hardware accelerated scrolling for the main window contents
      • In iOS 13, subframes and overflow: scroll regions also gain hardware acceleration for buttery smooth scrolling
      • Better support for subframes
      • Old techniques for fast scrolling are now unnecessary
        • -webkit-overflow-scrolling: touch
          • The implementation details of this method had implications on the Z-stacking of CSS elements and could break things which is why it could never be made the default
          • This CSS property no longer does anything at all on iOS 13 because it is no longer needed
        • TouchEvents: JavaScript libraries to emulate fast scrolling via TouchEvents
    • Viewport and text sizing
      • Web sites not designed for iPad should display to fit for iPad
      • No horizontal scrolling unless designed that way
      • All text should be legible for most without additional zooming
      • Some desktop Web sites are designed for a fixed width that are wider than an iPad
      • Incorrect viewport declarations are all over the place: non-responsive Web sites are declaring a viewport appropriate for a responsive site and the user winds up with a website whose content is partly inaccessible offscreen
      • WebKit ignores the viewport declaration altogether if your site's content is wider than it should be for that declaration
      • If your Web site is designed for horizontal scrolling, add "shrink-to-fit=no" to your viewport declaration and the old behavior will be restored
      • If you want full control over the viewport and sizing of text, strongly recommend adopting responsive design
    • Visual viewport API
      • "Visual viewport" vs. "layout viewport"
      • The viewport meta tag configures the layout viewport, which changes on responsive sites when a window is resized whether it be on the Mac, device rotation or split view/slideover resizing on iOS
      • The visual viewport is the portion of the layout viewport that is currently visible and unobscured onscreen when, say, a keyboard is presented atop a Web site
      • Our new API allows you to move controls to a more appropriate and accessible location when a keyboard is up
      • Register to the "resize" event on the window.visualViewport object
    • Streaming video
      • Use HTTP Live Streaming
      • HLS is available everywhere
      • Easy solution to a hard problem
      • Plays nice with CDNs
      • Gives you behavior for free like AirPlay support
      • However, some Web sites use media source extensions (MSE) which give more explicit control over the data being transferred
      • Enables things like dynamically selecting the resolution of video in response to bandwidth changes
      • MSE is supported for desktop sites on iPad in iOS 13 so existing MSE engines should just work
  • Best practices
    • Build one responsive website instead of parallel mobile and desktop sites
    • Use feature detection instead of user agent sniffing
      • Especially since iPad is a chameleon that can go in and out of iPad and desktop user agents
      • And with UIKit apps running on the Mac, you can't really be sure iOS-like user agents are really running on iOS anymore
    • Don't use Flash
      • "When we say iPad has desktop-class browsing, we mean modern desktop-class browsing, so that means no plugins"
      • Flash won't work in Safari at all in 2020
    • Let users decide if they want audio
      • WebKit prevents autoplay audio by default
      • Do not assume your audio will autoplay
    • Remember that some desktop browsers don't have mice
      • Hover should just be for decorative purposes, and not functional
    • Use built-in APIs
      • For example, if you want to customize text selection, instead of trying to reverse engineer how to detect it from basic interaction events, use selectionchanged events
      • (Yanik's note: yeah but Safari's implementation of these has historically been very buggy and led us to abandon planned features altogether...?)

Advances in App Background Execution

  • Overview of Background Execution
    • What is background execution?
      • Specifically we are talking about the app running when it is not in the foreground
      • Not background threads when the app is in the foreground
    • Why do we enter this state?
      • A request by the app: downloads, periodic updates, finishing a task started in the foreground
      • Event triggers: geofence, new health data notification
    • Important considerations for background exe
      • Power, Performance, Privacy
      • Power
        • Whenever your app is running, it's using power, and power has an impact on battery life
        • When designing background APIs, we try to be mindful about how long the window of execution should be for the use case it is designed for
        • If you finish early, you can tell the background APIs you finished early and we'll suspend your application immediately to save power
      • Performance
        • Things need to say smooth
        • Fast app launches and responsive UI are paramount
        • Don't forget your app isn't the only one running in the background
        • Your app can run in the background while another app is in the foreground, or multiple apps can run in the background at once
        • Smart CPU and memory limits to limit the impact on performance
        • Be aware of these limits so your app doesn't get terminated and take longer to launch next time
      • Privacy
        • While a user may be aware of all the times your app is in the foreground, they probably aren't as aware of all the time your application is running in the background
        • Be transparent and let users know what data your app is using when it is in the background
    • Each use case has its own background execution API with its own balance of these three requirements
  • Best practices
    • Messaging app that sends messages, makes callls, mute threads, downloading past attachments
      • Send messages: Core functionality of the app
        • User expects immediate completion
        • This might be the common case but maybe the network is congested or the server is slow and this could take additional time
        • Nothing is preventing the user from switching away to another app or locking their phone and going on with their day
        • We need to protect completion so the message sends reliably
        • Use Background Task Completion
        • Apps get additional runtime in the background
          • UIApplication beginBackgroundTask
          • or ProcessInfo performExpiringActivity from extensions
        • Intended for completing work started on foreground tasks like saving things to disk or completing user-invoked requests
        • If the task fails to complete within the window alloted to you, use the expiration handler to notify the user that the background operation failed via a local notification
        • Don't forget to end the background task from your completion handler so the app can be suspended as soon as it's appropriate
      • Making calls
        • VoIP push notifications
        • Special kind of push that launches your app so you can present appropriate UI for responding to a call
        • When registering for push notifications, set your PKPushRegistry's desired push types to include VoIP
        • New in iOS 13: you must report incoming calls with CallKit in the didReceiveIncomingPush callback
        • If you don't, the app will stop launching your app for VoIP pushes, presumably to stop background abuse happening over the VoIP backgrounding API
        • Other tips:
          • Set caller info in the push payload to have all the info immediately to present a call UI
          • Set apns-expiration to something small like 0 (deliver immediately or fail) so you don't receive a push notification minutes after the call is no longer ringing
          • If you prefer banner presentation, just use a standard push notification
            • If you do end-to-end encryption, you can also use notification service extensions to swap the encrypted contents of your notification with the decrypted contents
      • Muting threads
        • Many different threads
        • User might not want alerts for a specific thread
        • Message content may still be relevant, just not told every time
        • Alert the device new content is in, but not the user
        • Background pushes
        • Send a push with content-available: 1 without an alert, sound, or badge payload
        • The push just serves as a hint to the system that new content is available so that it can schedule a content fetch in a more timely manner, but it does not guarantee an immediate content fetch
      • Downloading past attachments
        • Opening the messaging app for the first time on a freshly restored device
        • You might immediately download a snapshot of recent messages, but you may want to defer the download of older content until after the user leaves the app as to not impact the responsiveness of the app
        • Discretionary Background URL Session
        • Made for deferred downloads, and provides info to the system for smarter download scheduling
          • Timeout intervals
          • Time window for roughly when you want the download to happen
          • Suggested workload size
        • Other users: batching analytics uploads or photo backup
  • New BackgroundTasks framework which handles additional use cases not covered by APIs until now
    • Deferrable maintenance work: actively syncing state, database cleanup, backups to the cloud
    • Right now, people are abusing of the background task API to complete these tasks as soon as the app enters the background
    • Wouldn't it be nice if all of this could be deferred until later, potentially when the device is charging and idle?
    • This is what the BackgroundTasks API is for
    • Available on iOS, iPadOS, tvOS, iPad Apps on Mac
    • Background Processing Tasks mode
      • Several minutes of runtime at system-friendly times
      • Maintenance owork or Core ML training and inference
      • We are giving you the ability to turn off the watchdog CPU monitor for especially intensive work for the duration of your task
      • You can do these tasks as long as the background processing task was scheduled when the app was in the foreground or if your application was used recently
    • Improved Background App Refresh Task
      • New API with same policies: 30 seconds of runtime to keep the app up to date throughout the day
      • How often your app is launched and when is based on the user's usage patterns
      • Less frequently used apps won't get as frequent refreshes
      • UIApplication fetch API is deprecated and not supported on the Mac
    • BGTaskScheduler is your interface to the activity scheduler that monitors battery level, app usage, connectivity, etc...
      • Submit a BGTaskRequest to the scheduler from the foreground app or from its extensions when it is running
      • When the system decides it is appropriate for your task to begin, your app will be woken up and given a BGTask instance corresponding to your request
      • Your app can now perform whatever it wanted to do, and then set the task as completed. When all tasks are completed, the app will be suspended
      • If you schedule multiple requests, it's possible the application sends you both of these at once
      • Time is alotted per app launch, not per task, so you need to be able to do these things concurrently in the same window of time
      • Even if extensions sent the original BGTaskRequest, the BGTask will be sent to the main containing application, never extensions
    • Additional Considerations
      • Don't set earliestBeginDate too far in the future
        • If the user doesn't launch your app around that date, it might fall outside the set of recently used apps, and your task may never get scheduled
        • Try to keep it to a week or less
      • Ensure that files are accessible while device is locked
        • That is usually when these maintenance tasks will be executed
        • FileProtectionType completeUntilFirstUserAuthentication
      • UIScene apps should call UIApplication requestSceneSessionRefresh
        • This notifies the system that the screenshot of the scene must be updated in the multitasking app switcher
      • BGTaskScheduler submit is a blocking call, so if you call it on app launch, consider doing so in a background thread to keep app launches fast
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment