Address autofill - minutes.md

WICG: Address Autofill

• TPAC

Attendees: Westin, Martin Lechner, Yoav Weiss, Anne van Kesteren, Christian Indra, Rick Byers, Adam Rice, Michal Mocny, Dominic Batre

• Yoav: Discuss Address Autofill
• autofill is a key feature for login and web forms etc
• significantly reduces user friction on the web
• ⅓ of guest checkouts on shopify do (not?) use autofill
• 41% increase in checkout completion when used
• Issues… Autofill is dynamic by definition
• <<Showing example of an address form using a specific region, where there is complexity, e.g. vatican city is a country not a city and doesn’t have a city field…>>
• <<The complexity is limitless>>
• The way this is solved today is often to use Hidden fields, in order to capture these fields from autofill, and then move the values into the actual input fields
• In other words, we grab as much info from autofill as possible, and then read the data.
• This is also a privacy risk. users are filling in data that users aren’t even aware is being shared.
• This might be used by malicious actors… but its the only way to implement a solution today
• Related, there is difficulties with testing autofill and there was a meeting earlier today about this
• At Shopify, there is a team of 10 people just to deal with complexity here.
• Earlier Idea: Decouple autofill from the DOM. We don’t want this any more.
• New Idea: Enhance form autofill
• Example, <input type=”full-address”>
• Then, the whole address is a single atomic unit. This allows the browser to decide what is the best way to provide the values, without unpacking into hidden fields.
• Today, these decisions are being made for the browser by the site author, which isn’t great
• Idea: Autofill event handler
  • Perhaps handle/preview before the form is filled in (visually)
• <<Showing example code snippet>>
  • <form autofill=”a b c” onautofill=...>
• This could address the use cases which are already being polyfilled via hidden fields.
  • This would address privacy issues.
• Open Questions:
  • There is an (ISO?) standard for Address Scheme. Potentially needs to be versioned and needs to evolve over time.
  • Iframe Credit Card data has a similar issue… Today it is common for Each input value to be in a unique form in a unique iframe, but then coordinate data between each other!!!
  • :autofull pseudoselector, and visual indicators. Used to show to the user which input was updated by the browser. Developers can override. What happens if the page manually moved autofill around– now there is no way to apply this visual hint.
• Dominic: <>
  • It used to be yellow, now its blue, we made a change where you cannot style this as the page author, so, in Chrome you cannot override styling
  • So, the browser ignores styling and things like background color is ignored
  • But, other things, like height, can adjust.
• Anne: how do you maintain contrast then?
• Dominic: We have some conventions but I don't recall the details. Depends on the surrounding colors?
• Yoav: But js could still provide a signal to provide those magical backgrounds?
• Dominic: Technical perspective, yes, Policy perspective, not sure. It would allow websites to fill in input fields without the user being aware of it
• Yoav: Final Open Question:
  • Browser extensions currently can hijack the autofill experience. This can result in a worse experience.
  • Very heavy handed. Would a JS api help here? By allowing extensions to play a role, without breaking everything?
  • We haven’t done a whole lot of thinking about this yet
• Yoav: we would first like to gain broader support for the general use case. We aren’t tied to any specific use case.
• Adam:
  • Previous proposal allowed the browser to show a pop-up with autofill data, and then later dropping the results down into the page.
  • Maybe we need a “rich address” rather than a “full address”
  • Second, the browser can use alternative UI’s based on that
  • More privacy preserving
• Yoav: What does “rich” mean?
• Adam: Has a specific format and broken down… ??... it allows the browser to change how it renders the input field instead of just sending all the information down to the page.
• Dominic: As an example, the UI could look like something like a business card. User could also check what they do / don’t want to submit to the site.
• Christian:
  • Long term, maybe browser UI makes sense?
• Michal:
  • If I edit the form with browser and then submit, then I have to check the form on the page again
• Yoav:
  • That is one reason we moved to this new proposal
  • But also, the act of autofill is dynamic
• Domenic:
  • There is more complexity when you think about addresses as structure
  • Internally we think of this as a tree
  • << discussing example name / address data structures >>
  • This hierarchies are currently evolving
  • You submit a form, then you have a form after being submitted
  • After this, we try to “fill the gaps”
  • << Example of how we walk the tree to go from strings of text into hierarchy >>
  • Because this is error prone, we differentiate between inferred vs explicitly filled information
  • If the user ever changes an autofill value, we can update what we think we know…
  • << Examples >>
  • All this means, if we move towards JS api… some of the values are “in flux” and the information being provided is actually inconsistent.
  • << Example >>
  • This creates challenges if the site via JS tries to understand all this data.
• Yoav:
  • So, the website validating this information as a mediary might get in the way of the feedback loop that autofill is using to learn
• Domenic: Not really. There is some risk if it evolves into some ways where the form is completely obviated, then maybe.
• Stephan: We already have this problem today, where some sites will update the <form> dynamically…
• Stephan: One concern I have from a browser perspective…
  • If we give the user a preview of the data we are providing, and then give that to the page, and then the page doesn’t actually use the data correctly, then, it might look like the browser is broken
• Christian:
  • Maybe we can change the API so that websites don’t get to call preventDefault
• Yoav:
  • An alternative API shape... don’t use preventDefault, but instead massage the input into output and the browser uses the new information?
  • We are just validating and modifying the values and browser continues to do its thing.
• Anne:
  • To what extent is there a lack of agreement.
  • Do various parties agree on what makes an address an address and do we have agreement?
  • Parties are browser and website
• Domenic:
  • Websites make their own choices…
  • Websites dictate because they create their own apis and they require fitting within the constraints of the api
  • There are so many options that we struggle with defining constraints
• Martin:
  • There is an ISO session to try to address this
• Domenic:
  • I have little insights here, but someone sent me this information
  • The source is the Banking industry and partially to address use cases such as money laundering
  • Banks across the world working together on a common set of rules
  • It appears that there is a lot of good stuff in there
  • But, it seems to me that it might be hard to actually support all addresses I’m aware of into the schemas being recommended
  • << Examples, i.e. Landmarks as an example >>
  • << Example, In india the address of a pizza place I ate at was “Opposite of Google” >>
• Stephan:
  • Banking purposes might be a bit different than e.g. eCommerce purposes
  • One might only be using for ID/verification, while the other needs to deliver to a specific physical location
• Domenic:
  • My hunch is that if we try to sort all of this out, it might delay shipping anything forever
  • We’ve been trying forever and it hasn't been solved.
  • That's why I like alternative proposals
  • For example, maybe we need more interactive iteration with autofill
• Anne:
  • Provides an example, about disagreements
• Christian:
  • This is already a problem today and we end up changing it, etc.
  • ??
• Yoav:
  • If I understand what Dominic is suggesting, I think maybe it is:
  • Trigger the behaviour that the browser does, and then, have a bit of a “dialog” with the form that is more back and forth where you can change the shape of the form
• Anne:
  • That sounds reasonable
  • That sounds like to solves at least the Country selection issue
  • At the start you established how important it is to get addresses correct, but, it sounds like there are larger issues with how the data is stored and shared
• Yoav:
  • Domenic helped me understand that this can be potentially infinitely hard and continually evolving
• Domenic:
  • We have tried and failed
  • For example, Address line 1 2 3, as arbitrary text
  • This still failed, because for <> you need to provide structure
• Martin:
  • In some places you require differentiating address number and street name, but street name (might have a number that makes things ambiguous?)
• Anne:
  • Yeah but, even where there were issues, we can still try to have some structure
  • We still have the same issue even if we resolve the country use case
• Yoav:
  • The multi-phase autofill use case with events will allow us to manually solve this but only if the site knows which data to update
• Anne:
  • Is this done with Heuristics?
• Yoav:
  • Yes, and we can try to improve autofill better (ie. more field types)?
  • But I think we will still need to resolve to Heuristics whenever the browser data structures disagree with the requirements of the page– so this will continue to get massaged
• Christian:
  • What if we do the initial schema, similar to the autocomplete values that exist today, share that, and then evolve from there?
  • I assume this would happen more than once over time.
• Martin:
  • Is this group also required to specify the evolution of the address? or should that happen elsewhere, such as the ISO group, and we would adapt gradually
  • Let’s decouple the need for a definition from the framework for using one
• Anne:
  • The specific types are part of the contract between the website and the browser
  • the “types=” are owned by the browser.
  • It could obviously be part of ISO to spec, but we still need to control it
• Yoav:
  • Right, but we could bring things in
  • We should do all those things, we should decouple them
• Stephen:
  • Summarize the two options that are currently under discussion:
  • 1. Have an event handler, browser will send data down, via schema, but allows the browser to inject their own UI into the flow such as a disambiguation UI
  • 2. Two-phase autofill model. Are 2 phases enough?
• Domenic:
  • Not sure if two is enough, some counties might have more layers, but I dont know
• Martin/Yoav:
  • << Discussion on 2 pass vs 3 pass >>
• Martin:
  • What prevents us from getting ALL the data from the browser in the first place?
  • The browser
• Christian:
  • << Description of the current model >>
• Yoav:
  • Right now JS handler to observe data
• Rick:
  • Ultimately this is about getting all the data from the browser
• Yoav:
  • Privacy property we want to meet is no hidden fields
• Rick:
  • No, I think we want to show everything to the user that we will send, before we send it.
• Yoav:
  • << Showing example of UI used today with ‘full-address” proposal >>
  • Then it's on the website to unpack
• Rick:
  • Okay, that seems neat, but isn’t setting hidden fields equivalent to sending data
• Christian…
• Rick:
  • Ah, the fundamental difference seems to be: who is in charge of the schema.
  • Should the site ask for the data to be delivered in some form, or should the browser say, here’s the form I’d like to provide
• Yoav:
  • Discussing 2-phase
• Rick:
  • One of the bits of information that the browser how is “what bits of information” do I have
• Anne:
  • Now we’re back to defining all the keys
• Rick: HTML already are defined, but
• Anne: these are not sufficient of complete
• Rick: We can extend this set
• Anne: we’re back with the same thing
• Rick: SImilar to locale
• Yoav:
  • This is not a locale…
  • We have a discussion a few days ago, and it's… very different
• Rick:
  • We might want to update the data, like locale, rather than update some c++ code rarely
• Anne:
  • module some political issues of defining that format
• Rick:
• Yoav:
• Anne:
  • I was looking how we store data
• Rick:
  • We can deal with some amount of inconsistency here
  • Even in the browser now users make different choices
• Anne:
  • I think it's okay for the data to be inconsistent, but the model shouldn’t be
• Christian:
  • The oncomplete values are in a schema today?
• Yoav:
  • …
  • Back to testing discussion where at least we can verify that
• Anne:
  • << Example of some hacks used today>>
• Rick:
  • Maybe we need to define “versioned” keys.
  • I.e. street name format 1 and street name format 2 via “address1” or whatever
• Christian:
  • There isn’t a single consistent require here
  • The more advanced the page the more control they want
  • …
  • this is causing a lot of issues and we are struggling to deal with this complexity
• Yoav:
  • Discussing followups
  • Should we have a regular call to followup regularly?
• Anne:
  • Was there an issue filed for whatwg/HTML?
  • Good to do that