Skip to content

Instantly share code, notes, and snippets.

@chriskrycho chriskrycho/overview.md
Last active May 7, 2019

Embed
What would you like to do?
principled forms ideas

A principled model for forms

Over the past two years, we’ve experimented with a bunch of different ways to build forms, including the whole range from what we might call mutate all the things! to do nothing without an action! Each of those ends of the spectrum has significant pain points. Indeed: we recently built another couple of large forms, one of which used lots of mutability and one of which did exactly the opposite: modifying everything via actions. Both had serious pain points. Thinking on that, plus Danielle Adams’ EmberConf talk “Mastering the Art of Forms”, crystallized a couple things for me into what I think is a coherent way of approaching forms in general. So here we go:

  1. Every form has its own model, where the model is a data structure representing each form field’s type, current value, validation rules, and current validity. The form validity is the composition of the validity of all its fields using the validations.
  2. If an input changes something about the rest of the form, that means there is more than one form in play, i.e. that there are sub-forms within the form (even if there is only one <form>).
  3. Form models are local and freely mutated – because form state is inherently ephemeral unless and until it is “committed” and then persisted in some way.
  4. Accordingly, the form model to be mutated is always either:
    • a new instance of a default for the form model (e.g. the empty form, or a form with preselected/prefilled options)
    • a copy of previously-persisted state, mapped to a form model (in what should be a pure function)
  5. Persisting a form model is, like creating a form model, a pure function that simply maps back to the target model type in the persistence layer.
  6. The form model is owned (and stored) at whatever level is required for top-level validation – i.e. presumably the layer/component responsible for persisting the data (and therefore also possibly preventing submission/persistence of the data until it is valid).
  7. The validity of a field is not just invalid or valid but also includes an unvalidated state, because forms begin unvalidated.

These principles come together to solve many of the problems we face.

  • ergonomics: the separation of form models from persistence models means that we can freely add extra metadata to the former and mutate it freely until we persist it, without worrying about mutating the (hopefully immutable!) persistence models.
  • typing: TypeScript gives us the ability to check that we’re doing it right (see the example below), while implementing much of this in a very general way.
  • validation: making form validation the composition of field validation means that there’s very little extra machinery required to get whole-form validity and field validity displayed, and that machinery is reusable across forms.
  • generality: this approach is equally applicable across any modern JS UI library. We can use it equally in current Ember components, future Glimmer components, and so on. Even if we moved to Elm or Fable-Elmish or the like, we could easily adapt the approach to immutability by continuing to distinguish between form and persistence models.

Example implementation

Consider an address form, rendered in the context of a modal. The persisted version of the address is something like this:

type Address = {
  streetAddress: string;
  building: Maybe<string>;
  city: string;
  zipCode: string;
};

Given that definition, a form model for an input might look like this:

type AddressForm = {
  streetAddress: {
    type: 'text';
    value?: string;
    validations: Array<Validator<string>>;
    validity: Validity;
  };
  building: {
    type: 'text';
    value?: string;
    validations: Array<Validator<string>>;
    validity: Validity;
  };
  city: {
    type: 'text';
    value?: string;
    validations: Array<Validator<string>>;
    validity: Validity;
  };
  zipCode: {
    type: 'text';
    value?: string;
    validations: Array<Validator<string>>;
    validity: Validity;
  };
};

Here, a Validity is defined as a union type over three possibilities: Unvalidated, Invalid, and Valid, where Invalid carries a reason with it. We need all three of these because a form’s initial state is Unvalidated.

enum ValidityType { Unvalidated, Invalid, Valid }

class Unvalidated {
  type: ValidityType.Unvalidated = ValidityType.Unvalidated
}

class Invalid {
  type: ValidityType.Invalid = ValidityType.Invalid;
  constructor(readonly reason: string) {}
}

class Valid {
  type: ValidityType.Valid = ValidityType.Valid;
}

type Validity = Unvalidated | Invalid | Valid;

A Validator<T> is just a function which takes a value of type T or undefined or null and returns a Validity:

type Validator<T> = (value?: T | null) => Validity;

A simple example is a required string:

const requiredString = (value?: string | null): Validity =>
  value !== undefined && value !== null
    ? new Valid()
    : new Invalid('field is required');

Similarly, for a number, we could set a minimum length:

const minValue = (min: number): Validator<number> =>
  (value: number): Validity =>
    value >= min
      ? new Valid()
      : new Invalid(`${value} must be at least ${min}`);

These are easily-composeable: given a list of Validators for a given type, you can get the overall validity with a simple map:

const isValid = someField.validators
  .map(validate => validate(someField.value))
  .every(validity => validity.type === ValidityType.Valid);

What’s more, because every field value on a form model has its associated validity with it, displaying an error message next to it is simply a matter of using the validity.reason if the validity.type is ValidityType.Invalid.

Now, the fully-written out form model involved a lot of repetition. It can be simplified dramatically:

type Field<T> = {
  type: 'text' | 'number' | ... ;
  value?: T;
  validations: Array<Validator<T>>;
  validity: Validity;
};

type Form<Model> {
  [K in keyof Model]: Field<Model[K]>;
};

Then the type of our address form model would simply be:

type AddressForm = Form<Address>;

We could further reduce the repetition and improve both the ease of using this and the type safety by making Field an interface that many classes implement, while supplying default validations:

const email = (value: string): Validated => /.+@.+\..+/.test(value)
  ? new Valid()
  : new Invalid(`${value} is not a valid email address`);

class Email implements Field {
  type = 'email';
  validity = new Unvalidated();
  value?: string;
  validators: Array<Validator<string>>;

  constructor(value?: string = undefined, validators: Validator[] = []) {
    this.value = value;
    this.validators = [email].concat(validators);
  }
}

There are plenty of others we could build, of course, and plenty of other validations. Making a field required or a validation optional (i.e. applicable only if the field actually has a value) also arises trivially out of this approach to validation:

const isVoid = (v: any): v is void =>
  v === undefined || v === null;

const optional = <T>(validator: Validator<T>) => (value?: T | null) =>
  isVoid(value) ? new Valid() : validator(value);

Then when we go to use these in an actual Ember component (e.g.), we have a straightforward implementation:

export default class MyForm extends Component {
  model?: Address;
  form: Form<Address>;
  constructor(owner, args) {
    super(owner, args);

    this.form = this.model
      ? AddressForm.fromAddress(this.model)
      : AddressForm.default();
  }
}

And then rendering it into a template, assuming we had a <FormField> component (note that I’m using Glimmer syntax for a bunch of this):

<form>
  <FormField @name='Street' @model={{this.form.streetAddress}} />
  <FormField @name='Building' @model={{this.form.building}} />
  <FormField @name='City' @model={{this.form.city}} />
  <FormField @name='Zip Code' @model={{this.form.zipCode}} />
</form>

The FormField component:

<div class='field'>
  <label for={{this.fieldId}}>{{@name}}</label>
  <input
    value={{@model.streetAddress.value}}
    id={{this.fieldId}}
    onchange=(update
  >
  {{#if @model.invalid}}
    <p class='invalid'>{{@model.validity.reason}}</p>
  {{/if}}
</div>

Here of course we can see that we’d want two things for ease of use: a form field component which always takes a Field<T> as its argument, and an Ember helper function we could invoke to easy check form validity.

@olo/principled-forms

@olo/principled-forms is a library for principled form handling, ready to use as the form handling logic for your front-end app in any JavaScript framework.

[API Docs](TODO)

Installation

Add the library to your project.

With yarn:

yarn add @olo/principled-forms

With npm:

npm install @olo/principled-forms

Motivation

Form handling is inherently complicated. At a minimum, every form has to account for:

  • unvalidated, invalid, and valid fields – and the corresponding notion of what constitutes “validity” for a given field
  • correct handling of optional and required fields
  • different kinds of fields – checkboxes, radio buttons, text input, range inputs, etc.
  • the relationship between the field validations and form validation and submission
  • the relationship of form data to the persistence layer
  • maintaining accessibility with different presentation styles

@olo/principled-forms is not an attempt to abstract over that complexity but to isolate it. (Every attempt to hide form complexity ultimately leaks!)

The goal is to provide coherent, well-tested, and carefully type-checked tools for dealing with the inherent complexity of forms, while leaving you assemble those answers in the context of your own forms. It is not the UI layer, it’s the set of tools and patterns—as many of them as possible enforced by TypeScript’s type system!—for making sure the data behind your UI layer makes sense. Here’s that list again, with the parts this library handles bolded:

  • unvalidated, invalid, and valid fields – and the corresponding notion of what constitutes “validity” for a given field
  • correct handling of optional and required fields
  • different kinds of fields – checkboxes, radio buttons, text input, range inputs, etc.
  • **the relationship between the field validations and form validation **and submission
  • the relationship of form data to the persistence layer
  • maintaining accessibility with different presentation styles

@olo/principled-forms is designed to be consumed by the UI layer in your app – whether that’s vanilla JS, old-school jQuery, or a brand new Ember, React, or Vue app. For a reference implementation which integrates this library and carefully addresses the accessibility concerns, see the associated Ember component library, @olo/ember-principled-forms! You can drop that into an existing Ember application and start using this approach today.

Usage

Let’s assume a fairly typical model of a user in an application, which is stored in our data store with this type definition:

type User = {
  email: string;
  ageInYears: number;
  phoneNumber?: string;
  name?: string;
};

The contract for the user requires them to have an email address (presumably so we can contact them) and an age (so we can avoid breaking the law in some jurisdictions), and allows them to supply a name (which is optional because many people in the world don’t actually have a single name they use in all circumstances) or a phone number (perhaps for texting them information).

Our form handling should account for the fact that some of these pieces of data are required to create a user, and some are not. It should also only allow the right kinds of data to go into those fields. Enter @olo/principled-forms! Let’s assume that the type definition for a User lives in our application at models/user, and see how we’d define a principled form using the library.

First, we’ll define the transformation from a persistence model to our form model, using the FromModel<T> type to help us write the function correctly:

import Form, { FromModel } from '@olo/principled-forms/form';
import Field from '@olo/principled-forms/field';

import User from 'models/user';

const baseUserFormModel: Form<User> = {
  email: Field.required(),
  ageInYears: Field.required(),
  name: Field.optional(),
  phoneNumber: Field.optional(),
};

// TODO: extract for later example
const userFormModelFromUser: FromModel<User> = (user) => ({
  email: Field.required({ value: user.email }),
  ageInYears: Field.required({ value: user.ageInYears }),
  name: Field.optional({ value: user.name }),
  phoneNumber: Field.optional({ value: user.phoneNumber }),
});

Then we can use that to take the backing persistence model value and turn it into a form model:

// Note that the `phoneNumber` field is absent entirely.
let user: User = {
  email: 'hello@example.com',
  ageInYears: 24,
  name: 'John Doe',
};

// Has the type `Form<User>`
let userFormModel = userFormModelFromUser(user);

This is the most minimal correct setup for a form model we could have. The userFormModel is now a Form<User> and its validity property is one of the Validity values from @olo/principled-forms/validity: Unvalidated, Invalid, or Valid. In this case, because the persistence store had valid data, userFormModel.validity would be Validity.Valid. Had the persistence store contained invalid data (for example, if it were missing the email value), the form

Importantly, if we used Field.optional() for email or Field.required() for phoneNumber, this wouldn’t type-check. The FromModel<T> type checks whether the fields on the persistence model type T are optional and requires that you you make a RequiredField using Field.required for non-optional fields and an OptionalField using Field.optional. (Bonus: if you’re using the True Myth Maybe type, it maps those to OptionalFields as well. The types below all include the places where the True Myth Maybe type is allowed or inferred from.)

But what exactly is one of these RequiredField or OptionalField types? In short: a Field type is lightweight wrapper that contains:

  • the value of the field, value
  • whether or not it’s required (thus the Required and Optional variants)
  • its current Validity (the same validity type as the Form<User> has; that’s not a coincidence!)
  • a list of its validators

The Field.required() and Field.optional() functions take a hash of options which allow us to specify all of these properties other than the validity, which is not something we’re allowed to set ourselves. The Type here is the form field type: 'text', 'email', 'number', etc.

type RequiredFieldConfig<T> = {
  type?: Type;
  value?: T;
  validators?: Array<Validator<T>>;
  messageIfMissing?: string;
};

type OptionalFieldConfig<T> = {
  type?: Type;
  value?: T | Maybe<T>;
  validators?: Array<Validator<T>>;
}

The email and name fields we defined provide good working examples:

email:

  • It can have any value at all that is of type string, even if that string value is "gobbledygook" and therefore totally invalid.
  • The field is a RequiredField instance. That means that when we validate it, if the value is undefined, null, or an empty string, its validity will be Validity.Invalid.
  • It has no validators, so it will only be checked for its presence or absence.

name:

  • Like email, it can have any value of type string.
  • The field is an OptionalField instance. That means that when we validate it, if its value is undefined, null, or an empty string, its validity will be Validity.Unvalidated. (More on this below.)
  • It has no validators, so it will only be checked for its presence or absence.

We can make these checks much more helpful by including some validators for each of them. A validator is just a function which matches this type definition:

type Validator<T> = (value: T) => Validated;

where Validated is an instance of three types (slightly different from the Validity type discussed elsewhere, which is just used internally to distinguish these types): Unvalidated, Invalid, or Valid. Unvalidated and Valid are empty types which only serve to mark what validity state something is in; Invalid also includes the reason the state is invalid as a string.

There are a number of validators supplied with the library, but you can also write your own – as long as they conform to that API, they’ll work (and they won’t type-check if they don’t match that API!).

Here, we might decide that we want to use a regular expression to check that the email field at least has the right shape, and that a name is at least 5 characters long (who knows why). We could define the fields with validations, using the regex and minLength helpers from the validators module:

import Field from '@olo/principled-forms/field';
import { minLength, regex } from '@olo/principled-forms/validators';

let email: Field<string> = Field.required({
  validators: [regex(/.+@.+\..+/)],
});

let name: Field<string> = Field.optional({
  validators: [minLength(5)],
});

When you validate a field, the resulting validity is the result of checking the value of the field against each of the validators in the array – the result will be an array of validities, where each one is either unvalidated, invalid (with an explanation of why it’s invalid), or valid. The whole field’s validity is:

  • Unvalidated if every validity in that resulting array is Unvalidated
  • Invalid if any validity in the resulting array is Invalid
  • Valid if every validity in the resulting array is Valid or if it is an optional field and the value is empty

You can have as many of these validators in the array as you want. Be careful with this power, though: you can build nonsense combinations that will never be valid, like [minLength(5), maxLength(3)]!

For a very simple example of defining your own validators, let’s build two new helpers. For the first, we’ll just write a function which requires that its string argument be a string including the letter ‘v’. For the second, let’s assume you need to set a minimum and maximum length for a field—to match what your database supports, or something like that. (We could do that by combining the minValue and maxValue helpers, but where’s the fun-and-learning in that?)

import Field from '@olo/principled-forms/field';
import { Valid, Invalid, Validator } from '@olo/principled-forms/validity';

let includesV: Validator<string> = (value) => value.includes('v');
let validatedString = Field.required({ validators: [includesV] });

function inclusiveRange(min: number, max: number): Validator<string> {
  return (value) => {
    if (value < min) {
      return Invalid.because(value cannot be less than ${min});
    } else if (value > max) {
      return Invalid.because(value cannot be greater than ${max});
    } else {
      return Valid.create();
    }
  };
}

let validatedNumber: Field.required({ validators: [inclusiveRange(2, 7)] });

Notice that we can constructor a validator in however simple or complex a fashion as we need, as long as you end up with a Validator over the appropriate type (string, number, etc.). Most of the time, though, you’re probably going to end up writing your custom helpers as higher-order functions which return validators.

Once you have a field defined this way, you can validate it:

import Field from '@olo/principled-forms/field';
import { minLength } from '@olo/principled-forms/validity';

// Both are `Unvalidated`
const moreThan5CharsReq = Field.required({ validators: [minLength(5)] });
const moreThan5CharsOpt = Field.optional({ validators: [minLength(5)] });

// => `Invalid`
const validatedReq = Field.validate(moreThan5CharsReq);

// => `Valid`
const validatedOpt = Field.validate(moreThan5CharsOpt);

Here, the results are both still Fields, but with their validity checked given their values – they started out as Unvalidated, but a required field with no value is Invalid after validation; while an optional field with no value is Valid, because “empty” is indeed a valid state for it to be in. Also, it’s important to note that the original variables (moreThan5CharsReq and moreThan5CharsOpt) are unchanged: we make a copy with the newly calculated validity, rather than mutating the old one in place.

Why do fields start as Unvalidated? Because when a user has not yet interacted with a form field, it should not be considered “invalid” – an empty field may be invalid in the broad scheme of whether a form is allowed to be submitted, for example, but it’s a perfectly valid starting state for the form. On the one hand, the user certainly shouldn’t be seeing error messages about it; on the other hand, we can’t let them submit a form with required fields which haven’t been validated! We capture this with the Unvalidated state.

So given a field with a current value, how would you change the field’s value

Now that we have a Form<User>, we can use the validation rules from the library to validate it:

import { validate } from '@olo/principled-forms/form';

let validatedUser = validate(userFormModel);

Now, of course, this isn’t all that helpful: we also need two more pieces:

  • validation of the actual values of the fields, not just checking whether they’re present or not
  • updates to the fields themselves over time, as the user interacts with the field

The Field.required() and Field.optional() invocations we saw above have a rich API (which is identical between the required and optional functions) for specifying how the field to be validated. The library also supplies a handful of convenient helper functions for common validation patterns. For example, let’s say that we wanted to specify that the email field must meet the basic pattern of an email address, the user must be at least 13 years old , the name field has to be at least 5 characters long (who knows why), and the phone number should match a U.S. phone number pattern.

import Field from '@olo/principled-forms/field';
import Form from '@olo/principled-forms/form';
import {
  minLength,
  minValue,
  regex,
} from '@olo/principled-forms/validators';

const userFormModel: Form<User> = {
  email: Field.required({ validators: [regex(/^.+@.\..+/)] }),
  dateOfBirth: Field.required({ validators: [minValue(13)] }),
  name: Field.optional({ validators: [minLength(5)] }),
  phoneNumber: Field.optional({
    validators: [regex(/^(1-)?\d{3}-\d{3}\d{4}/)],
  }),
};

Now not only are the required or optional fields checked for presence or absence, but their values are validated whenever the form is.

Design

There are four fundamental ideas in the library:

  1. A form model is distinct from, though derived from, a persistence model. When we’re operating on a form, we have to keep track of its validity. For the most part, in our persistence layer – whether that’s a server-side database, a Redux store, or something else – we usually assume (and nearly always hope) we have either valid data or no data, but not invalid data. Therefore, every form has its own form model, distinct from the persistence model. A form model can always be derived from its corresponding persistence model through a pure function, and vice versa—and the transformation between the two can by type-checked.

  2. The validity of a form is just the composition of the validity of all of its fields. That is: if all the fields in a form are in a valid state, the form is valid, but if any of the fields in a form are in an invalid state, the form as a whole is also invalid. This means that the Form.validate function provided by the library can validate any Form model defined in terms of the Field model types from the library.

  3. A form field may be in three states: unvalidated, invalid, or valid. Many approaches to form handling treat a given field as invalid or valid, but in fact form fields start out (and may remain) unvalidated. Distinguishing between these three states allows for complex form behavior to emerge clearly and cleanly – for example, validating an email address while the user is typing it in, without showing an “invalid email” message just as soon as the user types the first letter of their email!

  4. The required-vs.-optional rule for a field is distinct from other validation rules. It is tempting to treat whether a field is required or not as simply being the same kind of rule as other validation rules – the minimum value for a number or a regex pattern for a string, for example – but in fact the required or optional value for a form determines how to apply the other rules. For example: an optional field is not invalid if it is empty, even though empty values (undefined or "") may not match against whatever other validation rules are specified for the field.

Every design decision in the library falls out of these four ideas.

There are also four basic levels of abstraction in the library:

  • form models
  • field models
  • validity states
  • validators

The library is designed so that you primarily operate in terms of the provided functions and types from the form, field, and validity modules, along with validators for each field. A number of default validators are supplied, but you are also encouraged to write your own – they have a very simple API contract (which we’ll cover below, as well as in the API Docs).

One final note on the library design: it’s written in TypeScript, so whether or not you’re using TypeScript yourself, you may get benefits from your editor – VS Code, JetBrains IDEs, and others are able to use TypeScript’s type definition files to provide smart editing completion. The library is optimized for TypeScript consumption, though: it leans hard on the most advanced features in TypeScript’s type system, so you can know if you’ve set up your form model correctly. For an example of that, let’s jump in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.