options.md

Handling options in OTP APIs

Suggestion

• Use a map for options unless a map will not work for some (good) reason.

• Use a list if a map will not work, for example if there is some (necessary) order dependency between the options.

• Never use a record for options in a public API.

As rule of thumb, if it would be possible to use a record (except for the pesky detail that a record needs an include file and that it is not possible to add new fields without requiring all users to recompile their code), use a map instead.

Motivation

Since lists are more flexible, why not always use lists?

The flexibility of lists leads to more possibilites to introduce bugs. For example, if we have this option list:

[{use_foo,true}, {use_foo,false}]

should the foo feature be enabled? Different APIs in OTP would answere this question in different ways. Even the Erlang compiler is not internally consistent in this regard; depending on the option either the first or last occurrence of the option may win.

If we use a map, this question has been rendered irrelevant. There can only be one element in a map with the key use_foo.

Therefore, we should not use a list for options unless necessary. The reason that Erlang has lists, tuples, and maps instead of only lists is that they all have their strengths and weaknesses.

Other advantages of using maps for options

Maps are easy to construct (especially using the map syntax). Constructing lists is only slightly more concise.

It is often possible to use a map directly as the internal data structure without only minimal massaging to fill in default values. Here is an example from the Efficiency Guide:

DefaultMap = #{shoe_size => 42, editor => emacs},
MapWithDefaultsApplied = maps:merge(DefaultMap, OtherMap)

If we have a list, we will often need to first process it and translate the options to a record or map for internal use (if we care about performance).

When do we need to use lists for options?

When there is an order dependency between the options, or repeating an option has a special meaning.

Usually such option lists originates from a command line, or from a command line augmented by environment variables.

For example, the option list for the compiler is pieced together from the explicit options given to compile:file/2, the options given in the environment variable ERL_COMPILER_OPTIONS, and from options given in the module itself. Here, it is clear that a map would not work.

There are also the action lists in the gen_statem module. Originally, Raimo called them options, but that turned out to be confusing because the order really mattered.

In general, having an order dependency between options is often a mistake that can lead to bugs. If the order dependency is necessary, by all means use a list, but perhaps call it a list of actions or commands in the documentation to make the order dependency explicit.

Summary

• Most options don't have any order dependency and are best given in a map.

• Options for complex applications such as compiler necessarily are order dependent because the options are collected from the the command line, environment variables, and directives inside the code itself.

• When "options" need to be processed in a specific order, they should probably be called something else, such as actions or commands.

Appendix A: Why the proplists module is not always convenient

All BIFs in the runtime system are strict about which options they accept. There will be a badarg exception raised if there unknown or wrongly spelled options in an option list. We should probably be equally strict for most functions that takes options in Kernel and STDLIB.

While the proplists module has many convenient functions, it can't help with reject invalid options. Therefore, functions that do strict evaluation of options lists can't depend on the proplists (or they will need additional checking code to reject invalid options).

Current APIs use lists for options

One motivation for having options as a list in the APIs is that we have not used maps for this in the current APIs so we will end up in a mixed and inconsistent state when it comes to options in APIs. To solve this or at least improve the situation (if we go for maps) would be to add the use of maps for option to existing APIs (in parallel with lists).

Options as lists looks nicer in programs

When expressing options as a list it is possible to allow simple keys like binary in compile:file("m.erl",[binary]). There are a lot of simple key options in compile:file/2. Using maps in this situation would lead to more tedious writing and longer lines when calling functions with many simple key options. Compare compile:file("m.erl",#{binary => true, warn_missing_spec => true}) with compile:file("m.erl",[binary, warn_missing_spec]). I think we should check this with more examples from current APIs.

... we will end up in a mixed and inconsistent state when it comes to options in APIs.

Yes, but we already have inconsistent APIs. That should not prevent us from using better features in the language. We might want to add the possibility to pass a map for some older APIs, especially if we are doing work extending them anyway.

When expressing options as a list it is possible to allow simple keys like binary in compile:file("m.erl",[binary]). There are a lot of simple key options in compile:file/2. Using maps in this situation would lead to more tedious writing and longer lines when calling functions with many simple key options.

I did mention that the compiler's options are not suitable to put in a map for other reasons (order dependency). I have never said that we should use maps for everything; it is OK to use a list if a map is not suitable.

Using maps for options will lead problem finding good names

If we take for example the asn1ct:compile function and the options like they are today we have:

compile(Asn1module, Options) -> ok | {error, Reason}
Types
Asn1module = atom() | string()
Options = [Option| OldOption]
Option = ber | per | uper | jer | der | compact_bit_string | legacy_bit_string | legacy_erlang_types | noobj | {n2n, EnumTypeName} |{outdir, Dir} | {i, IncludeDir} | asn1config | undec_rest | no_ok_wrapper | {macro_name_prefix, Prefix} | {record_name_prefix, Prefix} | verbose | warnings_as_errors
OldOption = ber | per
Reason = term()
Prefix = string()

Trying to convert this to use maps could look like this:

Options = #{ 
    encoding_rule => ber | per | uper | jer
    bit_string => compact | legacy
    legacy_types => boolean()
    object_code => boolean()
    n2n => [EnumTypeName]
    i => [Dir]
    asn1config => boolean()
    undec_rest => boolean()
...
}

Actually, I like the map version of the asn1 options better than the current options. I think it would be an improvement. Some of the names could be improved, though. i could be include_paths, for example.

I'm all for options as maps, for all the reasons enumerated by @bjorng. This will lead to a more consistent API, as it does away with the different handling of duplicates by different modules. I think it is necessary that they should be added as an alternative for option lists, for backward compatibility.

Implementing this will be a long and tedious process, however, involving a lot of manual labor. Each module needs to be checked to see if and how exactly they handle their option lists, if it is possible to put up a map alternative, and if yes, implement an appropriate list-to-map conversion. Plus specs. Plus docs.

So if this proposal is going to be implemented, I think it will need a dedicated team behind it, or at least some kind of manager who sees to it being pushed through consistently and not abandoned half-way, which would indeed leave us with an even more inconsistent API than we have now.

I would love to have option maps, too 😃 But a problem I see with that is when it comes to retrieving options. The return value can only be a list or a map then, and if we are to move forward, it has to be a map, which again would break backward compatibility 😢