Clojurists Together Malli Project Update, part1

malli.md

Clojurists Together Project Update, Malli, part 1/3 (09-10/2021)

Have spent a lot of time working with Malli internals: on hammock, paper protos, reading about stuff done elsewhere, perf tuning and testing, better designs, in the mushroom forest, refactoring and finally, an initial new internal design I'm quite happy with.

The Performance gains

• Malli has now an 1-2 orders of magnitude faster Schema inferrer
• Malli's runtime validation and transformation performance is on most cases better than using idiomatic hand-written Clojure, while being fully declarative.
• Schema creation and transformations are 10-200 times faster than before
• Schema workers can be effectively cached yielding much better dev experience (and perf)
• Relevant resolved issues and PRs:
  • malli.providers/provide takes "forever" for large samples, #191
  • Performance of Schema and Worker Creation, #513
  • Schema AST - Fast Schema Creation and Transformations, #543
  • Perf part4, #539
  • faster mu/merge, #540
  • Schema AST, Cached and other performance improvement, #544
  • Cleanup, Entry parser protocol, worker caches and fast inferring, #550

The Release

All the current improvements are shipped in [metosin/malli "0.7.0-20211031.202317-3"], will test it properly againt real projects before releasing 0.7.0. See the full CHANGELOG.

malli2.md

Clojurists Together Project Update, Malli, part 2/3 (11-12/2021)

by Tommi Reiman.

Done Stuff

• Shipped one of the biggest releases of Malli, the 0.7.0 on Dec 7th! contained all the stuff done earlier + small fixes and more performance improvements & tests
• Wrote High-Performance Schemas in Clojure/Script with Malli 1/2 to celebrate the release
• Implemented a most wanted feature, a swappable default registry
• Helping users and contributors (should do much more of this)
• 27 Closed PRs
• Pushed five small patch releases (0.7.1 - 0.7.5) with bug fixes, updated dependencies and some nice schema provider improvements:
  • support for :map-of, :tuple and :enum types
  • support for type-hints
  • schema inferring via value decoding (e.g. :uuid, inst?)
  • performance improvements
• Improvements to clj-kondo interop

TODO

Zillion things to do and 2 months to go. Currently working on destructuring parser to enable first-class support inline typehints and to support inferring of function schemas from vanilla Clojure/Script. Seems like the easy features are all done, just the hard ones left :)

Cheers.

malli3.md

Clojurists Together Project Update, Malli, part 3/3 (01-02/2022)

by Tommi Reiman.

Done Stuff

• Shipped a major version of Malli, the 0.8.0 on Jan 23th! Biggest changes were: dev-tooling support for CLJS, Tools for Destructuring and support for the Plumatic Defn Syntax. Also the following smaller improvements:
  • malli.dev/start! uses malli.dev.pretty/reporter by default
  • allow :malli/schema to be defined via arglist meta-data, #615
  • BREAKING: local registries with schemas in vector syntax are stored as identity, not as form
  • BREAKING: :malli.provider/tuple-threshold has no default value
  • FIX: me/-resolve-root-error does not respect :error/path, #554
  • FIX: m/from-ast does not work with symbols or unamespaced keywords, #626
  • FIX: :+ parsing returns vector, not sequence
  • transformer names can be qualified, schema properties support :decode and :encode keys
  • updated dependencies
  • malli.dev.pretty/explain for pretty-printing explanations

• new malli.destructure ns for parsing Clojure & Plumatic destructuring binding syntaxes, see Destructuring.

(require '[malli.destructure :as md])

(-> '[a b & cs] (md/parse) :schema)
; => [:cat :any :any [:* :any]]

(-> '[a :- :string, b & cs :- [:* :int]] (md/parse) :schema)
; => [:cat :string :any [:* :int]]

• new malli.experimental namespace with schematized defn, automatically registers the functions schemas with m/=>.

(require '[malli.experimental :as mx])

(mx/defn kakka :- :int
  "inline schemas (plumatic-style)"
  [x :- :int] (inc x))

• Released also small patch releases (0.8.1 - 0.8.3) with bug fixes and small improvements
• Read and reviewed all open PRs and managed to close most of them

TODO

Nothing, I'm happy with the things I managed to complete during the 6 month period despite the priorities changed during it. Looking at the original list:

1. First-class Schema inferring and effective schema types

• MOSTY DONE: it's not first class but orders of magnitude faster with lot's of new features like inferring :tuple, :map-of, :enum, type-hints and inferring via decoding. I'm happy with the current state.

3. Re-visit Schema registries, lifecycle, caching and contexts

• DONE: the registry can be swapped by default, workers are cached on instances etc

5. Error messages and tooling integration: pretty-printing of schema creation

• MOSTY DONE: new tools for pretty explaining, more robust EDN printer etc

Extra stuff that was added:

1. SchemaAST - parseless method of creating Schemas
2. LiteSyntax - optional simple syntax for tools like reitit
3. Schema Creation Performance - massive improvements
4. Dev-tooling - improvements to clj, support for cljs (kudos to Dan Vingo), clj-kondo support

Future of Malli

There would be so much to do, need to figure out a sustainable way to work on those. These include:

• Better support for Dates #49
• 2-way transformation with TypeScript #656
• Simplified Types #264
• On fixing Recursive Generation #452
• 2-way transformation for JSON Schema, #54

Cheers.