0.4 Roadmap for the Tabular Data Ecosystem

roadmap.md

• Replace NAtype with Nullable{T}
  • This will substantially change the semantics of missing data
    • You will have to fully specify how missing values are handled
    • You cannot leave off thinking about missing values to the first time you hit one
    • Your code will become type-stable and will run much faster (~100x)
    • The name NA will be replaced by NULL
      • This implies isna will become isnull
    • There will be no more attempts to define functions over NA
      • This includes Booleans: there will be no more three-valued logic
• Replace DataArray with NullableArray
  • This will change the semantics of scalar access to produce Nullable objects
• Replace PooledDataArray with CategoricalArray and OrdinalArray
  • This will make clear that these data structures are the Julia form of R's factors
  • This will change the semantics of scalar access to produce Nullable{CategoricalVariable} and Nullable{OrdinalVariable}
• Standardize DataFrames by formalizing API and finalizing core primitives

Sounds great, except:

This will change the semantics of scalar access to produce Nullable{CategoricalVariable} and Nullable{OrdinalVariable}

As I said elsewhere, calling them CategoricalValue and OrdinalValue would be much clearer. :-)

Finally, I'm not sure what you mean exactly with the last point.

Shouldn't this roadmap be inside DataFrame repo (on a file or on the Wiki), and issues be opened for all items here (and attached with a milestone)?