djvanderlaan/#codelistdemo.md

## #codelistdemo.md

      
    Raw
  

              #codelistdemo.md
            
          
    We have the Iris dataset saved into a data package. There are two
resources: the data and the code list for the ‘Species’ variable:
> library(datapackage)
> dp <- opendatapackage("iris")
> dp
[iris] 

Location: <./>
Resources:
[iris] The Iris Dataset
[Species-codelist] Iris Species Codes
Get the data. We have the numeric codes in the ‘Species’ field:
> iris <- dpresource(dp, "iris") |> dpgetdata()
> head(iris)
  Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1          5.1         3.5          1.4         0.2       1
2          4.9         3.0          1.4         0.2       1
3          4.7         3.2          1.3         0.2       1
4          4.6         3.1          1.5         0.2       1
5          5.0         3.6          1.4         0.2       1
6          5.4         3.9          1.7         0.4       1
We can see there is a codelist associated with ‘Species’
> iris$Species
  [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 [38] 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 [75] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3
[112] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
[149] 3 3
attr(,"fielddescriptor")
Field Descriptor:
name    :"Species"
type    :"integer"
codelist:"Species-codelist"
And also get that codelist. The field descriptor info in the column has
a link back to the datapackage
> dpcodelist(iris$Species)
  code      label
1    1     setosa
2    2 versicolor
3    3  virginica
We can use this codelist to convert the field from its numeric values to
factor:
> dptofactor(iris$Species) |> head()
[1] setosa setosa setosa setosa setosa setosa
Levels: setosa versicolor virginica
We can also specify that all fields with an associated codelist should
be converted to factor:
> dpresource(dp, "iris") |> dpgetdata(to_factor = TRUE) |> head()
  Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1          5.1         3.5          1.4         0.2  setosa
2          4.9         3.0          1.4         0.2  setosa
3          4.7         3.2          1.3         0.2  setosa
4          4.6         3.1          1.5         0.2  setosa
5          5.0         3.6          1.4         0.2  setosa
6          5.4         3.9          1.7         0.4  setosa
One annoying little thing in R is in some operations remove attributes.
For example when subsetting, the link to the datapackage and codelist is
lost and the following no longer works:
> tmp <- iris[c(1, 51, 101), ]
> dptofactor(tmp$Species)
Warning:  Field does not have an associated code list. Returning original vector. 
[1] 1 2 3
In that case we have to get the codelist ourselves when we want to
convert to factor:
> codel <- dp |> dpresource("iris") |> dpfield("Species") |> dpcodelist()
> # or 
> # codel <- dp |> dpresource("Species-codelist") |> dpgetdata()
> dptofactor(tmp$Species, codelist = codel)
[1] setosa     versicolor virginica 
Levels: setosa versicolor virginica
And even if we don’t have any facilities for automatically use the
codelist (.e.g. just the facilities offered by version 1 of the datapackage spec), we can use existing functionality to see what the codelist is,
read the codelist and apply the codelist:
> dp |> dpresource("iris") |> dpfield("Species")
Field Descriptor:
name    :"Species"
type    :"integer"
codelist:"Species-codelist"
> codel <- dp |> dpresource("Species-codelist") |> dpgetdata()
> tmp$Species <- factor(tmp$Species, levels = codel[[1]], labels = codel[[2]])
> tmp
    Sepal.Length Sepal.Width Petal.Length Petal.Width    Species
1            5.1         3.5          1.4         0.2     setosa
51           7.0         3.2          4.7         1.4 versicolor
101          6.3         3.3          6.0         2.5  virginica

  
## datapackage.json
{
  "name": "iris",
  "resources": [
    {
      "name": "iris",
      "title": "The Iris Dataset",
      "format": "csv",
      "mediatype": "text/csv",
      "path": "iris.csv",
      "encoding": "utf-8",
      "schema": {
        "fields": [
          {
            "name": "Sepal.Length",
            "type": "number"
          },
          {
            "name": "Sepal.Width",
            "type": "number"
          },
          {
            "name": "Petal.Length",
            "type": "number"
          },
          {
            "name": "Petal.Width",
            "type": "number"
          },
          {
            "name": "Species",
            "type": "integer",
            "codelist": "Species-codelist"
          }
        ]
      }
    },
    {
      "name": "Species-codelist",
      "title": "Iris Species Codes",
      "format": "csv",
      "mediatype": "text/csv",
      "path": "Species-codelist.csv",
      "encoding": "utf-8",
      "schema": {
        "fields": [
          {
            "name": "code",
            "type": "integer"
          },
          {
            "name": "label",
            "type": "string"
          }
        ]
      }
    }
  ]
}

## iris.csv

          
            Sepal.Length
            Sepal.Width
            Petal.Length
            Petal.Width
            Species

            
              5.1
              3.5
              1.4
              0.2
              1

            
              4.9
              3
              1.4
              0.2
              1

            
              4.7
              3.2
              1.3
              0.2
              1

            
              4.6
              3.1
              1.5
              0.2
              1

            
              5
              3.6
              1.4
              0.2
              1

            
              5.4
              3.9
              1.7
              0.4
              1

            
              4.6
              3.4
              1.4
              0.3
              1

            
              5
              3.4
              1.5
              0.2
              1

            
              4.4
              2.9
              1.4
              0.2
              1

            
              4.9
              3.1
              1.5
              0.1
              1

            
              5.4
              3.7
              1.5
              0.2
              1

            
              4.8
              3.4
              1.6
              0.2
              1

            
              4.8
              3
              1.4
              0.1
              1

            
              4.3
              3
              1.1
              0.1
              1

            
              5.8
              4
              1.2
              0.2
              1

            
              5.7
              4.4
              1.5
              0.4
              1

            
              5.4
              3.9
              1.3
              0.4
              1

            
              5.1
              3.5
              1.4
              0.3
              1

            
              5.7
              3.8
              1.7
              0.3
              1

            
              5.1
              3.8
              1.5
              0.3
              1

            
              5.4
              3.4
              1.7
              0.2
              1

            
              5.1
              3.7
              1.5
              0.4
              1

            
              4.6
              3.6
              1
              0.2
              1

            
              5.1
              3.3
              1.7
              0.5
              1

            
              4.8
              3.4
              1.9
              0.2
              1

            
              5
              3
              1.6
              0.2
              1

            
              5
              3.4
              1.6
              0.4
              1

            
              5.2
              3.5
              1.5
              0.2
              1

            
              5.2
              3.4
              1.4
              0.2
              1

            
              4.7
              3.2
              1.6
              0.2
              1

            
              4.8
              3.1
              1.6
              0.2
              1

            
              5.4
              3.4
              1.5
              0.4
              1

            
              5.2
              4.1
              1.5
              0.1
              1

            
              5.5
              4.2
              1.4
              0.2
              1

            
              4.9
              3.1
              1.5
              0.2
              1

            
              5
              3.2
              1.2
              0.2
              1

            
              5.5
              3.5
              1.3
              0.2
              1

            
              4.9
              3.6
              1.4
              0.1
              1

            
              4.4
              3
              1.3
              0.2
              1

            
              5.1
              3.4
              1.5
              0.2
              1

            
              5
              3.5
              1.3
              0.3
              1

            
              4.5
              2.3
              1.3
              0.3
              1

            
              4.4
              3.2
              1.3
              0.2
              1

            
              5
              3.5
              1.6
              0.6
              1

            
              5.1
              3.8
              1.9
              0.4
              1

            
              4.8
              3
              1.4
              0.3
              1

            
              5.1
              3.8
              1.6
              0.2
              1

            
              4.6
              3.2
              1.4
              0.2
              1

            
              5.3
              3.7
              1.5
              0.2
              1

            
              5
              3.3
              1.4
              0.2
              1

            
              7
              3.2
              4.7
              1.4
              2

            
              6.4
              3.2
              4.5
              1.5
              2

            
              6.9
              3.1
              4.9
              1.5
              2

            
              5.5
              2.3
              4
              1.3
              2

            
              6.5
              2.8
              4.6
              1.5
              2

            
              5.7
              2.8
              4.5
              1.3
              2

            
              6.3
              3.3
              4.7
              1.6
              2

            
              4.9
              2.4
              3.3
              1
              2

            
              6.6
              2.9
              4.6
              1.3
              2

            
              5.2
              2.7
              3.9
              1.4
              2

            
              5
              2
              3.5
              1
              2

            
              5.9
              3
              4.2
              1.5
              2

            
              6
              2.2
              4
              1
              2

            
              6.1
              2.9
              4.7
              1.4
              2

            
              5.6
              2.9
              3.6
              1.3
              2

            
              6.7
              3.1
              4.4
              1.4
              2

            
              5.6
              3
              4.5
              1.5
              2

            
              5.8
              2.7
              4.1
              1
              2

            
              6.2
              2.2
              4.5
              1.5
              2

            
              5.6
              2.5
              3.9
              1.1
              2

            
              5.9
              3.2
              4.8
              1.8
              2

            
              6.1
              2.8
              4
              1.3
              2

            
              6.3
              2.5
              4.9
              1.5
              2

            
              6.1
              2.8
              4.7
              1.2
              2

            
              6.4
              2.9
              4.3
              1.3
              2

            
              6.6
              3
              4.4
              1.4
              2

            
              6.8
              2.8
              4.8
              1.4
              2

            
              6.7
              3
              5
              1.7
              2

            
              6
              2.9
              4.5
              1.5
              2

            
              5.7
              2.6
              3.5
              1
              2

            
              5.5
              2.4
              3.8
              1.1
              2

            
              5.5
              2.4
              3.7
              1
              2

            
              5.8
              2.7
              3.9
              1.2
              2

            
              6
              2.7
              5.1
              1.6
              2

            
              5.4
              3
              4.5
              1.5
              2

            
              6
              3.4
              4.5
              1.6
              2

            
              6.7
              3.1
              4.7
              1.5
              2

            
              6.3
              2.3
              4.4
              1.3
              2

            
              5.6
              3
              4.1
              1.3
              2

            
              5.5
              2.5
              4
              1.3
              2

            
              5.5
              2.6
              4.4
              1.2
              2

            
              6.1
              3
              4.6
              1.4
              2

            
              5.8
              2.6
              4
              1.2
              2

            
              5
              2.3
              3.3
              1
              2

            
              5.6
              2.7
              4.2
              1.3
              2

            
              5.7
              3
              4.2
              1.2
              2

            
              5.7
              2.9
              4.2
              1.3
              2

            
              6.2
              2.9
              4.3
              1.3
              2

            
              5.1
              2.5
              3
              1.1
              2

            
              5.7
              2.8
              4.1
              1.3
              2

            
              6.3
              3.3
              6
              2.5
              3

            
              5.8
              2.7
              5.1
              1.9
              3

            
              7.1
              3
              5.9
              2.1
              3

            
              6.3
              2.9
              5.6
              1.8
              3

            
              6.5
              3
              5.8
              2.2
              3

            
              7.6
              3
              6.6
              2.1
              3

            
              4.9
              2.5
              4.5
              1.7
              3

            
              7.3
              2.9
              6.3
              1.8
              3

            
              6.7
              2.5
              5.8
              1.8
              3

            
              7.2
              3.6
              6.1
              2.5
              3

            
              6.5
              3.2
              5.1
              2
              3

            
              6.4
              2.7
              5.3
              1.9
              3

            
              6.8
              3
              5.5
              2.1
              3

            
              5.7
              2.5
              5
              2
              3

            
              5.8
              2.8
              5.1
              2.4
              3

            
              6.4
              3.2
              5.3
              2.3
              3

            
              6.5
              3
              5.5
              1.8
              3

            
              7.7
              3.8
              6.7
              2.2
              3

            
              7.7
              2.6
              6.9
              2.3
              3

            
              6
              2.2
              5
              1.5
              3

            
              6.9
              3.2
              5.7
              2.3
              3

            
              5.6
              2.8
              4.9
              2
              3

            
              7.7
              2.8
              6.7
              2
              3

            
              6.3
              2.7
              4.9
              1.8
              3

            
              6.7
              3.3
              5.7
              2.1
              3

            
              7.2
              3.2
              6
              1.8
              3

            
              6.2
              2.8
              4.8
              1.8
              3

            
              6.1
              3
              4.9
              1.8
              3

            
              6.4
              2.8
              5.6
              2.1
              3

            
              7.2
              3
              5.8
              1.6
              3

            
              7.4
              2.8
              6.1
              1.9
              3

            
              7.9
              3.8
              6.4
              2
              3

            
              6.4
              2.8
              5.6
              2.2
              3

            
              6.3
              2.8
              5.1
              1.5
              3

            
              6.1
              2.6
              5.6
              1.4
              3

            
              7.7
              3
              6.1
              2.3
              3

            
              6.3
              3.4
              5.6
              2.4
              3

            
              6.4
              3.1
              5.5
              1.8
              3

            
              6
              3
              4.8
              1.8
              3

            
              6.9
              3.1
              5.4
              2.1
              3

            
              6.7
              3.1
              5.6
              2.4
              3

            
              6.9
              3.1
              5.1
              2.3
              3

            
              5.8
              2.7
              5.1
              1.9
              3

            
              6.8
              3.2
              5.9
              2.3
              3

            
              6.7
              3.3
              5.7
              2.5
              3

            
              6.7
              3
              5.2
              2.3
              3

            
              6.3
              2.5
              5
              1.9
              3

            
              6.5
              3
              5.2
              2
              3

            
              6.2
              3.4
              5.4
              2.3
              3

            
              5.9
              3
              5.1
              1.8
              3

## Species-codelist.csv

          
            code
            label

            
              1
              setosa

            
              2
              versicolor

            
              3
              virginica
	{
	"name": "iris",
	"resources": [
	{
	"name": "iris",
	"title": "The Iris Dataset",
	"format": "csv",
	"mediatype": "text/csv",
	"path": "iris.csv",
	"encoding": "utf-8",
	"schema": {
	"fields": [
	{
	"name": "Sepal.Length",
	"type": "number"
	},
	{
	"name": "Sepal.Width",
	"type": "number"
	},
	{
	"name": "Petal.Length",
	"type": "number"
	},
	{
	"name": "Petal.Width",
	"type": "number"
	},
	{
	"name": "Species",
	"type": "integer",
	"codelist": "Species-codelist"
	}
	]
	}
	},
	{
	"name": "Species-codelist",
	"title": "Iris Species Codes",
	"format": "csv",
	"mediatype": "text/csv",
	"path": "Species-codelist.csv",
	"encoding": "utf-8",
	"schema": {
	"fields": [
	{
	"name": "code",
	"type": "integer"
	},
	{
	"name": "label",
	"type": "string"
	}
	]
	}
	}
	]
	}
Sepal.Length	Sepal.Width	Petal.Length	Petal.Width	Species
5.1	3.5	1.4	0.2	1
4.9	3	1.4	0.2	1
4.7	3.2	1.3	0.2	1
4.6	3.1	1.5	0.2	1
5	3.6	1.4	0.2	1
5.4	3.9	1.7	0.4	1
4.6	3.4	1.4	0.3	1
5	3.4	1.5	0.2	1
4.4	2.9	1.4	0.2	1
4.9	3.1	1.5	0.1	1
5.4	3.7	1.5	0.2	1
4.8	3.4	1.6	0.2	1
4.8	3	1.4	0.1	1
4.3	3	1.1	0.1	1
5.8	4	1.2	0.2	1
5.7	4.4	1.5	0.4	1
5.4	3.9	1.3	0.4	1
5.1	3.5	1.4	0.3	1
5.7	3.8	1.7	0.3	1
5.1	3.8	1.5	0.3	1
5.4	3.4	1.7	0.2	1
5.1	3.7	1.5	0.4	1
4.6	3.6	1	0.2	1
5.1	3.3	1.7	0.5	1
4.8	3.4	1.9	0.2	1
5	3	1.6	0.2	1
5	3.4	1.6	0.4	1
5.2	3.5	1.5	0.2	1
5.2	3.4	1.4	0.2	1
4.7	3.2	1.6	0.2	1
4.8	3.1	1.6	0.2	1
5.4	3.4	1.5	0.4	1
5.2	4.1	1.5	0.1	1
5.5	4.2	1.4	0.2	1
4.9	3.1	1.5	0.2	1
5	3.2	1.2	0.2	1
5.5	3.5	1.3	0.2	1
4.9	3.6	1.4	0.1	1
4.4	3	1.3	0.2	1
5.1	3.4	1.5	0.2	1
5	3.5	1.3	0.3	1
4.5	2.3	1.3	0.3	1
4.4	3.2	1.3	0.2	1
5	3.5	1.6	0.6	1
5.1	3.8	1.9	0.4	1
4.8	3	1.4	0.3	1
5.1	3.8	1.6	0.2	1
4.6	3.2	1.4	0.2	1
5.3	3.7	1.5	0.2	1
5	3.3	1.4	0.2	1
7	3.2	4.7	1.4	2
6.4	3.2	4.5	1.5	2
6.9	3.1	4.9	1.5	2
5.5	2.3	4	1.3	2
6.5	2.8	4.6	1.5	2
5.7	2.8	4.5	1.3	2
6.3	3.3	4.7	1.6	2
4.9	2.4	3.3	1	2
6.6	2.9	4.6	1.3	2
5.2	2.7	3.9	1.4	2
5	2	3.5	1	2
5.9	3	4.2	1.5	2
6	2.2	4	1	2
6.1	2.9	4.7	1.4	2
5.6	2.9	3.6	1.3	2
6.7	3.1	4.4	1.4	2
5.6	3	4.5	1.5	2
5.8	2.7	4.1	1	2
6.2	2.2	4.5	1.5	2
5.6	2.5	3.9	1.1	2
5.9	3.2	4.8	1.8	2
6.1	2.8	4	1.3	2
6.3	2.5	4.9	1.5	2
6.1	2.8	4.7	1.2	2
6.4	2.9	4.3	1.3	2
6.6	3	4.4	1.4	2
6.8	2.8	4.8	1.4	2
6.7	3	5	1.7	2
6	2.9	4.5	1.5	2
5.7	2.6	3.5	1	2
5.5	2.4	3.8	1.1	2
5.5	2.4	3.7	1	2
5.8	2.7	3.9	1.2	2
6	2.7	5.1	1.6	2
5.4	3	4.5	1.5	2
6	3.4	4.5	1.6	2
6.7	3.1	4.7	1.5	2
6.3	2.3	4.4	1.3	2
5.6	3	4.1	1.3	2
5.5	2.5	4	1.3	2
5.5	2.6	4.4	1.2	2
6.1	3	4.6	1.4	2
5.8	2.6	4	1.2	2
5	2.3	3.3	1	2
5.6	2.7	4.2	1.3	2
5.7	3	4.2	1.2	2
5.7	2.9	4.2	1.3	2
6.2	2.9	4.3	1.3	2
5.1	2.5	3	1.1	2
5.7	2.8	4.1	1.3	2
6.3	3.3	6	2.5	3
5.8	2.7	5.1	1.9	3
7.1	3	5.9	2.1	3
6.3	2.9	5.6	1.8	3
6.5	3	5.8	2.2	3
7.6	3	6.6	2.1	3
4.9	2.5	4.5	1.7	3
7.3	2.9	6.3	1.8	3
6.7	2.5	5.8	1.8	3
7.2	3.6	6.1	2.5	3
6.5	3.2	5.1	2	3
6.4	2.7	5.3	1.9	3
6.8	3	5.5	2.1	3
5.7	2.5	5	2	3
5.8	2.8	5.1	2.4	3
6.4	3.2	5.3	2.3	3
6.5	3	5.5	1.8	3
7.7	3.8	6.7	2.2	3
7.7	2.6	6.9	2.3	3
6	2.2	5	1.5	3
6.9	3.2	5.7	2.3	3
5.6	2.8	4.9	2	3
7.7	2.8	6.7	2	3
6.3	2.7	4.9	1.8	3
6.7	3.3	5.7	2.1	3
7.2	3.2	6	1.8	3
6.2	2.8	4.8	1.8	3
6.1	3	4.9	1.8	3
6.4	2.8	5.6	2.1	3
7.2	3	5.8	1.6	3
7.4	2.8	6.1	1.9	3
7.9	3.8	6.4	2	3
6.4	2.8	5.6	2.2	3
6.3	2.8	5.1	1.5	3
6.1	2.6	5.6	1.4	3
7.7	3	6.1	2.3	3
6.3	3.4	5.6	2.4	3
6.4	3.1	5.5	1.8	3
6	3	4.8	1.8	3
6.9	3.1	5.4	2.1	3
6.7	3.1	5.6	2.4	3
6.9	3.1	5.1	2.3	3
5.8	2.7	5.1	1.9	3
6.8	3.2	5.9	2.3	3
6.7	3.3	5.7	2.5	3
6.7	3	5.2	2.3	3
6.3	2.5	5	1.9	3
6.5	3	5.2	2	3
6.2	3.4	5.4	2.3	3
5.9	3	5.1	1.8	3