Skip to content

Instantly share code, notes, and snippets.

@284km
Last active May 30, 2018 18:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 284km/9158d466a12bb03fcbd29d723f8fc92e to your computer and use it in GitHub Desktop.
Save 284km/9158d466a12bb03fcbd29d723f8fc92e to your computer and use it in GitHub Desktop.
(csv / libcsv) memory usage
### 10 columns, 1,000,000 lines

Calculating -------------------------------------
            unquoted     1.397B memsize (    61.000M retained)
                        16.000M objects (     2.000  retained)
                        28.000  strings (     1.000  retained)
              quoted     1.857B memsize (    81.000M retained)
                        26.000M objects (     2.000  retained)
                        29.000  strings (     1.000  retained)
   (libcsv) unquoted     1.413B memsize (     1.049M retained)
                        31.000M objects (     3.000  retained)
                         1.000  strings (     0.000  retained)
     (libcsv) quoted     1.412B memsize (    80.000  retained)
                        31.000M objects (     1.000  retained)
                         1.000  strings (     0.000  retained)

Comparison:
            unquoted: 1396650961 allocated
     (libcsv) quoted: 1411636904 allocated - 1.01x more
   (libcsv) unquoted: 1412686104 allocated - 1.01x more
              quoted: 1856651721 allocated - 1.33x more

### 10 columns, 100,000 lines

Calculating -------------------------------------
            unquoted   139.536M memsize (     6.100M retained)
                         1.600M objects (     2.000  retained)
                        28.000  strings (     1.000  retained)
              quoted   185.537M memsize (     8.100M retained)
                         2.600M objects (     2.000  retained)
                        29.000  strings (     1.000  retained)
   (libcsv) unquoted   142.071M memsize (     1.049M retained)
                         3.100M objects (     3.000  retained)
                         1.000  strings (     0.000  retained)
     (libcsv) quoted   141.022M memsize (    80.000  retained)
                         3.100M objects (     1.000  retained)
                         1.000  strings (     0.000  retained)

Comparison:
            unquoted:  139536241 allocated
     (libcsv) quoted:  141022184 allocated - 1.01x more
   (libcsv) unquoted:  142071384 allocated - 1.02x more
              quoted:  185537001 allocated - 1.33x more

### 5 columns, 100,000 lines

Calculating -------------------------------------
            unquoted   110.535M memsize (     3.100M retained)
                         1.100M objects (     2.000  retained)
                        28.000  strings (     1.000  retained)
              quoted   133.537M memsize (     4.100M retained)
                         1.600M objects (     2.000  retained)
                        29.000  strings (     1.000  retained)
   (libcsv) unquoted    82.071M memsize (     1.049M retained)
                         1.600M objects (     3.000  retained)
                         1.000  strings (     0.000  retained)
     (libcsv) quoted    81.022M memsize (    80.000  retained)
                         1.600M objects (     1.000  retained)
                         1.000  strings (     0.000  retained)

Comparison:
     (libcsv) quoted:   81022184 allocated
   (libcsv) unquoted:   82071384 allocated - 1.01x more
            unquoted:  110535101 allocated - 1.36x more
              quoted:  133536546 allocated - 1.65x more

### 10 columns, 10,000 lines

Calculating -------------------------------------
            unquoted    13.954M memsize (   610.121k retained)
                       160.193k objects (     2.000  retained)
                        28.000  strings (     1.000  retained)
              quoted    18.555M memsize (   810.121k retained)
                       260.213k objects (     2.000  retained)
                        29.000  strings (     1.000  retained)
   (libcsv) unquoted    15.140M memsize (     1.049M retained)
                       310.014k objects (     3.000  retained)
                         1.000  strings (     0.000  retained)
     (libcsv) quoted    14.090M memsize (    80.000  retained)
                       310.012k objects (     1.000  retained)
                         1.000  strings (     0.000  retained)

Comparison:
            unquoted:   13954361 allocated
     (libcsv) quoted:   14090304 allocated - 1.01x more
   (libcsv) unquoted:   15139504 allocated - 1.08x more
              quoted:   18555121 allocated - 1.33x more

### 5 columns, 10,000 lines

Calculating -------------------------------------
            unquoted    11.053M memsize (   310.121k retained)
                       110.163k objects (     2.000  retained)
                        28.000  strings (     1.000  retained)
              quoted    13.355M memsize (   410.121k retained)
                       160.174k objects (     2.000  retained)
                        29.000  strings (     1.000  retained)
   (libcsv) unquoted     9.140M memsize (     1.049M retained)
                       160.014k objects (     3.000  retained)
                         1.000  strings (     0.000  retained)
     (libcsv) quoted     8.090M memsize (    80.000  retained)
                       160.012k objects (     1.000  retained)
                         1.000  strings (     0.000  retained)

Comparison:
     (libcsv) quoted:    8090304 allocated
   (libcsv) unquoted:    9139504 allocated - 1.13x more
            unquoted:   11053221 allocated - 1.37x more
              quoted:   13354666 allocated - 1.65x more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment