Kris Zyp kriszyp

## CBOR comparisons.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                kriszyp
                / CBOR comparisons.md
            
            
              Last active
              November 17, 2021 03:40
            
              
                This is a comparison of different CBOR encoding techniques including packing and the proposed record structure tag (and combining both) with cbor-x JS library, and how affect size and performance.
              
          
    The cbor-x's packed implementation only packs whole strings that occur multiple times, it does not search for repeated prefixes or postfixes, as they would almost certainly be vastly more expensive. Strings are packed if they occur multiples in a data structures. When using packed + record tags, strings as keys are not searched for string repetition (since it assumed repetition will mostly be eliminated by the structure reuse).
The table shows encoded size for each technique, and the encoding and decoding performance. The last column also includes the gzipped size for comparison sake (no gzip performance, but generally is about 2-4x slower with gzipping in my tests). The table compares plain CBOR encoding, packed, record structures with a 1+1 definition tag and 1+2 tag, and the combination of packed and record structures.
The first comparison test uses an 8KB JSON data structure from our database of medical studies, that has a fairly complicated and dynamic structure:
https://github.com/kriszyp/cbor-x/blob/

  
## serialization-performance-small.csv

          
            operation
            op/s

            
              buf = Buffer(JSON.stringify(obj));
              526300

            
              obj = JSON.parse(buf);
              527300

            
              require('msgpackr').pack(obj);
              1081280

            
              require('msgpackr').unpack(buf);
              675080

            
              msgpackr w/ shared structures: packr.pack(obj);
              1284720

            
              msgpackr w/ shared structures: packr.unpack(buf);
              2770860

## serialization-performance.csv

          
            operation
            op
            ms
            op/s

            
              buf = Buffer(JSON.stringify(obj));
              82000
              5004
              16386

            
              obj = JSON.parse(buf);
              88600
              5000
              17720

            
              require('msgpackr').pack(obj);
              161500
              5002
              32287

            
              require('msgpackr').unpack(buf);
              94600
              5004
              18904

            
              msgpackr w/ shared structures: packr.pack(obj);
              178400
              5002
              35665

            
              msgpackr w/ shared structures: packr.unpack(buf);
              376700
              5000
              75340

            
              buf = require('msgpack-lite').encode(obj);
              30100
              5012
              6005

            
              obj = require('msgpack-lite').decode(buf);
              16200
              5001
              3239

            
              buf = require('notepack').encode(obj);
              62600
              5005
              12507

## string-functions.csv

          
            Function
            Characters
            Time

            
              Buffer.toString
              3
              180ns

            
              Buffer.toString
              10
              200ns

            
              Buffer.toString
              2000
              500ns

            
              Buffer.toString('latin1')
              2000
              415ns

            
              TextDecoder.decode
              10
              1700ns

            
              String.fromCharCode(...)
              3
              16ns

            
              String.fromCharCode(...)
              10
              46ns

            
              String.fromCharCode(...)
              2000
              4300ns

            
              String.fromCharCode.apply(array)
              3
              29ns

## gist:cdd6fa337eebb3d8ba55
import { mixin, declareFrom, after } from 'core/traits' or 'core/compose'

class Request {
	get() { }
}
class Cache {
	@after /* add something to the method so that mixin knows that it should run after the method it is overriding? */
	get() {
	}
	invalidate() { }

## gist:8446857
> foo = new Model();
> foo.bar = {baz:3}
> foo.property('bar').property('baz').observe(function(value){console.log('baz new value', value);});
> foo.set('bar', {baz:5});
baz new value 5 VM6359:2

## gist:7439619
Dylan was asking me about the own()'ing of aspects, so I thought I would try to write out the issues in case it would help. The issue associated with not own()ing the function (https://github.com/dojo/dijit/blob/1.9.1/_WidgetBase.js#L940), is that if a reference to the widget or the handle is retained after destruction, the function is still referenced just as it was when it was live. What has made this a debatable issue though, is that if a widget is referenced after it is destroyed, is that really a valid use case? Is leaving such a reference itself a memory leak, and should be fixed, rather than trying to make memory leaks be smaller leaks? The downside of own()ing all aspect handles is that own()ing has a cost itself, each owned reference takes up a slot in an array, so by own()ing aspects, we are effectively punishing the vast majority of widgets do not ever end up with a reference to a destroyed widget.
It might help to look at a few different scenarios:
A single widget: when a widget listens/aspects t

## gist:5464150
.segment {
	width: 33%;
	float: left;
};
.inside {
	margin: 20px;
}
input {
	width: 100%;
}

## dual-boot.html
<!DOCTYPE HTML>
<html>
	<head>
		<title>Dual Boot Test</title>
		<style type="text/css">
		</style>

		<script type="text/javascript" src="/dev/dojo-release-1.3.3-src/dojo/dojo.js"></script>
		<script>
			console.log("dojo 1.3", dojo);

## gist:1690583
# My Cool Package

Here is little introduction.

## module-a

A little intro to this module. here are the methods/props:

<pull-in-from-api-docs> <!-- get stuff from code -->
<example for="myMethod1">
	operation	op/s
	buf = Buffer(JSON.stringify(obj));	526300
	obj = JSON.parse(buf);	527300
	require('msgpackr').pack(obj);	1081280
	require('msgpackr').unpack(buf);	675080
	msgpackr w/ shared structures: packr.pack(obj);	1284720
	msgpackr w/ shared structures: packr.unpack(buf);	2770860
Function	Characters	Time
Buffer.toString	3	180ns
Buffer.toString	10	200ns
Buffer.toString	2000	500ns
Buffer.toString('latin1')	2000	415ns
TextDecoder.decode	10	1700ns
String.fromCharCode(...)	3	16ns
String.fromCharCode(...)	10	46ns
String.fromCharCode(...)	2000	4300ns
String.fromCharCode.apply(array)	3	29ns
	import { mixin, declareFrom, after } from 'core/traits' or 'core/compose'

	class Request {
	get() { }
	}
	class Cache {
	@after /* add something to the method so that mixin knows that it should run after the method it is overriding? */
	get() {
	}
	invalidate() { }
	> foo = new Model();
	> foo.bar = {baz:3}
	> foo.property('bar').property('baz').observe(function(value){console.log('baz new value', value);});
	> foo.set('bar', {baz:5});
	baz new value 5 VM6359:2
	Dylan was asking me about the own()'ing of aspects, so I thought I would try to write out the issues in case it would help. The issue associated with not own()ing the function (https://github.com/dojo/dijit/blob/1.9.1/_WidgetBase.js#L940), is that if a reference to the widget or the handle is retained after destruction, the function is still referenced just as it was when it was live. What has made this a debatable issue though, is that if a widget is referenced after it is destroyed, is that really a valid use case? Is leaving such a reference itself a memory leak, and should be fixed, rather than trying to make memory leaks be smaller leaks? The downside of own()ing all aspect handles is that own()ing has a cost itself, each owned reference takes up a slot in an array, so by own()ing aspects, we are effectively punishing the vast majority of widgets do not ever end up with a reference to a destroyed widget.
	It might help to look at a few different scenarios:
	A single widget: when a widget listens/aspects t
	.segment {
	width: 33%;
	float: left;
	};
	.inside {
	margin: 20px;
	}
	input {
	width: 100%;
	}
	<!DOCTYPE HTML>
	<html>
	<head>
	<title>Dual Boot Test</title>
	<style type="text/css">
	</style>

	<script type="text/javascript" src="/dev/dojo-release-1.3.3-src/dojo/dojo.js"></script>
	<script>
	console.log("dojo 1.3", dojo);
	# My Cool Package

	Here is little introduction.

	## module-a

	A little intro to this module. here are the methods/props:

	<pull-in-from-api-docs> <!-- get stuff from code -->
	<example for="myMethod1">