2015 NodeConf Notes

nodeconf_notes.md

NodeConf 2015 (June 11-13)

###Isomorphic JS

• challenges/tradeoffs in writing identical code client/server side
• write once, render anywhere
• State does not live in the template
• existing problem in server side rendering - how to present local timezone that represents client without passing client's timezone
• Serving side is first class citizen in React
• ways people have been getting around polyfills on polyfills, leveldb-browserify, leveljs
• node 12 has built in l10n and i10n, but still tricky to determine where logic for that should live
• react is checksum based diffing for determining how to update DOM, any change in how client renders initial state can interfere
• want guarantee with this input always get this output
• native stuff with polyfills
• node version differences could result in code not cross compatible browser/server code
• other concers: date library not shimmed, maps, character encoding - line endings, utf8
• wrapping in anon funcs and exporting iffy
• big shims do not compose
• dont mutate distributions, use module exports
• having modules that detext whether define is defined or undefined,
• webpack vs browserify as a way to bundle front end assets, broserify limits itself to npm modules but is simpler
• jquery plugins ubiquitous
• how to codify fine grained resource validation
• conditional building of frontend assets, instead of individual resources
• prebuilt together, deliver separately
• http 2.0 and what it means, bundle everything together used to be the way to go to reduce number of back and forths
• webpack follows a react-router, swtiching betn shims, shared js file global stuff

###Modular UI

• publish css html and js together as a package in npm, include via browerify/webpack
• if you publish npm module, assume it be used beyond server. e.g. browser, iot
• ui heavy, npm install , put it on page
• cssify (override, npm install, pull in style, inject into dom, functional, webpack ignores conditional or requires)
• webpack - require globally safe from require
• web componenets - scope css ; how package components,
• keep html in tooling
• imports make it harder to decide, explicit dependency tree
• beerfest is a browserify extension for images into base64 encoded string
• npm amenable to ui components
• did not miss cascading when using react or npm components
• package level control of components
• casading util extend
• runtime query stylesheet
• recycling dom elements itself
• wants twitter bootstrap for npm install, autocomplete, jsfest, datepicker
• need convention/standard for modular ui shipping on ui: web componenets are that standard, go through common dom api
• percelify, browserify plugin for css
• grunt fileblocks - if file matches pattern, add to bundle, css duble
• eliminate cascading, modular is better
• css is global is a problem
• it has to allow options to modify, component exposes the properties it expects to change.
• after selector, responsive media queries, react style component as a js object.
• web component problems: resource loading, lack of data binding; interacting with web component outside of web component.
• aurelia.io (client side framework) - register element
• images in npm base64 encode, left to module publisher to not publish large images
• fonts bower.
• packages just require dirname, and everything underneath is adde

###Debugging production node

• newrelic production websocket.io
• appdynamics java side, node side websocket support
• newrelic, over life time of request is something unusual
• loggly
• error in sentry, kafka for logging. (uber)
• strongloop, produciton tooling
• stream based, websockets
• dont serialize error logs
• aync harder to instrument, transaction. memoery leaks (v8 heaps vs buffers mdb)
• stack trace pprint with colors for scans
• v8 actual usagge is leaking
• heap memory represention via flamechart vs flamegraph in d3
• reproduce it locally - categrories of failure, restart
• make sure bad app state is cached
• cache once
• bad memeory performace
• accidentally synchronous code which you expect to be sync, blocking main thread common culprits
• os issues, sockets not opening
• wrap base class and wrap it with context to see arguments at every lines of stacktrace for function debugging
• statsd invoming outoing, opensockets, incoming outgoing latency. app specific metrics, if drivers are zero is bad (uber) i/o router messages, ad hoc afterwards
• something that hits every thing hit, +ve and -ve, set metric dashboard page.
• daemon, cpu and memory graph
• log regex, threshold, alert for lack of notifications over some time
• people not using domains (know domains but dont use them) i.e. do not throw errors
• asyncwrap (in io.js and v0.12), longjohn, cls
• load testing, jenkins
• one off scripts, one offs scripts, 10 production
• bypass authentication
• build microservices so that you can load test well
• unit performance testing
• server monitoring, dom slow,
• dom redraw performacnce
• deployment test run it till it clashes.
• garbage collector
• system level tracing in linux
• garbage collector debugging
• tracing
• flame graphs when we see issues spikes in latency, p95, 95th percentile

###Community Discussion/Argumentation

• inclusion taking it from other communities like python
• making changes in core, breaking changes pulling in internally
• x wants A, y wants B, how can they work together to make sure that both happens
• perception of cultural of argumentation, self fulfilling prophecy
• code community; laser focus on what happens in core.
• github issues
• levedb problems
• 300-400 community
• shared sense of values about the project
• nodebots
• helping enormous number of people, open arms to them
• make obvious about availability of mentorship
• cognitive argument vs effective argument, -> objective set of values
• ask power circle; dont get it from others

###ES6 in io.js

• const and let
• classes
• generators

###Electron shell

• use web technologies to write cross platform desktop apps
• cross platform browser development
• renderer
• vs node webkit

###Full stack iojs

• const for require statements since likely never change it
• {link, greeting} === {link: link, greeting: greeting}
• render it on server side instead of client
• how do i display state
• isomorphic, taking state from one side and make viewable on other
• dom element might have model
• fluxible, context get service from, client side wrapper
• latency compensation
• javascript loading menu
• react does not yet have meteors way of representing

###Technical Commity governance policy

• joyent, io.js
• reconvene into one project
• foundation, node open governance.
• technical steerng policy
• open to others influence
• preload modules, relationship with v8
• v8s main priority of chrome
• work with features with v8 as they ship for them
• big users of v8

###JS Module package managers

• combine npm with browserify/package manager/jspm
• require js
• how vendor code is getting packaged
• shims (existing api) vs polyfills (backfill for apis that dont exist yet)
• commonJS or UMD model used, people are not using script tags, UMD model
• jspm works against github and uses it as package manager
• bower frustating, where is the script, no convention, single source dependency managemnent; bower gives you flat structure
• use require to include everything from a directory
• sourcemap from a directory
• backend dependencies
• frontend dependencies, packages differently, diferent versions of same module
• recursive dependencies are an issue too
• flattening the curve make available at top level
• css loading
• ember cl compile and concat
• dependency management strategies
• strategy: every webpage module, recursively include, breaking out, adapter module,
• strategy: using fa webpack, common js qay requires css, css loader
• single page everything together
• isomorphic , frontend and backed, modules, 52000 packages in npm worked on browser as well as server
• strategy: require statements separate js modules
• grunt or gulp to minify
• monolithic utility libraries
• bootstrap modularly
• wripping our selector portion of selector
• individual features pulled in a la carte
• make frontend more like npm modules
• flatten recursive dependency trees, bundle.
• different concerns on frontend
• small module context
• address things as you need indivudally
• slow processor, native browser specs
• input range type, range within range
• component, modularize
• jquery, leaves state on the dom, more maintainable javascript
• query(selection) example of how jquery can be component by component replaced
• ecosystem supporting individual modules together, monoliths together
• better ways to describe in npm
• dont punish newer or mobile browser, punish the ie8 or ie9 by including polyfills

###STREAMS

• object streams in , backpressure , set high watermark, object stream pubsub
• high watermark how much you should buffer when writer is faster than reader
• core streams, dont pause incoming tcp streams too much
• pause tcp socket
• streams 3 vs streams v2
• readable emit data, vs wraitable asking for data
• flowing vs non-flowing state
• readable streams, num bytes, undefined or 0
• mreadme 0 marks, just give me however many bytes you have read.
• deque 5 butes ast a time to the filesystem. .buffer attribute, everytime, writeable part, how far write before filesystem.
• sync vs async stream - loop as much as as it could, _write mixed sync and async
• buffer is implemented as linked list
• writec underlying resource, _write ability to handle
• transform stream to coalesce into one.
• filestream desire block size,
• block event loop, is bad potentially block
• pipe: pipe sets closure event listeners on various events, add listener to readable on data; trigger flowing mode;
• first initial spark gets it to writing things;
• flow read whatever it can, write sets up false, drain on its size
• when writeable blocks, buffers shock absobers for when things pause, after done writing and thing cannot read anymore, streams, destrysoon, do next
• reading read up the buffer, queue, _read, wait for draing _read; maximize and traffic, underlying resource, writeable
• writeable calls _write with chunk encoding and callback, if do notcallback, return false, write with chunk, pile up a buffer,
• absolutely cannot eat another byte, double shock absorbing,
• readable tries to max specific amount of time writing
• duplex has buffers on either side
• transforms are subclasses of buffers
• transforms move from loopback duplex
• duplexify - node writeable and readbble, method one streamhandle errors, duplexify
• end of stream - logic to set up when stream is ending on end; readable, half open, streams2 - list of streams only pushing data , bufffers empty, buffers empty, tcp empty, no more data will be emitted
• http server in node, figure out when they end, big file, stream will not emit end, client error; emit closed, might get close event, readable has ended and buffer has ended
• end of streams pass a callback, fuction stream dn callbacp
• pump - close on end, two streams, one prematurely ends, streams 1 check if either has destroy cand call, streams 2 if none corect end event, http server, streaming large file to end, filestream never closes file descriptior after this has happend, out of file descriptiors
• in level request, strreaming bunch of data, forget to propagate data
• listening for error , memory leaks for pipe
• sendfile modularity, talking over one method pipe
• pumpify - Combine an array of streams into a single duplex stream using pump and duplexify; - pass it an array of streasms eror handling, encapsulate
• readable streams,
• sreams cannot control what other streams people are using. globally agreed upon construct comes from core.
• through and through2
• readable stream and writeable
• disseminate practical streams best practices
• buffers
• pulsestreaml
• in duplex, reable and writeable.
• promises , eventemitters have issues ordering of events, wg-streams, promises streams in a browser
• classifying asynchrony

###Distributing node modules for getting PRs

• tap
• coveralls
• travis CI
• linters (automated tests commenting on PRs is better for code style testing)
• nyc module (code coverage tool on top of istanbul)
• istabul- cordova - pr, func
• npm scripts are standard way to expose to committers what is possible on repo
• github-changes
• collborator -> geberate collaborator
• hapi project, good success getting maintainer/contribs
• pull me tender module
• travis, coveralls
• semver how lax on dependncies
• angular commit message convention to generate changelog; bugfix/feature and commit changes; semantic release;
• Really import to have good examples showing most important use cases for api
• picking a good module scope is important for getting collaborators
• unconscious awareness, exp learning to contribute; just publish it
• mit org; publish module under your handle on github and use it everywhere; process of reinforcement
• people cant see every mod you publish; you have to publicize
• use emoticons on issues and pr
• node-modules.com
• publish on github notify; first publsh; star it;
• github notifications; closed and and open
• iterate on concise yet searchable package description;
• npm cli search
• node modules cli tool
• good examples for beginners really important
• clear cobtribution.md
• automatically elect people to be maintainers
• tech, community, package publisher, web development

###Libuv

• cluster master and cluster worker, custer master and core
• share socket incoming
• give hardware instruction to send packet, lets pause this thread
• most os designed around interrupt then restart thread model
• only kick off hardware operaton, do not put thread on pause
• node doesnt have to pause when network call, when io operaton is happening
• dns, tcp connect, start program, create window
• super command get result of all of them, epoll, kqueue
• everything is a file descriptor linux
• bucket of file descriptor, associate each event into it,
• socket fileuv
• plasters over one conceptiual model one interface
• thread pool - network connections, os most disk operations, create file,
• run it through thread pool
• libuc runs a lot of stuff in thread pool
• user mode scheduling, kernel mode switch doubling
• networking, truly async networking without threads involved in any platform
• currently fixed size of threadpool, limit is num processors
• why file system operations do not have kernel async
• linux kernel async file systeml os cache, 14 bytes package json, few bytes read a whole lot, write it back, leveraging the cache, data is kept swhere in memory
• cache is a page fault; particular file live somwhere in memory but does not load is memory, when os processor or mem manager; raise interrupt; switch to kernel mode
• when page selectors, low level async primitives
• block boundaries; virtual threads
• fs sync libuv
• if four cores, only one node process, all jscrpt one thread, libuv thread pools blocking ops that dont block main thread
• javascript not designed to have mutliple cores, multiple jscript vms could be a possibility, all cpu - cluster different
• node primarliy for web servers
• make sched decision, worker batch io.js multiple vms in a node process
• javascripty patterns, modern os inverse of libuv
• go/rust move work around threads
• shared state
• q.write thread pool,
• reorder of queue
• socket inside libuv and remove it aynchronously, is it possible.
• uv_tcpt
• raise an interrupt
• uv_tvptt s
• uv_read()
• uv_write()
• future pluggable import stream feature in so that people can plugin
• not discard data, reuse port cant reuse other sockets

###Node.js performance monitoring(DIY)

• c++ folks instrumentation
• e.g. publicization node perf issues such as walmart memory leak and netflix cpu slowness
• express static routes, piled up
• longer the chain takes, longer takes for express to serve up
• nodemon, v8-profiler, v8::headsnapshot, v8::capuprofile
• v8 profiling function to get data out of read
• v8 compiler - heap memory, gc profiler
• process.memoryUsage
• gc happened
• building mem leaks, every request, add to array
• create mem leak e.g
• heap structure of interconnected paths
• chrome collect javascript cpu profile
• shallow size
• function closures leak
• heap snapshot
• v8 profiler, mem leaks
• flame graph
• sunbirst
• interpreting flame graphs, common red flags - pyramids (if you think a function should have taken a lot of time, but in fact, it is a sliver, but lot of steps necessary to get to that file, in that case maybe call that function directly) and plateu (this function is taking up a lot of time)
• http stream on messages
• latency from cpu exec time, disembiguate
• node inspector, dtrace
• chrome tracing deprecated
• one way to help performance is to find synchronous code, and put it into different process
• better native module, identify non optimal functions
• async listener, module shims
• async wrap in iojs
• context local storage, event emitters, filtering
• performance monitorig over system boundaries
• selectively instrument functions that users care about, instrument function over function by hand
• http server framework instrument and any database
• appneta vs appdynamics vs dynatrace vs newrelic
• trace instrument the stack
• tracing bwtween microservices
• cross service interaction
• perf monit tools
• zipkin from twitter
• node irehydra
• writing v8 happy code involves writing predictable code, e.g same number of arguments for functions, not using polymorphic functions(i.e. not different types), so that v8 can optimize function
• https://github.com/thlorenz/v8-perf
• latency tooling, flame graph