Node.js Essential Packages

node-essential-packages.md

Node.js Essential Packages

Adrian's collection of super useful NPM packages.

Utilities

Async

• async - Higher-order functions and common patterns for asynchronous code.

Buffers

• binary - Unpack multibyte binary values from buffers and streams. Ability to specify endianness and signedness.
• file-type - Detect the file type of a Buffer/Uint8Array.
• safe-buffer - Safer Buffer API. Uses the built-in implementation when available.

File System

• fs-extra - Adds file system methods that aren't included in the native fs module and adds promise support to the fs methods.
• fs-write-stream-atomic - Writes to a tmp file and does an atomic fs.rename to move it into place when it's done.
• graceful-fs - Normalize behavior across different platforms and environments, and make filesystem access more resilient to errors.
• readdirp - Recursively reads the contents of a directory. Exposes a stream api.
• walkdir - Find files. Walk a tree of any depth.
• write-file-atomic - A fs.writeFile extension. makes operations atomic and allows you set ownership of the file.

JSON

• bfj - Asynchronous streaming functions for large JSON data sets.
• jsonfile - Read/write JSON files safely.
• JSONStream - Streaming JSON.parse and JSON.stringify.

Logging

• draftlog - Create updatable log lines into the terminal.

Parsing

• CSV

  • csv-write-stream - A CSV encoder stream that produces properly escaped CSVs.
  • csv-parser - Streaming CSV parser. Converts CSV into JSON at at rate of around 90,000 rows per second.
  • json2csv - Converts JSON into csv with column titles and proper line endings.

• Text Extraction

  • textract - Extracts text from a multitude of document types.

Pattern Matching

• globby - Match files using the patterns the shell uses.
• micromatch - Faster alternative to minimatch and multimatch.

Queue

• async.queue - async#queue method as module.
• run-queue - Promise based, dynamic priority queue runner, with concurrency limiting.

Rate Limiting

• limiter - Generic rate limiter for node.js.
• async-throttle - Minimal throttling.
• lodash.debounce - The lodash method _.debounce.

Retrying

• async-retry - Minimal async retrying.
• retry - Abstraction for exponential and custom retry strategies for failed operations.
• p-retry - Retry a promise-returning or async function.

Scraping / Extracting

• node-read - Get Readable Content from any page.

Streams [docs]

• concat-stream - Collect all the data from a stream into a single buffer.
• duplexer3 - Like duplexer2 but using Streams3 without readable-stream dependency.
• duplexify - Take a writable and a readable, and turn them into a single duplex stream.
• end-of-stream - Waits for stream to finish or error and then calls cb with (err).
• event-stream - Construct pipes of streams of events.
• flush-write-stream - Make a custom writable stream.
• from2 - Make a custom readable stream.
• is-stream - Check if something is a stream.
• mute-stream - Bytes are silently dropped, rather than being passed through.
• parallel-transform - Run transforms in parallel without changing the order.
• pump - Pipe streams together and close all of them if one of them closes.
• pumpify - Combine an array of streams into a single duplex stream using pump and duplexify.
• stream-each - Iterate all the data in a stream.
• stream-throttle - Rate limiter for Node.js streams.
• through2 - Make a custom transform stream.
• through2-concurrent - Process streams in parallel.

CLI Development

• mri - Fast, lightweight alternative to minimist and yargs-parser.
• update-notifier - Inform users of your package of updates in a non-intrusive way.
• inquirer - Collection of common interactive command line user interfaces.

Human Language

• abbrev - Calculate the set of unique abbreviations for a given set of strings.

Cryptography

• crypto - Native crypto module. Wrapper for OpenSSL's hash, HMAC, cipher, decipher, sign and verify functions.
• hasha - Get the hash of a buffer/string/stream/file.
• unique-slug - Generate a unique character string suitible for use in files and URLs.

HTTP

Utilities

• accepts - Higher level content negotiation based on negotiator.
• content-type - Create and parse HTTP Content-Type header according to RFC 7231.
• get-port - Get an available port.
• got - A nicer interface to the built-in http module.
• negotiator - HTTP content negotiator for Node.js.
• qs - Securely parse and stringify querystrings.
• raw-body - Parse request bodies.

Microservices

• micro - Asynchronous HTTP microservices.
  • micro-compress - Compression middleware.
  • micro-cors - CORS middleware.
  • micro-jwt-auth - Json web token(jwt) authorization wrapper.
  • micro-cookie - Cookie parsing.
  • micro-sentry - Send micro errors to the Sentry service.

Security

• helmet - Sets sane defaults for various HTTP headers.
• csurf - CSRF protection middleware.
• nsp - Detects known vulnerabilities in your projects.

WebSockets

• ws - Blazing fast, and thoroughly tested WebSocket client and server implementation.

Networking

• airpaste - A 1-1 network pipe that auto discovers other peers using mdns.
• bonjour - Publish services on the local network or discover existing services.
• castnow - Commandline chromecast player.
• chromecasts - Query local network for Chromecasts and have them play media.
• multicast-dns - Low level multicast-dns.

Debugging

• Chrome DevTools built-in debugger - Ships with Chrome DevTools. Replaces node-inspector.
• Node/V8 built-in debugger - Out-of-process debugging utility included with Node.
• debug - Small debugging utility.
• llnode - Plugin for the LLDB debugger. Advanced, low-level debugger.
• longjohn - Produce useful stack traces.

Testing

Unit Testing

• assert - Provides a simple set of assertion tests. A native Node module.
• ava - Minimal and fast. Runs test files in parallel as separate processes. Forces atomic tests.
• tape - Minimal, tap-producing test harness.
• sinon - Test spies, stubs and mocks.

Integration Testing

Functional Testing

• supertest - Useful for constructive, descriptive web server tests.

Load Testing

• toxy - HTTP proxy to simulate server failure scenarios and unexpected network conditions.
• skipfish
  • bombards server with valid and invalid requests.
  • improperly closes connections (catches memory leaks).
  • executes common attacks (massive dictionary of common attack vectors).
• ab - Apache Benchmark.
• gor - Captures and replays live HTTP traffic into a test environment.
  • replays actual requests on test server to emulate realistic traffic.
  • exposes layer of errors that can't be easily found via automated and manual testing.
    • concurrency
    • server environment-specific bugs
    • errors due to requests called in a particular order
• wrk - Multithreaded HTTP benchmarking tool.

Linting

• xo - Opinionated but configurable ESLint wrapper. Enforces strict and readable code.

Code Coverage

• nyc - Instanbul driven test coverage.

Monitoring

• vtop - A graphical activity monitor for the command line.