Skip to content

Instantly share code, notes, and snippets.

@tusharf5
tusharf5 / read-large-files-in-node.md
Last active June 10, 2024 09:48
Read and Process Very Large Files line by line in Node.js With less CPU and Memory usage.

Reading Big Files in Node.js is a little tricky. Node.js is meant to deal with I/O tasks efficiently and not CPU intensive computations. It is still doable though but I'd prefer doing such tasks in languages like python, R etc. Reading, Parsing, Transforming and then Saving large data sets (I'm talking millions of records here) can be done in a lot of ways but only a few of those are efficient. Following snippet is able to parse millions of records without wasting a lot of CPU (15% - 30% max) and (40 MB - 60 MB max) memory. It is based on Streams.

The following program expects the input to be a csv file source eg. big-data.unpr.csv It saves the result as ndjson and not json as working with huge datasets is easier when done using ndjson format.

@joelmasters
joelmasters / next_nginx.md
Last active November 1, 2023 20:30 — forked from kocisov/next_nginx.md
How to setup next.js app on nginx with letsencrypt

How to setup next.js app on nginx with letsencrypt/getssl

next.js, nginx, reverse-proxy, ssl

Next.js package setup

Your server.js file should look like the following. Make sure you're serving static files!

const { createServer } = require('http');
const { parse } = require('url');
const next = require('next');
@kocisov
kocisov / next_nginx.md
Last active June 10, 2024 19:45
How to setup next.js app on nginx with letsencrypt
@maximkott
maximkott / mask.js
Last active April 15, 2024 20:43
mask a string in javascript
export const maskString = (
value: any,
mask: string,
maskPatterns: Record<string, RegExp | ((char: string) => boolean)>
) => {
value = value || '';
mask = mask || '';
maskPatterns = maskPatterns || {};
let maskedValue = '';
@h4ssi
h4ssi / Open command window here as administrator.reg
Created July 30, 2016 13:37
reg file to add "Open command window here as administrator" option in shift+rightclick explorer context menu
Windows Registry Editor Version 5.00
; heavily based on http://superuser.com/a/374396/589519
[-HKEY_CLASSES_ROOT\Directory\shell\runas]
[HKEY_CLASSES_ROOT\Directory\shell\runas]
@="Open command window here as administrator"
"HasLUAShield"=""
"Extended"=""
@craigminihan
craigminihan / gist:b23c06afd9073ec32e0c
Last active September 21, 2023 12:47
Build GCC 4.9.2 for C/C++ on CentOS 7
sudo yum install libmpc-devel mpfr-devel gmp-devel
cd ~/Downloads
curl ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.9.2/gcc-4.9.2.tar.bz2 -O
tar xvfj gcc-4.9.2.tar.bz2
cd gcc-4.9.2
./configure --disable-multilib --enable-languages=c,c++
make -j 4
make install