Skip to content

Instantly share code, notes, and snippets.

@bacloud23
bacloud23 / compress_words.js
Last active December 12, 2022 03:05
This is a very easy and intuitive solution to compress a big list of strings (better if strings are words). It simply removes small words while indexing them in the bigger containing words by start index and length of the small word; For instance: `fish` and `ish` is ommited while indexed into `fisher: [[0,4],[1,3]`
View compress_words.js
// Author: bacloud14(at)geemail(dot)com
/*
This is a very easy and intuitive solution to compress a big list of strings (better if strings are words).
It simply removes small words while indexing them in the bigger containing words by start index and length of the small word;
For instance: `fish` and `ish` is ommited while indexed into `fisher: [[0,4],[1,3]`
Other considerations:
- One and two characters words are not indexed because if indexed, they will result into a bigger model
(each indexed word result into `new Uint8Array(2)` in memory but again into two characters in the serialized model)
- The bellow list actually resulted into a bigger model (sad), but I'm sure a little thing is missing to regain that. I'm
@bacloud23
bacloud23 / stats.js
Last active September 4, 2022 09:09
Get any (with this limitation: Works only admiting (sub elements of V array)'s keys have the same value types) object statistics (like Python/Pandas DataFrame#summary method)
View stats.js
// Works only admiting (sub elements of V array)'s keys have the same value types
// oo = {
// v: [{
// a: 3,
// c: 10
// }, {
// a: 5,
// c: 100
// }],
// abee: [1, 5, 4, 8, 7, 9, 10, 1000],
@bacloud23
bacloud23 / mongo-aggregation.js
Created August 18, 2022 15:33
A full DB search mining tf-idf. this is used for autocomplete functionality - extracting only important keywords
View mongo-aggregation.js
//.replace(/(\b(\w{1,3})\b(\W|$))/g,'').split(/\s+/).join(' ')
const routine = `function (text) {
const stopwords = ['the', 'this', 'and', 'or', 'id']
text = text.replace(new RegExp('\\b(' + stopwords.join('|') + ')\\b', 'g'), '')
text = text.replace(/[;,.]/g, ' ').trim()
return text.toLowerCase()
}`
// If the pipeline includes the $out operator, aggregate() returns an empty cursor.
const agg = [
{
View docker-compose.yml
version: '3'
services:
vaultwarden:
image: vaultwarden/server:latest
container_name: vaultwarden
restart: always
environment:
- SIGNUPS_ALLOWED=false
- INVITATIONS_ALLOWED=false
@bacloud23
bacloud23 / client_chat.js
Last active July 4, 2022 11:59
Who said chat with websockets is easy ? These are most of the parts: client and server side. Server side is relying on Fastify and Fastify websockets.. You need to glow parts together at your convenience.
View client_chat.js
import { getChannels, newSocket } from './sockets/refresh.js'
recoverState()
if (newSocket()) getChannels()
@bacloud23
bacloud23 / _IPLookup.js
Last active June 21, 2022 21:47
Efficient storage of IPs v4. Memory/Search/Add. Can hold any prefixed Strings (IP, Path, Routes...) with the following assumption: very very big array with more changes of part(i) than part(i-1) -- Also see Radix Trie for more
View _IPLookup.js
// This is an implementation in JavaScript of the most simple yet efficient map suggested here https://stackoverflow.com/a/44950027/1951298
// Nothing special, I just like my JavaScript implementation.
const ips = ["100.1.108.246", "101.0.80.218", "101.108.122.200"]
const bucket = {}
function pushToBucket(bucket, ip) {
const intIp = ip.split('.').map(Number)
var part1, part2, part3, part4
;[part1, part2, part3, part4] = intIp
View offline_translator.js
// I WILL TRY TO CLEAN UP CODE !!!
const path = require('path')
const __filePath = path.join(__dirname, './wiki.multi.en.vec')
const filePath__ = path.join(__dirname, './wiki.multi.ar.vec')
const cachePath = path.join(__dirname, './cache.model')
if (typeof String.prototype.trim === 'undefined') {
String.prototype.trim = function () {
@bacloud23
bacloud23 / one-to-many-cache-mechanism.js
Last active April 23, 2022 23:20
This is my attempt to cache a post (like in blogs, called Listing here) and also to cache pages (like first, second, .. page that contains 9 Listings). The thing to mention here is that whenever one Listing is updated anywhere in the app, this modification must propagate through caches !!! DB is MongoDB, Cache is Redis, and my attempt is kinda m…
View one-to-many-cache-mechanism.js
// FIRST, it is to mention this doesn't handle race conditions all over the app per requests !!!
// ... that's another story ! A user could update a document and at the same time, another user requests that document (from cache)
// Check https://blog.theodo.com/2019/09/handle-race-conditions-in-nodejs-using-mutex/
// At first someone could argue it is just simple, whenever a Listing is modified,
// just delete it from Listing cache and iterate through all pages and remove pages containing the Listing.
// Well! this is TRUE ! and I didn't thought about that (or thought it would be performance degrading)
// So what I'm doing, is to keep a list of updated documents (with up = 3)
// When a user first visits a Listing, update up 3 -> 1, 2 -> nil
@bacloud23
bacloud23 / Constraints.js
Last active April 15, 2022 23:19
Super stupid 😝
View Constraints.js
// Constraints to ease code complexity. These constraints reflect
// which operations to run on some endpoint on some environment
const S = require('fluent-json-schema')
const { illustrations, fontFamilies } = require('./hallux.js')
const config = require('config')
const TAG_SIZE = config.get('TAG_SIZE')
const login = S.object()
.prop('username', S.string().format(S.FORMATS.EMAIL))
.prop('password', S.string().minLength(3).maxLength(40))
@bacloud23
bacloud23 / Pipeline.js
Last active April 15, 2022 23:21
This is a JavaScript pipeline with side effects implementation. I coded this for a web-app on Node side. It helped me to have a clear and robust pattern to ease complexity at scale. Practically, it helps "Post handlers" because they all get data from `request.body`, modify it, add to it,... while alwayse maintaining two values: `pipeline.result`…
View Pipeline.js
// author: b@cloud14[at]gmail.com
// This file is not self contained, I'm using it for my web-app on back-end.
// This gist is an illustration of a simple pattern that could help dealing each time on same data (req.body here) with different logics.
const geoEncoder = require('../../data/geo/geoJSONEncoder')
const { constraints } = require('../constraints/constraints')
const { html, reb, rew } = require('../constraints/regex')
const sanitizeHtml = require('sanitize-html')
const nlp = require('wink-nlp-utils');
const coordinates = geoEncoder.getBorders()