Skip to content

Instantly share code, notes, and snippets.

View kevcodez's full-sized avatar
🚀

Kevin Grüneberg kevcodez

🚀
View GitHub Profile
@kevcodez
kevcodez / pooling.ts
Last active April 20, 2025 03:01
medium_mongodb_performance_pooling
const client = await MongoClient.connect('mongodb-url', {
minPoolSize: 20,
maxPoolSize: 100,
});
@kevcodez
kevcodez / httpClient.ts
Last active January 24, 2024 02:11
Native Node.js HTTP Client with retries, proxy support, timeouts
import { ProxyAgent } from 'undici';
const SAFE_HTTP_METHODS = ['GET', 'HEAD', 'OPTIONS'];
const IDEMPOTENT_HTTP_METHODS = SAFE_HTTP_METHODS.concat(['PUT', 'DELETE']);
const HTTP_STATUS_TO_RETRY = [408, 429, 500, 501, 502, 503, 504];
const DEFAULT_TIMEOUT = 20_000;
const retryDenyList = new Set([
'ENOTFOUND',
@kevcodez
kevcodez / WasabiFileUploader.java
Last active April 29, 2023 22:20
Uploading files to Wasabi Cloud Storage using AWS Java SDK
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.CannedAccessControlList;
async timestampCreateSeries({
key,
retentionInSeconds,
labels,
}: {
key: string;
retentionInSeconds: number;
labels: string[][];
}) {
const args: (string | number)[] = [key, 'RETENTION', retentionInSeconds, 'DUPLICATE_POLICY', 'LAST'];
@kevcodez
kevcodez / document.json
Created October 7, 2022 15:16
medium_mongodb_performance_sample_document
{
"_id": "1232312323",
"name": "Apple",
"isin": "US123456890",
"marketCap": 1233413233.123,
"ipoDate": "2000-01-01",
"lastPrice": 123.23,
"dividends": [
{
"date": "2022-01-01",
@kevcodez
kevcodez / query.sh
Created October 7, 2022 15:15
medium_mongodb_performance_include_field
db.assets.find({
marketCap: { $gt: 10000 }
}, {
projection: { isin: 1 }
})
@kevcodez
kevcodez / query.sh
Last active October 7, 2022 15:08
medium_mongodb_performance_include_field
db.assets.find({
marketCap: { $gt: 10000 }
}, {
projection: { isin: 1 }
})
@kevcodez
kevcodez / client_options.ts
Created October 7, 2022 15:04
medium_mongodb_performance_client_options
const client = await MongoClient.connect('mongo-url', {
// perf optimization - https://github.com/mongodb/node-mongodb-native/releases/tag/v4.3.0
enableUtf8Validation: false,
});
@kevcodez
kevcodez / parallel.ts
Created October 7, 2022 15:02
medium_mongodb_performance_parallel_fetching
async function getQuotesForPeriod(
assetIdentifier: string,
from: Date,
to: Date
): Promise<Quote[]> {
const quotesCollection = await getQuotesCollection();
/**
* When fetching multiple years of data, queries are fast, but retrieval not so much.
* To improve query performance, we split the interval into a few chunks and query/fetch the data in parallel,
@kevcodez
kevcodez / mongo.ts
Created October 7, 2022 15:02
medium_mongodb_performance_node_snappy
const client = await MongoClient.connect(mongoConfig().mongoUrl, {
compressors: 'snappy', // make sure Snappy is installed
});