Skip to content

Instantly share code, notes, and snippets.

@sloev
sloev / fetch.py
Last active November 19, 2021 05:50
common crawl python cdx_toolkit
try:
import requests
import cdx_toolkit
import tqdm
import OpenSSL
from lxml import html
except:
print("you need to do some installs first:\npip install requests cdx_toolkit tqdm pyOpenSSL lxml")
exit(1)
@maitrungduc1410
maitrungduc1410 / create-vod-hls.sh
Last active April 12, 2025 01:00
Bash scripts to create VOD HLS stream with ffmpeg (Extended version)
#!/usr/bin/env bash
START_TIME=$SECONDS
set -e
echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1
# comment/add lines here to control which renditions would be created
renditions=(
Rank Type Prefix/Suffix Length
1 Prefix my+ 2
2 Suffix +online 6
3 Prefix the+ 3
4 Suffix +web 3
5 Suffix +media 5
6 Prefix web+ 3
7 Suffix +world 5
8 Suffix +net 3
9 Prefix go+ 2
@mbstacy
mbstacy / EKSvsEC2.md
Last active December 2, 2024 05:11
EKS vs EC2 Kubernetes Costs

Kubernetes Cost Comparison EC2 vs EKS

EKS Kubernetes

AWS EKS service provides the Kubernetes control plane. Price for EKS Service is $0.20 per hour / cluster.

EKS Use Case Monthly Cost

Three clusters with four worker nodes.

@parties
parties / renameReactJsToJsx.sh
Last active July 29, 2025 15:44
rename all *.js files containing React markup to *.jsx
# finds all *.js files that have either `</` or `/>` tags in them and renames them to *.jsx
find ./src -type f -name '*.js' -not -name '*.jsx' -not -name '*.ejs' -exec bash -c 'grep -l -E "</|/>" "$0"' {} \; -exec bash -c 'mv "$0" "${0%.js}.jsx"' {} \;
@MuhsinFatih
MuhsinFatih / fix-macos-python.md
Last active September 3, 2025 02:53
How to recover from messed up python installation on mac, and never have to mess with apple's python confusion factory

I am assuming you are here because like me, you installed a bazillion different python interpreters on mac and the whole thing is a spagetti. Today, I finally fixed my python installation. Whatever I install for python2 or python3 using pip JUST.WORKS.. My god! finally.

What the hell?

Here is what I had messed up, which you also probably did:

  • I had too many different python interpreters
  • Too many different symlinks which I lost track of
  • almost no package I installed with pip worked without a headache
  • any attempt to fix using online resources made it worse.
@snakers4
snakers4 / parse_cc_index.py
Last active September 14, 2023 20:00
Plain common crawl pre-processing
import gc
import gzip
import time
import json
import shutil
import os,sys
import tldextract
import collections
import pandas as pd
from tqdm import tqdm
@bradtraversy
bradtraversy / django_cheat_sheet.md
Last active March 23, 2025 20:59
Django command cheat sheet

Django 2.x Cheat Sheet

Creating a virtual environment

We need to create a virtual env for our app to run in: More Here Run this command in whatever folder you want to create your venv folder

python -m venv ./venv
@steven2358
steven2358 / ffmpeg.md
Last active October 18, 2025 13:57
FFmpeg cheat sheet
@rupakg
rupakg / serverless-application-for-long-running-process-fargate-lambda.md
Last active August 31, 2021 21:41
Post: How to use AWS Fargate and Lambda for long-running processes in a Serverless app

How to use AWS Fargate and Lambda for long-running processes in a Serverless app

Build an ETL job service by fetching data from a public API endpoint and dumping it into an AWS Redshift database.

Published on Jan. 11th, 2018 at https://serverless.com/blog/serverless-application-for-long-running-process-fargate-lambda/

thumbnail

AWS dropped so many serverless announcements at re:Invent, the community is still scrambling to make sense of them all. This post is all about AWS Fargate.

In this article, I will show you how to create an end-to-end serverless application that extracts thumbnails from video files. But, oh no, processing video files is a long-running process! Whatever will we do?