Skip to content

Instantly share code, notes, and snippets.

@Tinram
Tinram / bulk_loader.py
Created September 8, 2022 17:48
faster MySQL LOAD DATA
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Bulk Loader
Bulk load into a new InnoDB table via a temporary MyISAM memory table.
~2x faster than LOAD DATA direct into InnoDB table.
From https://gist.github.com/Tinram/18bcbbaeaee648ccde1b8ddfeb1d6f63
Note: MEMORY table 2x faster, but requires max_heap_table_size pre-calculation; memory shunting/algos = poor performance.
@Tinram
Tinram / faster_mysql_bulk_load.sql
Last active September 6, 2022 12:50
Seeking a faster MySQL LOAD DATA INFILE
# MySQL 5.7.33
# 1 million rows, 66MB CSV file
# 2022-09-06
## standard way into an InnoDB table; slow
# LOAD DATA LOCAL INFILE 'products.csv' INTO TABLE import FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES (col1, col2, col3, col4)
-- Query OK, 1000000 rows affected (1 min 10.47 sec)
@Tinram
Tinram / GTID_replication.txt
Created August 16, 2021 14:39
GTID replication between two MySQL 8 docker containers
# Just proof-of-concept GTID replication, no security in the following set-up:
$ docker pull mysql
# Create Docker containers from offical MySQL image
$ docker run --name master -e MYSQL_ROOT_PASSWORD=password -d mysql
$ docker run --name replica -e MYSQL_ROOT_PASSWORD=password -d mysql
# get docker master container IP:
docker inspect master | grep IP
@Tinram
Tinram / Garbage.js
Last active May 2, 2018 13:26
Text garbage generator. Creates junk text for HTML prototypes etc.
var Garbage = {
/**
* Garbage text filler generator.
* @author Tinram
*/
/**