Tom Ridge tomjridge
tomjridge
/ equivalence_relation_and_inverse.sql
Created
August 31, 2023 08:23
SQL partial equivalence relation, extension, inverse to get master representative
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- master station using pure sql, rather than python | |
| -- we have an initial table of tiplocs | |
| create table tmp_tiploc | |
| ( | |
| tiploc TEXT NOT NULL | |
| ); | |
| insert into tmp_tiploc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (** Test bigarray casting using https://discuss.ocaml.org/t/cast-bigarray-kind/8469/8 *) | |
| let arr1_i = Bigarray.(Array1.init Int C_layout 512 (fun i -> i)) | |
| let _ = | |
| Printf.printf "Bigarray int size in bytes: %d\n" Bigarray.(kind_size_in_bytes int); | |
| assert(Bigarray.(kind_size_in_bytes int) = 8) (* for 64bit arch *) | |
| let _arr2_c = | |
| Ctypes.bigarray_start Ctypes.array1 arr1_i |> fun (pi:int Ctypes.ptr) -> |
tomjridge
/ gist:c3649108d8c616932f4d8e81d57c4121
Created
September 8, 2021 09:43
irmin(master) vs irmin(sqlite), 200k commits
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| make -k mk_trace_stats | |
| dune exec -- ./irmin/bench/irmin-pack/trace_stats.exe pp -f master,master_stat_summary.json -f lite,lite_stat_summary.json | |
| -- setups -- | |
| | master | lite | |
| Hostname | pc1156-nvmp3 | pc1156-nvmp3 | |
| Word Size | 64 bits | 64 bits | |
| Start Time | 2021/09/08 08:40:55 (GMT) | 2021/09/07 18:56:49 (GMT) | |
| Inode Config | mls:32 bf:32 sh:256 | mls:32 bf:32 sh:256 | |
| Store Type | pack | pack | |
| Path Conversion | none | none |
tomjridge
/ gist:a96d288b00e4514dd209338a50d2255e
Created
September 7, 2021 12:08
irmin master vs lite, master from a while ago; lite has log sizes 400k, whereas master had 500k I believe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- setups -- | |
| | master | lite | |
| Hostname | pc1156-nvmp3 | pc1156-nvmp3 | |
| Word Size | 64 bits | 64 bits | |
| Start Time | 2021/08/24 13:09:22 (GMT) | 2021/09/07 10:45:35 (GMT) | |
| Inode Config | mls:32 bf:32 sh:256 | mls:32 bf:32 sh:256 | |
| Store Type | pack | pack | |
| Path Conversion | none | none | |
| | master | lite |
tomjridge
/ gist:1e8464b8f13afbc186f27d42356e8c66
Created
September 7, 2021 11:36
index+sqlite, 400k log, 100k commits
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| +000us application Will check commit hashes against reference. | |
| Merging 400001 entries | |
| Merge took 17.000000 | |
| Merging 400001 entries | |
| Merge took 20.000000 | |
| Merging 400001 entries | |
| Merge took 22.000000 | |
| Merging 400001 entries | |
| Merge took 26.000000 | |
| Merging 400001 entries |
tomjridge
/ gist:43fa45bb71ecbb01fcbeecc2e9b399f4
Created
August 24, 2021 12:13
irmin master vs mmap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| dune exec -- ./irmin/bench/irmin-pack/trace_stats.exe pp -f master,master_stat_summary.json -f mmap,mmap_stat_summary.json | |
| -- setups -- | |
| | master | mmap | |
| Hostname | pc1156-nvmp3 | pc1156-nvmp3 | |
| Word Size | 64 bits | 64 bits | |
| Start Time | 2021/08/24 11:55:23 (GMT) | 2021/08/24 11:48:41 (GMT) | |
| Inode Config | mls:32 bf:32 sh:256 | mls:32 bf:32 sh:256 | |
| Store Type | pack | pack | |
| Path Conversion | none | none |
tomjridge
/ gist:a041ddf1f714e671ea3e45a7c7c0a1ba
Last active
August 24, 2021 10:32
irmin, with and without mmap, benchmarks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| First ** BENCH entry is for plain irmin master; second is with mmap | |
| ** BENCH irmin master tom_m <2021-08-24 Tue 11:16> | |
| --- | |
| -*- mode: compilation; default-directory: "/tmp/l/github/irmin/" -*- | |
| Compilation started at Tue Aug 24 11:16:26 |
tomjridge
/ gist:a5782882d05e7de6341f2a3f65365100
Created
August 23, 2021 13:54
index current master, make bench
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -*- mode: compilation; default-directory: "/tmp/l/github/index/" -*- | |
| Compilation started at Mon Aug 23 14:49:14 | |
| make -k bench | |
| { | |
| "config": { | |
| "key_size": 32, | |
| "value_size": 13, | |
| "nb_entries": 10000000, | |
| "log_size": 500000, |
tomjridge
/ gist:1f2e5693baf52d71a7e766f802aa029c
Created
August 23, 2021 13:47
Current index master, with mmap-wip patch applied, make bench
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -*- mode: compilation; default-directory: "/tmp/l/github/index/" -*- | |
| Compilation started at Mon Aug 23 14:43:34 | |
| make -k bench | |
| { | |
| "config": { | |
| "key_size": 32, | |
| "value_size": 13, | |
| "nb_entries": 10000000, | |
| "log_size": 500000, |
tomjridge
/ gist:4e25330ec1be680f8c3b87b98b3ad53e
Created
August 23, 2021 11:22
index master bench, to compare with mmap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -*- mode: compilation; default-directory: "/tmp/l/github/index/" -*- | |
| Compilation started at Mon Aug 23 12:20:01 | |
| make -k bench_as_mmap_wip | |
| +21107403us [WARNING] index_unix Semaphore merge { id=8 } was blocked for 1.122s | |
| +23154010us [WARNING] index_unix Semaphore merge { id=9 } was blocked for 1.226s | |
| +25153913us [WARNING] index_unix Semaphore merge { id=10 } was blocked for 1.289s | |
| +27109163us [WARNING] index_unix Semaphore merge { id=11 } was blocked for 1.384s | |
| +29158370us [WARNING] index_unix Semaphore merge { id=12 } was blocked for 1.48s | |
| +31303899us [WARNING] index_unix Semaphore merge { id=13 } was blocked for 1.572s |
NewerOlder