Skip to content

Instantly share code, notes, and snippets.

Keybase proof

I hereby claim:

  • I am firefishy on github.
  • I am firefishy (https://keybase.io/firefishy) on keybase.
  • I have a public key ASDl3d4SDUHEeEPFoE8m4x-J_Cohq7ZYhDKZ0eAHHq8I_Ao

To claim this, I am signing this object:

@Firefishy
Firefishy / docker-compose.yml
Created October 8, 2020 19:02
Using docker to get latest btrfs-progs
version: "2"
services:
app:
build:
context: .
dockerfile: Dockerfile
devices:
# Which disk(s) to expose to docker...
- "/dev/md1:/dev/md1"
10
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 888487 54220 122540 942707 54220 1788780.870 0
write: 1974923 2662 2666 1977585 2662 223280.255 0
verify: 487 17321 34884 17808 17321 133.236 0
Non-medium error count: 194
cd /path-to-dir/LUREF_NGL/; find . -type f -iname '*.tif' >input-files.txt
docker run -it --rm -v /path-to-dir/LUREF_NGL:/data osgeo/gdal:alpine-normal-v2.4.1 sh -l
(in docker) gdalbuildvrt -resolution highest -r nearest ANA_LUREF_NGL_DTM.vrt -input_file_list input-files.txt
# Rest not in docker...
gdaladdo -ro --config COMPRESS DEFLATE --config COMPRESS_OVERVIEW DEFLATE --config ZLEVEL 9 --config BIGTIFF_OVERVIEW IF_SAFER --config GDAL_TIFF_OVR_BLOCKSIZE 512 -r nearest ANA_LUREF_NGL_DTM.vrt 4 16 64 256 1024 4096
gdaldem hillshade ANA_LUREF_NGL_DTM.vrt lu_hillshade_2017.tif -co BIGTIFF=YES -co TILED=YES -co COMPRESS=DEFLATE -co GDAL_NUM_THREADS=ALL_CPUS -of GTiff -b 1 -z 1.0 -s 0.5 -az 315.0 -alt 45.0
gdaladdo -ro --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL --config BIGTIFF_OVERVIEW IF_SAFER --config GDAL_TIFF_OVR_BLOCKSIZE 512 -r average lu_hillshade_2017.tif 4 16 64 256 1024 4096
gdalwarp -t_srs epsg:3857 -r lanczos -multi -wo NUM_THREADS=ALL_CPUS A
@Firefishy
Firefishy / docker-compose.yml
Created October 21, 2019 16:44
PostgreSQL docker-compose managing a data volume
version: '3'
services:
db:
image: postgres:9
volumes:
- db-data:/var/lib/postgresql/data
volumes:
db-data:
@Firefishy
Firefishy / gist:928cfb2d4a4bd9d4dcbeb9e6aa5fd8a0
Created January 3, 2019 20:27
% OpenStreetMap Tile Bandwidth Traffic increase over last ~ 6 months
SN: 603.983%
TR: 480.698%
MY: 395.515%
SA: 229.806%
UY: 217.383%
VN: 177.495%
TH: 176.646%
DZ: 158.348%
PK: 156.190%
IR: 147.665%
2018-09-22 17:41:38.417 7f776c7c3700 -1 /build/ceph-13.2.1/src/osd/PGLog.h: In function 'static void PGLog::_merge_object_divergent_entries(const PGLog::IndexedLog&, const hobject_t&, mempool::osd_pglog::
list<pg_log_entry_t>&, const pg_info_t&, eversion_t, missing_type&, PGLog::LogEntryHandler*, const DoutPrefixProvider*) [with missing_type = pg_missing_set<true>; mempool::osd_pglog::list<pg_log_entry_t>
= std::__cxx11::list<pg_log_entry_t, mempool::pool_allocator<(mempool::pool_index_t)14, pg_log_entry_t> >]' thread 7f776c7c3700 time 2018-09-22 17:41:38.421384
/build/ceph-13.2.1/src/osd/PGLog.h: 921: FAILED assert(objiter->second->version > last_divergent_update)
ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f778f1554e2]
2: (()+0x2e26a7) [0x7f778f1556a7]
3: (void PGLog::_merge_object_divergent_entries<pg_missing_set<true> >(PGLog::IndexedLog const&, hobject_t const&, std::__cxx11::list<p
-2> 2018-09-22 17:56:11.830 7fe76e603700 5 -- 10.89.121.203:6809/8690 >> 10.89.121.202:6810/3429 conn(0x55fe26068300 :6809 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=118 cs=1 l=0). rx osd.10 s
eq 57 0x55fe25e41600 pg_query(1.1b4s5 epoch 41227) v4
-1> 2018-09-22 17:56:11.830 7fe76e603700 1 -- 10.89.121.203:6809/8690 <== osd.10 10.89.121.202:6810/3429 57 ==== pg_query(1.1b4s5 epoch 41227) v4 ==== 162+0+0 (2599670235 0 0) 0x55fe25e41600 con 0x55
fe26068300
0> 2018-09-22 17:56:11.830 7fe7533ca700 -1 /build/ceph-13.2.1/src/osd/PGLog.h: In function 'static void PGLog::_merge_object_divergent_entries(const PGLog::IndexedLog&, const hobject_t&, mempool::osd
_pglog::list<pg_log_entry_t>&, const pg_info_t&, eversion_t, missing_type&, PGLog::LogEntryHandler*, const DoutPrefixProvider*) [with missing_type = pg_missing_set<true>; mempool::osd_pglog::list<pg_log_e
ntry_t> = std::__cxx11::list<pg_log_entry_t, mempool::pool_allocator<(mempool::pool_index_t)14, pg_log_entry_t> >]' thread 7fe7533ca700 tim
-1> 2018-09-22 17:51:49.602 7f74bef84700 1 -- 10.89.121.204:6802/6710 <== osd.23 10.89.121.206:6809/3478 91 ==== pg_info((query:41162 sent:41162 1.91s0( v 5903'139734 (2469'136649,5903'139734] local-lis/les=41161/41162 n=66920 ec=71/51 lis/c 41152/40494 les/c/f 41153/40495/0 41157/41161/41017) 0->3)=([40494,41160] intervals=([40597,40598] acting 0(1),2(3),3(4),16(5),23(0)),([41017,41021] acting 1(2),2(3),3(4),6(1),8(5),23(0)),([41124,41130] acting 0(1),1(2),3(4),16(5),23(0)),([41157,41160] acting 0(1),1(2),2(3),16(5),23(0))) epoch 41162) v5 ==== 1384+0+0 (164758133 0 0) 0x5585a0bdea00 con 0x5585993cf500
0> 2018-09-22 17:51:49.602 7f74a2548700 -1 *** Caught signal (Aborted) **
in thread 7f74a2548700 thread_name:tp_osd_tp
ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable)
1: (()+0x9169e0) [0x558522b249e0]
2: (()+0x11390) [0x7f74c356f390]
3: (gsignal()+0x38) [0x7f74c2aa2428]
4: (abort()+0x16a) [0x7f74c2aa402a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int
-1> 2018-09-22 17:56:11.862 7fe760e04700 3 osd.5 41227 handle_osd_map epochs [41222,41227], i have 41227, src has [3142,41227]
0> 2018-09-22 17:56:11.874 7fe7533ca700 -1 *** Caught signal (Aborted) **
in thread 7fe7533ca700 thread_name:tp_osd_tp
ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable)
1: (()+0x9169e0) [0x55fda92789e0]
2: (()+0x11390) [0x7fe772bee390]
3: (gsignal()+0x38) [0x7fe772121428]
4: (abort()+0x16a) [0x7fe77212302a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7fe774559630]