Skip to content

Instantly share code, notes, and snippets.

View nickumia-reisys's full-sized avatar
🐢

Nicholas Kumia nickumia-reisys

🐢
View GitHub Profile
@nickumia-reisys
nickumia-reisys / 01-fcs-catalog-db-backup.sh
Last active December 10, 2021 18:26
Data.gov catalog migration script
#!/bin/bash
# Run this script on FCS catalog-harvester-xyz
set -o errexit
set -o pipefail
set -o nounset
AWS_DEFAULT_REGION=us-gov-west-1
service_name=catalog-db
@nickumia-reisys
nickumia-reisys / solr_index.sh
Last active December 14, 2021 06:19
Solr Index Migration
#!/bin/bash
package_list=($(jq -r '.result' package_list.json))
function check_status () {
while ! (cf tasks catalog | grep -q "$1 .*\(SUCCEEDED\|FAILED\)"); do
sleep 1
done
if (cf tasks catalog | grep -q "$1 .*SUCCEEDED"); then
echo "Success: $2"
@nickumia-reisys
nickumia-reisys / Dockerfile
Last active May 20, 2022 15:37
Local Solr Server with Auth
FROM solr:8.11-slim
ADD security.json $SOLR_HOME
@nickumia-reisys
nickumia-reisys / convert_relevant_content.js
Last active August 1, 2022 21:50
Figure out what webpages are duplicates in data.gov website migration
var TurndownService = require('turndown')
const fs = require('fs');
fs.readFile(process.argv[2], 'utf8', (err, data) => {
if (err) {
console.error(err);
return;
}
@nickumia-reisys
nickumia-reisys / md2json.py
Last active August 18, 2022 03:02
Maybe I'll update this description when I'm less tired :(
import json, sys
json_build = {}
with open('summary', 'r') as summ:
items = summ.readlines()
for i in items:
title = i.split(']')[0].split('[')[-1]
link = i.split('](#')[1].split(')')[0]
@nickumia-reisys
nickumia-reisys / .gitignore
Last active November 28, 2022 19:43 — forked from nickumia/datagov_cluster.py
Download all keywords from catalog.data.gov
*.pkl
from nltk.corpus import wordnet
import json
import re
def compare_two_words(a, b):
aw = wordnet.synsets(a)
bw = wordnet.synsets(b)
# Take the first definition
import json
import xmltodict
with open('dmarc.xml', 'rb') as d:
dmarc = xmltodict.parse(d)
success = 0
fail = 0