Skip to content

Instantly share code, notes, and snippets.

View DonRichards's full-sized avatar

Don Richards DonRichards

  • Johns Hopkins University
  • Knoxville, TN
  • 09:12 (UTC -04:00)
  • X @highlander411
View GitHub Profile
@DonRichards
DonRichards / README.md
Created July 17, 2024 14:21
Super Res Down size image to use for Thumbnails

This make the image super clear and reduces the size to a width of 1024

Modified Real-ESRGAN/inference_realesrgan.py: Made changes to enhance debugging but might not be needed.

Incomplete bash history

pipenv shell
git clone https://github.com/xinntao/Real-ESRGAN.git
cd Real-ESRGAN
@DonRichards
DonRichards / hashes_merge_aws_logs.py
Created July 2, 2024 14:19
Processes multiple JSON files provided by S3 when copying a bucket and extract and display unique `RelativePath` and `SrcChecksum` pairs.
import os
import json
import pandas as pd
"""
This script processes multiple JSON files to extract and display unique `RelativePath` and `SrcChecksum` pairs.
1. Loads JSON files from a specified directory.
2. Extracts `RelativePath` and `SrcChecksum` from the `Verified` list in each JSON file.
3. Tracks unique `RelativePath` and their corresponding `SrcChecksum` to avoid duplicates.
4. Identifies and logs instances where the same `RelativePath` has different `SrcChecksum` values.
@DonRichards
DonRichards / check_port.sh
Created April 10, 2024 17:57
Troubleshooting isle-dc build conflicts.
#!/bin/bash
DOCKER_COMPOSE_FILE="docker-compose.yml"
# Check if docker-compose file exists
if [ ! -f "$DOCKER_COMPOSE_FILE" ]; then
echo "Docker Compose file not found at $DOCKER_COMPOSE_FILE"
exit 1
fi
# Extract ports from the 'ports' section of 'traefik' service
#!/bin/bash
# DRAFT: untested at the moment
# FITS File Processor for Dataverse Upload
# -----------------------------------------
# This script iterates through each .fits file in the specified directory, extracts the star number from
# the file name, and uses this information to construct a JSON payload. It then executes a curl command
# to upload each file to a specified Dataverse server using the provided API token and Persistent ID.
# The output of each curl command, along with relevant data, is logged to 'log.txt' for record-keeping
# and debugging purposes.
@DonRichards
DonRichards / dspace_queries.md
Last active September 25, 2023 20:56
Useful DSpace sql queries

Recently Created (within 30 days) that are missing TN files

SELECT 
    h.handle
FROM 
    handle h
JOIN
    item i
ON
    h.resource_id = i.uuid
@DonRichards
DonRichards / Grab_media_file_url_paths.md
Last active September 12, 2023 18:27
Use IDC's Metadata export CSV file to find and pull the Original File's URL path into the CSV.

Grab.py retrieves URLs for either "Intermediate Files" or "Original Files" and appends them to a CSV file.

The script includes a feature for resuming its progress in case of an interruption or error. As a precaution, it adjusts the URLs from the production server to redirect them to a staging server.

Additionally, the script incorporates error-handling mechanisms, utilizing a try-catch-retry approach, to gracefully manage timeouts.

Setup to run

python3 -m venv .venv
source .venv/bin/activate
@DonRichards
DonRichards / get_all_taxonomy_terms.sh
Last active March 20, 2023 18:07
Export all taxonomy terms for Islandora 2.x
#!/usr/bin/env bash
# This generates the header bases by what terms were returned.
# This makes merging them complicated.
curl https://islandora.traefik.me/content/taxonomy-terms/export > taxonomy_terms_000.csv
for i in {1..10000..100}
do
echo "Loop $i"
Starting diagnostics
[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?
@DonRichards
DonRichards / log
Last active December 19, 2022 21:52
make up
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5600 100 5600 0 0 13078 0 --:--:-- --:--:-- --:--:-- 13053
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1704 100 1704 0 0 5063 0 --:--:-- --:--:-- --:--:-- 5071
Building Drupal image with base: ghcr.io/jhu-sheridan-libraries/idc-isle-dc/drupal:upstream-20200824-f8d1e8e-83-g8c8a6b4
[+] Building 43.9s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0
Starting diagnostics
[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?