Skip to content

Instantly share code, notes, and snippets.

Avatar

Tylar 7yl4r

View GitHub Profile
View notes.md

next steps

  1. find funding for 2030 effort
  2. producing products (product = map layers in AWS)

product production details

  1. want to auto-update the layer when data submitted 2. crowd-source submission system goes to AWS then converted to xyz data files in AWS buckets 1. then can use VM in paperspace via parsec 4. not working in ArcGIS bc it is too big - too many polygons (300billion +).
  2. tried ArcOnline & Enterprise also. Enterprise could maybe do it w/ more cores.
View example.html
<!doctype html>
<html lang="en-gb" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
<title>|7y|4r|</title>
<meta name="description" content="web portfolio">
<meta name="author" content="7yl4r">
<link rel="stylesheet" href="css/bootstrap.min.css" />
View python-r-tooling-map.md
language package manager configuration file(s) Package index environment manager
R . DESCRIPTION cran packrat
python pip Pipfile, requirements.txt pypi pipenv
python conda environment.yml conda-forge .
@7yl4r
7yl4r / list.md
Created Dec 10, 2021
sat image display examples by starting filetype
View list.md
@7yl4r
7yl4r / gam_excerpt.R
Last active Aug 16, 2021
Cara's RVC GAM snippet
View gam_excerpt.R
# for each column in data table
richness.partial.deviance.list = for(j in names(gam_data[,3])) {
# ??? why are we looping here?
# ??? Is this code not the same as setting j="abundance_sum" and not looping?
# ??? Maybe there are multiple columns with name "abundance_sum"? Seems odd.
if(j=="abundance_sum") { # only this one abundance column
# fit a GAM with the following model:
# $y_i = \alpha + f_1(X_1i) + f_2(X_2i) + g_1( X_3i ) + g_2(X_4i ) + \epsilon_i$
# $y_i = \alpha + f_1(YEAR) + f_2(PROT) + g_1(LONG, LAT)* YEAR + g_2(STRAT) + \epsilon_i$
# ??? why are some of these f_* and others g_*?
@7yl4r
7yl4r / create_subsample_ids.R
Last active Feb 22, 2021
Creates an OBJECTID and Sample_ID for "child" rows that don't have them using "parent" rows as lookup by lat+lon+date.
View create_subsample_ids.R
# Creates an OBJECTID and Sample_ID for "child" rows that don't have them using "parent" rows as lookup by lat+lon+date.
# """
# Child rows are identified by OBJECTID == 0.
# Pseudocode:
# read in OJBECT != 0 lat/lon/date & IDs map & + increment starting @ 1
# for row OJBECT == 0:
# OJBECT_ID = lookup lat / lon / date in our map
# """
#
# NOTE: this doesn't work. We gave up after finding that the subsample rows were not actually subsamples.
View ImportError trace
# === config details:
[airflow@imars-airflow01 imars_dags]$ echo $AIRFLOW_HOME
/home/airflow
# === relevant excerpt from `tree`
[airflow@imars-airflow01 ~]$ tree $AIRFLOW_HOME/dags
/home/airflow/dags
└── imars_dags
├── imars_dags
@7yl4r
7yl4r / triple_donut.py
Created Feb 17, 2020
Create donut plot split into 3 sections.
View triple_donut.py
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots()
size = 0.3
vals = [
25,
50,
75
View wv-proc-setup.md

install PGC's imagery_utils

# set up gdal
# NOTE: could maybe use https://github.com/PolarGeospatialCenter/gdal-full ?
sudo apt install -y gdal-bin libgdal-dev python3-pip
pip3 install "pygdal==$(gdal-config --version).*"
# TODO: more here probably...

# install code from github
git clone https://github.com/7yl4r/imagery_utils
@7yl4r
7yl4r / file_move.md
Last active Jan 27, 2020
how to move selected files from text file listing one file per line
View file_move.md

Use rsync, then move files out of folders and delete empty folders:

# starting in /tpa_pgs/
mkdir fl_se
rsync -avzh --recursive --files-from=./files_to_move/files_to_move_sefl.txt ./pgc_batches/imagery/ ./fl_se/.

# cd to region and cleanup if needed:
find ./ -mindepth 2 -type f -exec mv {} . \;
find ./ -type d -empty -delete