Skip to content

Instantly share code, notes, and snippets.

@mrkatey
Last active February 16, 2024 12:40
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save mrkatey/d069bb8f9e4d7f61fb82562d2bde8f7b to your computer and use it in GitHub Desktop.
Save mrkatey/d069bb8f9e4d7f61fb82562d2bde8f7b to your computer and use it in GitHub Desktop.
Tons of Awesome Resources (data, ai etc.)

Links

Learning

Stable Diffusion

Intro video to ControlNet

Linear Algebra A-Z

Linear Algebra Essentials

Interactive Linear Algebra

Linear Algebra for Deep Learning

Archives

A Archive

Image Matching

Egocentric to Aerial 2021

Image Processing

Stereo Photogrammetry

Paper: Image Deblur survey of techniques

DEM from Photos: PhotoMOD Lite

QGIS Object Detection Plugin: Mapflow

Physics-Based OpenCV: PhyCV

GIMP ML

Single-shot object detection

Train single-shot object detector

Free and Opensource Inpainting Solution

Secure Programming

RUST (language)

Black-Hat Rust

Rustlings

Rust by Example

Real-World Geoviewing / Related

Depth map from 2 images

Intro to Stereo Views and Rectification

Bundler Structure from Motion for "random" images

MicMac Software

Camera Transform

Georeference any image onto 3D plane

Super-Glue Image Matching/Search

Image search based on SURF feature matching

UE + Geo

Search Engine Creation

Whoosh Search Engine Creator

Encryption

Digital Dissent

Unclonable Functions

Transcription

Whisper

Voice Cloning

Voice Clone Tool

Image to Image Translation

Diffusion based approaches

Diffusers Library

Cycle Diffusion

MCT

Palette: Image to Image

NER

12-class NER Model

Training Custom NER Model

Simple to follow tutorial

HuggingFace Tutorials

Hugging Face Tutorials: Custom Data

Hugging Face Tutorials: Token Classification

Courses

Hacking

Intro to Hacking

Data Analysis

Spatial Analysis

Geographic Data Analysis

Geographic Data Science

Complexity Science

Ton of Links

https://www.complexityexplorer.org/courses/98-introduction-to-dynamical-systems-and-chaos-2019/segments/8559?summary

https://www.youtube.com/watch?v=ZiwogMtbjr4&list=PLoROMvodv4rOca_Ovz1DvdtWuz8BfSWL2

Stochastic Processes

Chaos and Markets (papers)

https://pure.uva.nl/ws/files/1805206/4_ltlapril2001.pdf

Deep Learning

Deep Learning for Coders

Suggested Reading

Process 100GB+ files on a laptop

Early Internet Paper by NSA

https://www.quantamagazine.org/symbolic-mathematics-finally-yields-to-neural-networks-20200520/

Books

https://direct.mit.edu/books/oa-monograph/5335/Power-of-PositionClassification-and-the https://direct.mit.edu/books/book/5338/Artificial-CommunicationHow-Algorithms-Produce https://direct.mit.edu/books/oa-edited-volume/5331/Cognitive-Robotics https://direct.mit.edu/books/oa-monograph/5329/The-Art-of-Abduction https://direct.mit.edu/books/oa-edited-volume/5312/Computational-Thinking-Education-in-K-12Artificial https://direct.mit.edu/books/oa-monograph/5281/Social-EngineeringHow-Crowdmasters-Phreaks-Hackers https://direct.mit.edu/books/oa-monograph/5255/Treacherous-Play https://direct.mit.edu/books/oa-monograph/5243/Teaching-Computational-ThinkingAn-Integrative https://direct.mit.edu/books/book/5205/Command-and-PersuadeCrime-Law-and-the-State-across https://direct.mit.edu/books/book/5093/The-Constitution-of-AlgorithmsGround-Truthing https://direct.mit.edu/books/book/4932/Distributed-LedgersDesign-and-Regulation-of https://direct.mit.edu/books/book/4850/Digital-Entrepreneurship-in-AfricaHow-a-Continent https://direct.mit.edu/books/book/4850/Digital-Entrepreneurship-in-AfricaHow-a-Continent https://direct.mit.edu/books/book/4850/Digital-Entrepreneurship-in-AfricaHow-a-Continent https://direct.mit.edu/books/book/4850/Digital-Entrepreneurship-in-AfricaHow-a-Continent https://direct.mit.edu/books/book/5035/Translational-NeuroscienceToward-New-Therapies https://direct.mit.edu/books/oa-monograph/5345/The-Art-of-Insight-in-Science-and https://direct.mit.edu/books/oa-monograph/5342/BoostingFoundations-and-Algorithms https://direct.mit.edu/books/oa-monograph/5339/Street-Fighting-MathematicsThe-Art-of-Educated https://direct.mit.edu/books/book/4339/Artificial-Life-IXProceedings-of-the-Ninth https://direct.mit.edu/books/book/4339/Artificial-Life-IXProceedings-of-the-Ninth https://direct.mit.edu/books/book/5066/Why-Architects-Draw https://direct.mit.edu/books/book/4663/Turtle-GeometryThe-Computer-as-a-Medium-for https://www.ime.unicamp.br/~dias/Intoduction%20to%20Statistical%20Learning.pdf https://hastie.su.domains/CASI_files/PDF/casi.pdf https://github.com/ghoshark/E_Books/blob/master/Practical%20Statistics%20for%20Data%20Scientists.pdf https://www.book2look.com/embed/9781317526988

Think x books

https://greenteapress.com/wp/think-complexity-2e/

Bayesian Statistics

https://directory.doabooks.org/handle/20.500.12854/32870 https://directory.doabooks.org/handle/20.500.12854/84560 https://directory.doabooks.org/handle/20.500.12854/74880 https://directory.doabooks.org/handle/20.500.12854/80958 https://directory.doabooks.org/handle/20.500.12854/76386 https://directory.doabooks.org/handle/20.500.12854/68899 https://directory.doabooks.org/handle/20.500.12854/30625 https://directory.doabooks.org/handle/20.500.12854/78483 https://directory.doabooks.org/handle/20.500.12854/50220 https://directory.doabooks.org/handle/20.500.12854/76874 https://directory.doabooks.org/handle/20.500.12854/68375 https://directory.doabooks.org/handle/20.500.12854/77055

Bayes for Hackers

Complexity Science

https://directory.doabooks.org/handle/20.500.12854/27697 https://directory.doabooks.org/handle/20.500.12854/53742 https://directory.doabooks.org/handle/20.500.12854/74901

Papers on Complexity Science (all free)

https://www.pnas.org/doi/epdf/10.1073/pnas.0900943106 https://www.pnas.org/doi/pdf/10.1073/pnas.0908800106 https://ndg.asc.upenn.edu/wp-content/uploads/2016/04/Centola-et-al-2007-PA.pdf https://www.pnas.org/doi/pdf/10.1073/pnas.122653799 https://barabasi.com/f/632.pdf

Interesting papers

D. Hosfelt. “Fearless security: memory safety.” Mozilla Hacks.

D. Hosfelt. “Fearless security: thread safety.” Mozilla Hacks.

S. Klabnik and C. Nichols. “What Is Ownership?.” The Rust Programming Language.

S. Klabnik and C. Nichols. “References and Borrowing.” The Rust Programming Language.

S. Klabnik and C. Nichols. “The Slice Type.” The Rust Programming Language.

K. McAllister. “Memory exploit mitigations #15179.” GitHub.

K. McAllister. “RFC: Memory exploit mitigation.” GitHub.

D. Micay. “Enable PIE by default on Linux for full ASLR #16340.” GitHub.

N. Matsakis. “Integer overflow #560.” GitHub.

G. Lehel and N. Matsakis. “Integer overflow.” GitHub.

A. Turon. “Tracking issue for integer overflow (RFC 560) #22020.” GitHub.

H. Wilson. “Myths and legends about integer overflow in Rust.” Huon on the Internet.

B. Anderson. “Stabilize -C overflow-checks #1535.” GitHub.

B. Anderson. “Stable overflow checks.” GitHub.

N. Froyd. “Add -C overflow-checks option #40037.” GitHub.

R. Á. de Espíndola. “rustc requires executable stack #798.” GitHub.

A. Seipp. “Make sure librustrt.so is linked with a non-executable stack. #1066.” GitHub.

D. Micay. “Rust binaries should not have an executable stack #5643.” GitHub.

D. Micay. “Mark the assembly object stacks as non-executable #5647.” GitHub.

A. Clark. “Explicitly disable stack execution on linux and bsd #30859.” GitHub.

S. Klabnik and C. Nichols. “Unsafe Rust.” The Rust Programming Language.

“Improper restriction of operations within the bounds of a memory buffer (CWE-119).” MITRE CWE List.

“Concurrency issues (CWE-557).” MITRE CWE List.

S. Davidoff. “How Rust’s standard library was vulnerable for years and nobody noticed.” Medium.

https://ocw.mit.edu/courses/6-042j-mathematics-for-computer-science-fall-2010/93cad640cf3ed0b23ef70688f452d4d5_MIT6_042JF10_notes.pdf

Spatio-temporal Analysis

Models/Tools/ML Stuff

Research Management

3D from single image !!!

Contrast Enhancement !!!

DEM from Stereo Aerial !!!

import otbApplication

# Set the names of the input images
image1 = "image1.tif"
image2 = "image2.tif"

# Create an instance of the StereoRectification application
app = otbApplication.Registry.CreateApplication("StereoRectification")

# Set the parameters for the application
app.SetParameterString("inp", image1)
app.SetParameterString("inp2", image2)
app.SetParameterString("outp", "dem.tif")

# Execute the application
app.ExecuteAndWriteOutput()

print("DEM generated successfully!")

Image Similarity Search Library

Graph-Tool: A Python module for Graph data

Graph-Tool Docs

SBERT

VADER: Sentiment Analysis

Datasets

Documents

List of data sets

Massive Collection of Twitter Data Sets

Interesting but small datasets

Archive.org Curated Dataset List

Massive Collection of Curated Datasets

Long list of curated datasets available for download

Single/limited datasets

750 GB of Common Crawl Text without Boilerplate

Usenet archive

GeoCities

18 GB Scientific Research 1,000,000 papers + Citation links

1.7 M Research Papers

1 M News Headlines

1 M News Headlines 2

Reddit Comments up to more than 30GB!

10 Million web domains by Page Crawl Ranking

Open Sanctions Dataset

355 Billion Rows of data extracted from Research Papers

Biographies

2.5 Million biographies

1.7 Billion Reddit Comments

Tools

Deploy ML Model using Grad.io

Linux Server Security

Tutorials

ArcGis for Python

How to secure a Linux server

Convert Tabular Dataset to Graph Dataset

Random Research Papers

Customs Coding Automation using ML

Customs Coding Automation #2

Automated Tree Building

Pen-Testing

Practice/Training

Pen Testing Example Resource

Cultural Research

Films featuring Surveilance

Films featuring cybersecurity

Site Generators

FastAPI-Vue

Generate better SD prompts using ChatGPT

Stable Diffusion Prompt Improvement Engine
Readme: Takes as input a description of an image and improves on it by converting any non-specific elements into concrete objects that can easily be interpreted.

Videos

Image editing using Diffusion


Hardware

PC Cases

Mini Tower

Deployment

Code

DO

Graph Neural Network

Papers

Survey of Graph Neural Networks and Applications Graph Deep Learning: State of the Art and Challenges

Automated Approaches

AutoGraphLearning Framework

@noveing
Copy link

noveing commented Jan 2, 2024

@noveing
Copy link

noveing commented Feb 16, 2024

10 Science Books:
"Feynman Lectures on Physics" by Richard P. Feynman, Robert B. Leighton, and Matthew Sands
Math requirement: Basic algebra, trigonometry, and calculus
"Classical Mechanics" by Serway, Jewett, and Levitt
Math requirement: Basic algebra, trigonometry, and calculus
"Introduction to Thermal Physics" by Daniel V. Schroeder
Math requirement: Basic algebra, trigonometry, and calculus
"Classical Mechanics" by J. E. Shockley
Math requirement: Basic algebra, trigonometry, and calculus
"Classical Mechanics" by T. W. B. Kibble and B. Carter
Math requirement: Basic algebra, trigonometry, and calculus
"Principles of Quantum Mechanics" by David J. Griffiths
Math requirement: Basic algebra, trigonometry, and calculus, with some linear algebra and differential equations
"Electricity and Magnetism" by John D. Jackson
Math requirement: Basic algebra, trigonometry, and calculus, with some linear algebra and differential equations
"Classical Dynamics of Particles and Systems" by Herbert Mark Kuhn and Robert L. Morse
Math requirement: Basic algebra, trigonometry, and calculus, with some linear algebra and differential equations
"Fundamentals of Physics" by Halliday, Resnick, and Walker
Math requirement: Basic algebra, trigonometry, and calculus, with some linear algebra and differential equations
"Classical Mechanics" by Herbert Goldstein, Charles P. Poole, and John Safko
Math requirement: Basic algebra, trigonometry, and calculus, with some linear algebra and differential equations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment