Skip to content

Instantly share code, notes, and snippets.

View rjurney's full-sized avatar

Russell Jurney rjurney

View GitHub Profile
@rjurney
rjurney / networkx_matches_dgl.py
Created September 29, 2023 03:30
I made a networkx network and a DGL network. They match :) I am pleased.
# The DGL network has sentence encoded JSON for node features.
In [26]: g
Out[26]:
[Graph(num_nodes=27770, num_edges=352807,
ndata_schemes={'x': Scheme(shape=(384,), dtype=torch.float64)}
edata_schemes={})]
# The networkx network parsed the records and has inidividual fields for analysis
In [27]: G
Out[27]: <networkx.classes.digraph.DiGraph at 0x14fe5c4c0>
@rjurney
rjurney / dmesg
Last active September 26, 2023 20:45
Heracles Ubuntu 20.04 Logs - Machine that Shuts Down for No Reason
# ChatGPT says:
1) The NVIDIA module is having trouble retrieving NUMA (Non-Uniform Memory Access) node information for a specific device. NUMA is a method for setting up memory in multiprocessor systems to better match memory access times with processor cores. This might be related to the multi-GPU setup you mentioned.
2) The NVIDIA kernel module for your graphics cards is being loaded. The version is 530.30.02, and it was compiled on Wed Feb 22 04:11:39 UTC 2023.
3) This is a warning from the ACPI subsystem in the kernel, related to device configuration and power management. The specific warning pertains to an argument type mismatch for a given method. Such ACPI warnings can result from inconsistencies between the computer's BIOS/UEFI firmware and the OS's expectations.
4) Lastly, the nvidia-drm, nvidia_uvm, and other NVIDIA related entries refer to the NVIDIA graphics driver's components being initialized. The Direct Rendering Manager (drm) is used in graphics pipelines, while nvidia_uvm refers to NVIDIA'
@rjurney
rjurney / academic.py
Last active September 26, 2023 13:04
Q&A on all your academic papers…
import logging
import os
from langchain.chains import ConversationalRetrievalChain
from langchain.document_loaders import PyPDFDirectoryLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.llms import OpenAI
from langchain.memory import ConversationBufferMemory
from langchain.vectorstores import Chroma
@rjurney
rjurney / centrality.groovy
Created September 9, 2023 01:37
Feature engineering using JanusGraph for Relato's business graph - customer recommender system in 2014
// Use this to start up a session
conf = new BaseConfiguration()
conf.setProperty("storage.directory", "/Users/rjurney/Software/marketing/titan/data")
conf.setProperty("storage.backend", "berkeleyje")
graph = TitanFactory.open(conf)
// Get a graph traverser
g = graph.traversal()
// Various centralities to use as features - JSONize and save
@rjurney
rjurney / Dockerfile
Last active August 10, 2023 03:50
Dockerfile to use a named Anaconda Python 'conda' environment as a kernel in a Jupyter notebook with Poetry
# Start from a Jupyter Docker Stacks version
FROM jupyter/scipy-notebook:python-3.10.11
# Work in the jovyan user's home directory
WORKDIR "/home/${NB_USER}"
# Needed for poetry package management: no venv, latest poetry, GRANT_SUDO don't work :(
ENV POETRY_VIRTUALENVS_CREATE=false \
POETRY_VERSION=1.4.2 \
GRANT_SUDO=yes
@rjurney
rjurney / assortativity.md
Created July 6, 2023 06:35
What is wrong with this Markdown? Why won't Jupyter parse it?

Assortativity

Assortativity in networks refers to a correlation pattern observed in real-world networks where nodes are preferentially connected to other nodes that are like (or unlike) them in some way. This is essentially a bias in connection preference.

--ChatGPT4

A related term is assortative mixing:

In the study of complex networks, assortative mixing, or assortativity, is a bias in favor of connections between network nodes with similar characteristics. In the specific case of social networks, assortative mixing is also known as homophily. The rarer disassortative mixing is a bias in favor of connections between dissimilar nodes.

@rjurney
rjurney / colorPalette.js
Created June 17, 2016 02:33
Kelly's 22 Colours of Maximum Contrast
var colorPalette = [“#fdfdfd”, “#1d1d1d”, “#ebce2b”, “#702c8c”, “#db6917”, “#96cde6”, “#ba1c30”, “#c0bd7f”, “#7f7e80”, “#5fa641”, “#d485b2”, “#4277b6”, “#df8461”, “#463397”, “#e1a11a”, “#91218c”, “#e8e948”, “#7e1510”, “#92ae31”, “#6f340d”, “#d32b1e”, “#2b3514”];
@rjurney
rjurney / train_test_dev_split.py
Created October 17, 2019 22:33
How to create a 0.7/0.2/0.1 Train/Test/Dev split of a dataset
from sklearn.model_selection import train_test_split
X_train, X_test_dev, y_train, y_test_dev = train_test_split(
df['_Body'],
df['_Index'],
test_size=0.3,
random_state=1337,
)
X_dev, X_test, y_dev, y_test = train_test_split(
X_test_dev,
@rjurney
rjurney / poetry.toml
Last active April 12, 2023 21:46
Poetry update can't solve a Python 3.9+ project with JUST pytest as a dev dependency... what is going on here?
[virtualenvs]
create = false
@rjurney
rjurney / start_dask.sh
Last active February 9, 2023 07:52
A script to start single node Dask with as many workers as your machine has processor cores
#!/bin/bash
# Launch the scheduler
nohup dask scheduler --host 127.0.0.1 --port 9000 --protocol tcp --dashboard --no-show 2>&1 >> /tmp/dask.log &
nohup nohup dask worker --nworkers=-1 tcp://127.0.0.1:9000 2>&1 >> /tmp/dask.log &