Skip to content

Instantly share code, notes, and snippets.

View thvasilo's full-sized avatar

Theodore Vasiloudis thvasilo

View GitHub Profile
@m-radzikowski
m-radzikowski / script-template.sh
Last active May 4, 2024 04:13
Minimal safe Bash script template - see the article with full description: https://betterdev.blog/minimal-safe-bash-script-template/
#!/usr/bin/env bash
set -Eeuo pipefail
trap cleanup SIGINT SIGTERM ERR EXIT
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd -P)
usage() {
cat <<EOF
Usage: $(basename "${BASH_SOURCE[0]}") [-h] [-v] [-f] -p param_value arg1 [arg2...]
@jpbarto
jpbarto / jpbarto-sagemaker-lifecycle.sh
Last active June 10, 2020 15:40
Default lifecycle configuration for SageMaker notebooks
#!/bin/bash
#
# This script takes about 5 minutes to run and will delay the availability of
# your SageMaker notebook.
# v1.3, release 05may2020
#
# THIS SCRIPT INSTALLS:
# - THEIA IDE w/ Jupyter Server proxy
# - UPGRADES JUPYTERLAB
# - INSTALL VOILA (https://github.com/voila-dashboards/voila)
@jpbarto
jpbarto / ide-lifecycle.sh
Last active May 15, 2020 20:03
Lifecycle script for SageMaker notebook startup which installs Theia-IDE
#!/bin/bash
set -e
sudo -u ec2-user -i <<'EOP'
## INSTALL THEIA IDE FROM SOURCE
EC2_HOME=/home/ec2-user
mkdir ${EC2_HOME}/theia && cd ${EC2_HOME}/theia
### begin by installing NVM, NodeJS v10, and Yarn
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.5/install.sh | bash

A Tour of PyTorch Internals (Part I)

The fundamental unit in PyTorch is the Tensor. This post will serve as an overview for how we implement Tensors in PyTorch, such that the user can interact with it from the Python shell. In particular, we want to answer four main questions:

  1. How does PyTorch extend the Python interpreter to define a Tensor type that can be manipulated from Python code?
  2. How does PyTorch wrap the C libraries that actually define the Tensor's properties and methods?
  3. How does PyTorch cwrap work to generate code for Tensor methods?
  4. How does PyTorch's build system take all of these components to compile and generate a workable application?

Extending the Python Interpreter

PyTorch defines a new package torch. In this post we will consider the ._C module. This module is known as an "extension module" - a Python module written in C. Such modules allow us to define new built-in object types (e.g. the Tensor) and to call C/C++ functions.

@exit99
exit99 / 2to3
Created January 5, 2016 21:27
Bash cmd for upgrading all python files recursively to python3
find . -exec ls -dl \{\} \; | grep ".*\.py" | awk '{print $9}' | xargs 2to3 -w
@dgrtwo
dgrtwo / party_df_tidiers.R
Created November 10, 2015 23:55
Tidy distributed models on a partitioned data frame from dplyr
## helper and setup functions
wrap_party_df <- function(func) {
function(x, object, ...) {
n <- col_name(substitute(object))
# have to create an expression since we cannot rely on local
# variables
args <- list(...)
expr <- substitute(do.call(func, c(list(.[[colname]][[1]]), args)),
@jarutis
jarutis / blas.sh
Last active May 26, 2016 22:30
get native blas running with dl4j on centos 6
# install general deps
sudo yum groupinstall "Development Tools"
sudo yum install wget unzip
# install java
cd /opt/
sudo wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.tar.gz"
sudo tar xzf jdk-8u60-linux-x64.tar.gz
cd /opt/jdk1.8.0_60/
@soulslicer
soulslicer / GPU-EC2.sh
Created July 9, 2014 21:17
OpenCL on GPU instance
# This script installs everything you need on an EC2 GPU Instance
# Create an Ubuntu 12.04 GPU Instance with at least 16GB of free space
# Base packages
sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install gcc git cmake
# CUDA/Nvidia installations
wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1204/x86_64/cuda-repo-ubuntu1204_6.0-37_amd64.deb
@computermacgyver
computermacgyver / density_functions.R
Last active June 23, 2016 17:36
Function for R and ggplot2 to create log scale density plots from dataframe, spiting on a factor. #R #ggplot2 #densityPlot #GPLv2 #CHI2014
#!/usr/bin/R
# Functions for R and ggplot2 to create log-scale density plots
# Main function: density.log(...)
# Produce a dataframe used to produce a density plot
# Input:
# data: dataframe,
# var: variable to plot on y-axis
# split: factor to split on