Skip to content

Instantly share code, notes, and snippets.

@linuxster
linuxster / ILP.ipynb
Last active August 9, 2023 16:43
ILP example in python
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import socket
import sys
import time
import struct
host = 'localhost'
port = 8888
buffersize = 1024
N = 1000000
server_address = (host, port)
@application2000
application2000 / how-to-install-latest-gcc-on-ubuntu-lts.txt
Last active May 7, 2024 10:38
How to install latest gcc on Ubuntu LTS (12.04, 14.04, 16.04)
These commands are based on a askubuntu answer http://askubuntu.com/a/581497
To install gcc-6 (gcc-6.1.1), I had to do more stuff as shown below.
USE THOSE COMMANDS AT YOUR OWN RISK. I SHALL NOT BE RESPONSIBLE FOR ANYTHING.
ABSOLUTELY NO WARRANTY.
If you are still reading let's carry on with the code.
sudo apt-get update && \
sudo apt-get install build-essential software-properties-common -y && \
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \
@wesm
wesm / parquet-benchmark-20170210.py
Created February 10, 2017 18:07
Parquet multithreaded benchmarks
import gc
import os
import time
import numpy as np
import pandas as pd
from pyarrow.compat import guid
import pyarrow as pa
import pyarrow.parquet as pq
import snappy
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@linar-jether
linar-jether / PySpark DataFrame from many small pandas DataFrames.ipynb
Created July 8, 2018 10:15
Convert a RDD of pandas DataFrames to a single Spark DataFrame using Arrow and without collecting all data in the driver.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@andyweizhao
andyweizhao / cuda_installation_on_ubuntu_18.04
Last active October 11, 2021 17:56 — forked from Mahedi-61/cuda_11.8_installation_on_Ubuntu_22.04
cuda 9.0 complete installation procedure for ubuntu 18.04 LTS
#!/bin/bash
## This gist contains step by step instructions to install cuda v9.0 and cudnn 7.3 in ubuntu 18.04
### steps ####
# verify the system has a cuda-capable gpu
# download and install the nvidia cuda toolkit and cudnn
# setup environmental variables
# verify the installation
###
#!/usr/bin/env bash
set -eu
PWD="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
SRC_DIR=$(realpath "${PWD}/..")
CXX_SRC=${SRC_DIR}/cpp
# The following can be set
: "${CMAKE:=cmake}"
@jasonrig
jasonrig / run_spark_cluster.sh
Created August 9, 2019 03:47
Example SLURM job script to start a Spark cluster
#!/bin/bash
#SBATCH --job-name spark-cluster
#SBATCH --account=qh82
#SBATCH --time=02:00:00
# --- Master resources ---
#SBATCH --nodes=1
#SBATCH --mem-per-cpu=1G
#SBATCH --cpus-per-task=1
#SBATCH --ntasks-per-node=1
# --- Worker resources ---
@cjnolet
cjnolet / cuml-kmeans-mnmg-api.md
Last active August 17, 2022 05:35
Simple example of cuML's K-Means Single-GPU (SG) and Multi-Node Multi-GPU (MNMG) APIs compared to Scikit-learn and Dask-ML

Comparing cuML K-Means API Against Scikit-learn & Dask-ML

First, a quick code example of K-Means in Scikit-learn

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans

n_centers = 5

X, _ = make_blobs(n_samples=10000, n_centers=n_centers)