Skip to content

Instantly share code, notes, and snippets.

View msalvaris's full-sized avatar

Mat msalvaris

48 followers · 31 following
View GitHub Profile
@msalvaris
msalvaris / python logging
Created November 4, 2018 19:53
def _setup_logger(debug=False):
logger = logging.getLogger(__name__)
level = logging.DEBUG if debug else logging.ERROR
logger.setLevel(level)
sh = logging.StreamHandler(sys.stdout)
sh.setFormatter(logging.Formatter(logging.BASIC_FORMAT))
logger.addHandler(sh)
@msalvaris
msalvaris / svgbug.html
Created July 3, 2018 10:40
<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />
<title>SVGBug</title><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<style type="text/css">
/*!
*
* Twitter Bootstrap
@msalvaris
msalvaris / MPI Debug output
Created June 27, 2018 10:07
Horovod Debug Output
[0] MPI startup(): Intel(R) MPI Library, Version 2017 Update 3 Build 20170405 (id: 17193)
[0] MPI startup(): Copyright (C) 2003-2017 Intel Corporation. All rights reserved.
[0] MPI startup(): Multi-threaded optimized library
[0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[4] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[5] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
[6] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-ib0
@msalvaris
msalvaris / mount_network_drive
Created March 7, 2018 13:40
Mount a Network drive on Windows to WSL
# from https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/
sudo mkdir /mnt/d
sudo mount -t drvfs D: /mnt/d
@msalvaris
msalvaris / open_ports
Last active March 8, 2018 14:33
#!/usr/bin/env bash
# Opens the usual ports
name=msvm
resource_group=msvmrg
az vm open-port -g $resource_group -n $name --port 9999 --priority 1010 # Jupyter
az vm open-port -g $resource_group -n $name --port 6006 --priority 1050 # Tensorboard
az vm open-port -g $resource_group -n $name --port 3000 --priority 1060 # Grafana
@msalvaris
msalvaris / resize_os_disk
Last active March 16, 2018 14:35
Resizes VM OS Disk
#!/usr/bin/env bash
# Resizes os disk of existing vm
# Example
# resize_os_disk myVM myResourceGroup
name=$1 # myVM
resource_group=$2 # myResourceGroup
# Stop VM
az vm deallocate --resource-group $resource_group --name $name
@msalvaris
msalvaris / delete_rg
Last active March 2, 2018 16:36
Delete resource group
#!/usr/bin/env bash
# Deletes resource group, useful for deleting VMs etc.
# arguments:
# resource_group_name
az group delete --name $1
@msalvaris
msalvaris / install_azcopy
Last active August 8, 2018 15:18
Install AzCopy on Linux
#!/usr/bin/env bash
# Install with sudo
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial main" > /etc/apt/sources.list.d/dotnetdev.list'
apt-get update -y
apt-get install -y dotnet-sdk-2.0.2
wget -O azcopy.tar.gz https://aka.ms/downloadazcopyprlinux
@msalvaris
msalvaris / create_vm
Last active April 2, 2018 09:59
Create a vm
#!/usr/bin/env bash
# Sample script to create Ubuntu DSVM
# We specify a large os disk because by default it is 50GB which is often insufficient
location=eastus
resource_group=msvmrg
name=msvm
az group create --location $location --name $resource_group
@msalvaris
msalvaris / lstm_utils.py
Created April 27, 2017 14:45
A simple function to generate variable window sequences for LSTMs
def _generate_start_and_end(num_elements, sequence_length):
start_gen = chain(repeat(0, times=sequence_length),
range(1, num_elements))
end_gen = chain(range(1, sequence_length),
range(sequence_length, num_elements))
for start, stop in zip_longest(start_gen, end_gen, fillvalue=num_elements):
yield start, stop
def generate_variable_window(timeseries_array, sequence_length):