kylegallatin

Kubernetes Practical Intro for Data Scientists

Today we're going to show you how to deploy an application on Kubernetes. You'll only need to Docker Desktop installed locally with Kubernetes enabled.

Testing Our Application with Docker

To deploy an app on Kubernetes, we first need to make sure it works locally with Docker. Use the example fastapi app here: https://github.com/kylegallatin/fast-bad-ml

# clone repo

zoellner

Note: There are better ways to do this by now. Check https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information for details

In order to access packages in private github repositories a Dockerfile might contain statements like this:

RUN git config --global url."https://${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"
RUN npm install --ignore-scripts --quiet && npm cache clean --force
RUN git config --global --unset url."https://${GITHUB_TOKEN}@github.com/".insteadOf

benmarwick

# Adapted from https://stackoverflow.com/a/7267364/1036500 by Andrie de Vries

# This is it: theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0.5))

library(ggplot2)

td <- expand.grid(
  hjust=c(0, 0.5, 1),
  vjust=c(0, 0.5, 1),
  angle=c(0, 45, 90),

brson

Edit: This list is now maintained in the rust-anthology repo.

• Introduction
  • Understanding Over Guesswork
  • An Alternative Introduction to Rust
  • Rust and CSV Parsing
• Ownership
  • Where Rust Really Shines
  • Rust Means Never Having to Close a Socket
• The Problem with Single-threaded Shared Mutability

john-science

Reading & Writing GZIP Files in Python

I have been testing various ways to read and write text files with GZIP in Python. There were a lot of uninteresting results, but there were two I thought were worth sharing.

Writing GZIP files

If you have a big list of strings to write to a file, you might be tempted to do:

f = gzip.open(out_path, 'wb')

for line in lines:

vasanthk

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

marcelm

#!/usr/bin/env python3
"""
Running this script is (intended to be) equivalent to running the following Snakefile:

include: "pipeline.conf"  # Should be an empty file

shell.prefix("set -euo pipefail;")

rule all:
    input:

CristinaSolana

1. Clone your fork:

git clone git@github.com:YOUR-USERNAME/YOUR-FORKED-REPO.git

2. Add remote from original repository in your forked repository:

cd into/cloned/fork-repo
git remote add upstream git://github.com/ORIGINAL-DEV-USERNAME/REPO-YOU-FORKED-FROM.git
git fetch upstream