manugarri

How to get a stack trace for each thread in a running Python script

Sometimes a Python script will simply hang forever with no indication of what is going wrong. Perhaps it's polling a service that will never return a value that allows the program to move forward.

Here's a way to see where the program is currently stuck, using pyrasite a tool for injecting code into running Python processes.

Install gdb and pyrasite

Install gdb.

manugarri

manugarri

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

manugarri

from fastai.text import *
from fastai.structured import proc_df
import pandas as pd
import numpy as np

class MixedInputModelWithText(nn.Module):
    def __init__(self, emb_szs, n_cont, emb_drop, out_sz, szs, drops,
                 y_range=None, use_bn=False, is_reg=True, is_multi=False, n_text=0):
        super().__init__()
        for i, (c, s) in enumerate(emb_szs): assert c > 1, f"cardinality must be >=2, got emb_szs[{i}]: ({c},{s})"

manugarri

func TestSetCookie(t *testing.T) {
  // Create a new HTTP Recorder (implements http.ResponseWriter)
  recorder := httptest.NewRecorder()
  
  // Drop a cookie on the recorder.
  http.SetCookie(recorder, &http.Cookie{Name: "test", Value: "expected"})
  
  // Copy the Cookie over to a new Request
  request := &http.Request{Header: http.Header{"Cookie": recorder.HeaderMap["Set-Cookie"]}}
  

manugarri

Host *.gerardorf.org !bastion.gerardorf.org
  User admin
  IdentityFile ~/.ssh/gerardorf-private-key
  ProxyCommand ssh -W %h:%p admin@bastion.gerardorf.org

Host bastion.gerardorf.org
  Hostname XXX.XXX.XXX.XXX
  User admin
  IdentityFile ~/.ssh/gerardorf-private-key
  ForwardAgent yes

manugarri

{
"IAB1": "Arts & Entertainment",
"IAB1-1": "Books & Literature",
"IAB1-2": "Celebrity Fan/Gossip",
"IAB1-3": "Fine Art",
"IAB1-4": "Humor",
"IAB1-5": "Movies",
"IAB1-6": "Music",
"IAB1-7": "Television",
"IAB2": "Automotive",

manugarri

from sklearn.cross_validation import train_test_split
from keras.preprocessing import sequence, text
from keras.models import Sequential
from keras.layers import (Dense, Dropout, Activation, Embedding, LSTM,
                            Convolution1D, MaxPooling1D)

# Embedding
max_features = 20000
maxlen = 100
embedding_size = 32

manugarri

#!/usr/bin/python2


# Copyright (C) 2016 Sixten Bergman
# License WTFPL
#
# This program is free software. It comes without any warranty, to the extent
# permitted by applicable law. 
# You can redistribute it and/or modify it under the terms of the Do What The
# Fuck You Want To Public License, Version 2, as published by Sam Hocevar. See

manugarri

Automation for the People

Long ago, the first time I read "The Pragmatic Programmer", I read some advice that really stuck with me.

"Don't Use Manual Procedures".

This in the chapter on Ubiquitous Automation. To summarize, they want you to automate all the things.

The trouble was that I hadn't much of an idea how to actually go