ktmud

Easy parallel python with concurrent.futures

As of version 3.3, python includes the very promising concurrent.futures module, with elegant context managers for running tasks concurrently. Thanks to the simple and consistent interface you can use both threads and processes with minimal effort.

For most CPU bound tasks - anything that is heavy number crunching - you want your program to use all the CPUs in your PC. The simplest way to get a CPU bound task to run in parallel is to use the ProcessPoolExecutor, which will create enough sub-processes to keep all your CPUs busy.

We use the context manager thusly:

with concurrent.futures.ProcessPoolExecutor() as executor:

ktmud

license: mit

ktmud

Tensorflow 1.7 with CUDA on macOS Sierra 10.12.6, Anaconda, and Python 3.6

Largely based on the Tensorflow 1.6 gist, this should hopefully simplify things a bit. Mixing homebrew python2/python3 with pip ends up being a mess, so here's an approach to uses the built-in python27.

Requirements

• NVIDIA Web-Drivers 378.05.05 for 10.12.6
• CUDA 9.0 Toolkit
• cuDNN 7.0.5 (latest release for mac os)
• Python 3.6

ktmud

def sizeof_fmt(num, suffix='o'):
    """Readable file size

    :param num: Bytes value
    :type num: int
    :param suffix: Unit suffix (optionnal) default = o
    :type suffix: str
    :rtype: str
    """
    for unit in ['', 'k', 'M', 'G', 'T', 'P', 'E', 'Z']:

ktmud

#!/bin/sh

# Converts a mysqldump file into a Sqlite 3 compatible file. It also extracts the MySQL `KEY xxxxx` from the
# CREATE block and create them in separate commands _after_ all the INSERTs.

# Awk is choosen because it's fast and portable. You can use gawk, original awk or even the lightning fast mawk.
# The mysqldump file is traversed only once.

# Usage: $ cat "./mydump.sql" | ./mysql2sqlite | sqlite3 database.sqlite
# Example: $ mysqldump -u root -pMySecretPassWord myDbase | ./mysql2sqlite | sqlite3 database.sqlite

ktmud

R to python useful data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. 
The beauty of dplyr is that, by design, the options available are limited. 
Specifically, a set of key verbs form the core of the package. 
Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. 
Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. 
The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package). 

dplyr is organised around six key verbs

ktmud

jQuery Release Notes

2.X

• 2.1.1 (2014/05/01)
• 2.1.0 (2014/01/24)
• 2.0.3 (2013/07/03)
• 2.0.2 (2013/05/30)
• 2.0.1 (2013/05/24)
• 2.0.0 (2013/04/18)

ktmud

# -*- encoding:utf-8
"""
(c) https://gist.github.com/577116

HTML5 microdata parser for python 2.x/3.x

- it requires lxml
- microdata specification: http://dev.w3.org/html5/md/
"""
try: from urllib.parse import urljoin