JeyKJey

import numpy as np
import multiprocessing as multi

def chunks(n, page_list):
    """Splits the list into n chunks"""
    return np.array_split(page_list,n)
 
cpus = multi.cpu_count()
workers = []
page_list = ['www.website.com/page1.html', 'www.website.com/page2.html'

JeyKJey

# library to generate user agent
from user_agent import generate_user_agent
# generate a user agent
headers = {'User-Agent': generate_user_agent(device_type="desktop", os=('mac', 'linux'))}
#headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux i686 on x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.63 Safari/537.36'}
page_response = requests.get(page_link, timeout=5, headers=headers)

asdfgeoff

from typing import Callable, Union
import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.utils.validation import check_X_y, check_array
from IPython.display import display

array_like = Union[list, np.ndarray]
matrix_like = Union[np.ndarray, pd.DataFrame]

mdwhatcott

#!/usr/bin/env python

"""
This script scans the current working directory for changes to .go files and 
runs `go test` in each folder where *_test.go files are found. It does this 
indefinitely or until a KeyboardInterrupt is raised (<Ctrl+c>). This script 
passes the verbosity command line argument (-v) to `go test`.
"""

apolloclark

#!/bin/bash

# update apt-get
export DEBIAN_FRONTEND="noninteractive"
sudo apt-get update

# remove previously installed Docker
sudo apt-get purge lxc-docker*
sudo apt-get purge docker.io*

tylerneylon

""" A function that can read MNIST's idx file format into numpy arrays.

    The MNIST data files can be downloaded from here:
    
    http://yann.lecun.com/exdb/mnist/

    This relies on the fact that the MNIST dataset consistently uses
    unsigned char types with their data segments.
"""

johnolafenwa

#Import needed packages
import torch
import torch.nn as nn
from torchvision.datasets import CIFAR10
from torchvision.transforms import transforms
from torch.utils.data import DataLoader
from torch.optim import Adam
from torch.autograd import Variable
import numpy as np

JeffPaine

Moved

Now located at https://github.com/JeffPaine/beautiful_idiomatic_python.

Why it was moved

Github gists don't support Pull Requests or any notifications, which made it impossible for me to maintain this (surprisingly popular) gist with fixes, respond to comments and so on. In the interest of maintaining the quality of this resource for others, I've moved it to a proper repo. Cheers!

chdorner

my database had 72k annotations at the time I ran these benchmarks, here's the result:

$ python scripts/batch_bench.py conf/development-app.ini dumb
Memory summary: start
      types |   # objects |   total size
=========== | =========== | ============
       dict |       13852 |     12.46 MB
  frozenset |         349 |     11.85 MB
VM: 327.29Mb

robphoenix

This is unmaintained, please visit Ben-PH/spacemacs-cheatsheet

Useful Spacemacs commands

• SPC q q - quit
• SPC w / - split window vertically
• SPC w - - split window horizontally
• SPC 1 - switch to window 1
• SPC 2 - switch to window 2
• SPC w c - delete current window