Skip to content

Instantly share code, notes, and snippets.

@hay
Last active January 6, 2021 15:19
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save hay/a09e180c6064906b5271 to your computer and use it in GitHub Desktop.
Hay's dev cheatsheet

Hay's dev cheatsheet

So, how do i...

Index

Web development

Fixing Internet Explorer 11 bugs

Add a class if the browser is Internet Explorer 11 or lower

if ('ActiveXObject' in window) {
    document.documentElement.className = 'is-ie';
}

Fixing childen of a flexbox container exceeding their parent width

Just add width: 100% to the elements, or if you're really lazy:

.container > * {
    width: 100%;
}

SCSS

Getting both value and index from a SASS list

$colors: red, blue, yellow, green;
@for $index from 1 through length($colors) {
    li:nth-child(#{length($colors)}n + #{$index}):before {
        background-image: url('../img/bullet-#{nth($colors, $index)}.svg');
    }
}

Javascript

Is there a quicker way to assign a default value if the result is undefined?

Instead of this:

const bar = foo.bar ? foo.bar : 'bar';

Or this:

let bar;

if (foo.bar) {
  bar = foo.bar;
} else {
  bar = 'bar';
}

Try this

const bar = foo.bar || 'bar';

Promises

// Transforming jQuery's $.get to a Promise
function get(url) {
    return new Promise(function(resolve, reject) {
        $.ajax({
            method : "GET",
            url : url,
            success : function(data) {
                resolve(data);
            },
            error : function() {
                reject('Something went wrong');
            }
        }
    });
}

get('http://example.com').then(function(data) {
    console.log(data);
});

async/await

const URL = 'https://api.github.com/gists/a09e180c6064906b5271';

async function getGist() {
    try {
        const res = await fetch(URL);
        const data = await res.json();
        return data;
    } catch (err) {
        console.log(err);
    }
}

getGist().then(gist => console.log(gist.description));

Lodash

Transform an array with objects to an object with keys and values

_.mapValues(_.keyBy(data, 'key'), 'value');

Transform this:

[
    { key : 'foo', value : 'bar'},
    { key : 'baz', value : 'qux'}
]

Into this:

{
    'foo' : 'bar',
    'baz' : 'qux'
}

PHP

Get the path of the current script

echo __FILE__;

Get the directory of the current script

echo realpath(dirname(__FILE__));

Fix those pesky PDO SQLSTATE[HY000] [2002] No such file or directory errors

  • Try using 127.0.0.1 instead of localhost for DB host
  • Add a utf-8 charset

Do a 301 permanent redirect

header("Location: https://www.example.com", true, 301);

Git

Check in an empty directory

Create a .gitignore with this content:

# Ignore everything in this directory
*
# Except this file
!.gitignore

Add something to your last commit

Use git add just as you would for a normal commit. Then:

git commit --ammend

Edit last commit message

git commit --amend -m "New commit message"

Resolve all merge conflicts by overwriting with all local or remote files (source)

To use local files

grep -lr '<<<<<<<' . | xargs git checkout --ours

To use remote files

grep -lr '<<<<<<<' . | xargs git checkout --theirs

Remove local (untracked) files from a current branch

git clean -f

Directories as well

git clean -fd

.gitignore files as well

git clean -fdx

Show a single file from a specific revision

git show <revision>:<filename>

Remove an already pushed tag in a Github / remote repo

git tag -d yourtag
git push origin :refs/tags/yourtag

Undo the last commit

git reset --hard HEAD^

If you don't care about the changes, or

git reset --soft HEAD^

If you do.

If you've already pushed, try this:

git revert HEAD

Show all files that will get added when doing a git add .

git status -u

Python

Dicts and lists

Iterate over keys and values in a dict

obj = { "key" : val }

# Python 2
for key, val in obj.iteritems():
    print key, val

# Python 3
for key, val in obj.items():
    print(key, val)

Extend / merge a dict with more values

a = { "foo" : 1 }
a.update({ "bar" : 2 })

print(a) # { "foo" : 1, "bar" : 2 }

Get the first item in a dict

a = { "a" : {}, "b" : {} }
a.itervalues().next()

Get the first N items in a dict

{k:v for k,v in list(d.items())[0:N]}

Remove a key from a dict

my_dict.pop("key", None)

Get a default value for a dict if the key does not exist

a = { "foo" : "foo" }
a["foo"] // "foo"
a.get("bar", "bar") // "bar"
a["bar"] // KeyError: "bar"

Get the index number when looping over a list (array)

l = [1,2,3]
for index, val in enumerate(l):
    print(index, val) # "0, 1", "1, 2", "2, 3"

Sort a list by a user defined function

a = [ { "foo" : "bar" }, {"foo" : "baz"} ]

a.sort(key = lambda i:i["foo"])

Or

a = sorted(a, key = lambda i:i["foo"])

Sort a list with dicts by a key in the dicts?

a = [ { "name" : "Bert" }, { "name" : "Ernie" }

from operator import itemgetter
b = sorted(a, key = itemgetter("name") )

Sort a list with lists on one of the items?

a = [ ["foo", 3], ["foo", 1], ["foo", 2] ]
a.sort(key = lambda i:i[1]) // [ ["foo", 1], ["foo", 2], ["foo", 3] ]

Sort a dict by its values?

x = { "a" : 3, "b" : 2, "c" : 9 }
dict(sorted(x.items(), key=lambda item: item[1]))

Get the key of a dict where the value is the largest

a = { "foo" : 10, "bar" : 20, "baz" : 5 }
max(a, key = (lambda k:a[k])) # 'bar'

Count the number of times an item appears in a list?

from collections import Counter

l = ["foo", "bar", "foo", "baz"]
print(Counter(l))

# Prints:
# {'foo': 2, 'baz': 1, 'bar': 1}

Flatten a list with nested lists?

The ugly way

sum(l, [])

The slighty more Pythonic (and faster) way

[item for sublist in l for item in sublist]

And the fastest way

from itertools import chain
list(chain.from_iterable(l))

Get the unique values in a list

Convert to a set, then to a list again

list(set(l))

Remove falsy values from a list

Python 2

values = filter(None, [1, None, 3])

Python 3

values = list(filter(None, [1, None, 3]))

Split a list in evenly sized chunks

lst = list(range(0, 180))

chunks = [ lst[i:i + 50] for i in range(0, len(lst), 50) ]

Or if you want a function:

def batch(iterable, n = 1):
    for i in range(0, len(iterable), n):
        yield iterable[i:i + n]    

Filter a list using comprehensions

l = [5, 3, 7, 24, 2, 22]

l = [ x for x in l if x > 10 ] # l == [24, 22]

Dict comprehensions

d = {k:v for k,v in dct.iteritems() } # Python 2
d = {k:v for k,v in dct.items() } # Python 3

Strings

Format a string

Either

"Ham, %s and %s" % ("eggs", "spam")

Or

"Ham, {food1} and {food2}".format(food1 = "eggs", food2 = "spam")

Or (since Python 3.6)

food1 = "eggs"
food2 = "spam"
f"Ham, {food1} and {food2}"

Pad a number with zeroes

If the number is already a string

number.zfill(3)

URL encode a string

Python 2 urllib.quote_plus("Lots of weird charachters:ª•¶•ª¶™•ª¢∆")

Python 3 urllib.parse.quote_plus("Lots of weird charachters:ª•¶•ª¶™•ª¢∆")

Check if any string in a list appears in another string

target = "a lovely string"
strings = ("nice", "kind", "lovely")

if any(s in target for s in strings):
    print(s) # 'lovely'

Regular expressions

import re

REGEX = re.compile("…[^…]*…")

# Replace all occurences
text = REGEX.sub(" ", text)

# Get all matches
matches = REGEX.findall(text)

# Split by regex
parts = REGEX.split(text)

Dates and times

Get the current time as a ISO8601-formatted string

import datetime
datetime.datetime.now().isoformat("T")

Files

Get the directory of the current executing file

import os
PATH = os.path.dirname(os.path.realpath(__file__))

In reverse

a.sort(key = lambda i:i[1], reverse = True) // [ ["foo", 3], ["foo", 2], ["foo", 1] ]

Check if a file exists?

import os.path
os.path.isfile( filename )

Get a filename without extension

import os
os.path.basename(filename)[0]

To also remove the path to the filename

os.path.splitext(os.path.basename(filename))[0]

Logging

import logging
logger = logging.getLogger(__name__)

logger.critical("Log with critical level")
logger.error("Log with error level")
logger.warning("Log with warning level")
logger.info("Log with info level")
logger.debug("Log with debug level")

logging.basicConfig(level=logging.DEBUG) # Set default level to DEBUG

Classes

class Parent:

    foo = "Unknown"

    def __init__(self, foo):
        self.foo = foo

    def bar(self):
        print(self.foo)

class Child(Parent):
    def __init__(self, foo):
        super().__init__(foo)

Use __getitem__ for easy list access

class Uppercaser:
    def __getitem__(self, key):
        return key.upper()

up = Uppercaser()
print(up["hello"]) # HELLO

Use properties (getters and setters)

class Namer:
    def __init__(self, name):
        self._name = name

    @property
    def name(self):
        return self._name.upper()

    @name.setter
    def name(self, value):
        self._name = value            

Errors and Exceptions

def test():
    raise Exception("Something happened")

try:
    test()
except Exception as e:
    print(e.message)
else:
    # Do something else

Note that you can re-raise an exception

try:
    test()
except Exception as e:
    raise e

Custom exceptions

    class MyException(Exception):
        pass

    try:
        raise MyException("oops")
    except MyException as e:
        print("oops")

XML

To parse XML use xmltodict, to make it behave like a dict.

import xmltodict

doc = xmltodict.parse("""
<record id="123">
    <key>foo</key>
</record>
""")

print(doc["record"]["@id"]) # '123'
print(doc["record"]["key"]) # foo

To write back XML you can use xmltodict's unparse

print(xmltodict.unparse(doc, pretty = True)) # <record id="123">....

Or use lxml.etree.tostring()

from lxml import etree
print(etree.tostring(doc))

argparse

import argparse

parser = argparse.ArgumentParser(description = "Pretty description here")
parser.add_argument('head', type = str, nargs='?')
parser.add_argument('face', type = str, nargs='?')
parser.add_argument('-o', '--output', type = str, default = "out.jpg")
parser.add_argument('-v', '--verbose', action="store_true")
parser.add_argument('-c', '--caching', choices=('default', 'redis'))
args = parser.parse_args()

if not args.head and args.face:
    ArgumentParser.print_help()

Other stuff

Simplify a complex loop?

Use a generator. Instead of

for point in points:
    for x in point:
        if x > 5:
            plot(x)

Do

def iterpoints():
    for point in points:
        for x in points:
            if x > 5:
                yield x

for x in iterpoints():
    plot(x)

Increment an integer? foo++ doesn't work. (source)

foo += 1

Or

foo = foo + 1

Break into debugger from the code itself

import pdb;pdb.set_trace()

Encode URLs?

from urllib import quote_plus as encode
encode('Something with spaces and ••ª¶ª•¶∆˙˚∆ charachters')

Transform a CSV file using dicts

import csv

# Get the fieldnames first
infile = open("in.csv", "r")
reader = csv.DictReader(infile)
fieldnames = reader.next().keys()
infile.close()

# Now open the in and outfile
infile = open("in.csv", "r")
outfile = open("out.csv", "w")
reader = csv.DictReader(infile)
writer = csv.DictWriter(outfile, fieldnames = fieldnames)

# Print the field names in the first row
writer.writerow(
    dict(
        (fn, fn) for fn in fieldnames
    )
)

for row in reader:
    # Do something with the values

    # And then write it
    writer.writerow(row)

infile.close()
outfile.close()

How do i remove the mysterious "ValueError: View function did not return a response" with WSGI?

Check if there's no print statements in your code.

Parsing HTML using BeautifulSoup

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'html5lib')
videos = soup.select(".video")

for v in videos:
    stats = v.select_one(".stats").get_text()

    return {
        "stats" : stats,
        "url" : v.get("href", None),
        "title" : v.select("h1")[0].get_text(),
    }

Download a file over HTTP

import urllib.request
urllib.request.urlretrieve('http://www.example.com/songs/mp3.mp3', 'mp3.mp3')

Execute a shell command

import subprocess
subprocess.check_call(cmd, shell = True)

To do a 'silent' call, without any logging

import os, subprocess
subprocess.check_call(cmd, shell = True,
    stdout = open(os.devnull, 'wb')
)

Virtualenv

Make sure virtualenv is installed

pip install virtualenv

This will create an venv in a directory called env

virtualenv env

Then to activate it

source env/bin/activate

Installing local libs (from the venv)

pip install /path/to/lib

Or use venv in python3

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

Multithreading

from concurrent.futures import ThreadPoolExecutor

def square(val):
    return val * val

values = [5, 4, 12, 40]

with ThreadPoolExecutor(max_workers = 10) as executor:
    executor.map(square, values)

Command line (for *nix-likes)

  • See this excellent resource.

Login with SSH to an unreachable server via another server

Host          internal.hostname.tld
User          user
HostName      internal.hostname.tld
ProxyCommand  ssh user@external.hostname.tld nc %h %p 2> /dev/null

Weird rsync errors

Errors like this could be anything:

rsync error: error in rsync protocol data stream

But make sure your SSH keys have proper permissions

chmod -R 700 ~/.ssh

Convert a batch of image files

Use mogrifyto convert a bunch of JPG files to fit within a 1500x1500 frame (take longest side), and set quality to 60%

mogrify -resize 1500x1500 -quality 60 *.jpg

Same thing, but crop the images as well so they fit in the 1500x1500 frame, cropped from the center of the image.

mogrify -resize 1500x1500^ -crop 1500x1500+0+0 -gravity center *.jpg

Convert pages from a multi-page PDF to separate JPG files (3000px longest side), trimming excess whitespace

convert -resize 3000x doc.pdf -trim image-%d.jpg

Convert a set of image files to a PDF

convert *.png doc.pdf

Yes, it really is that simple.

Sequentially rename a bunch of files

Use the rename command to sequentially number a bunch of JPG files to 01.jpg, 02.jpg, etc.

rename -N 01 's/.*/$N.jpg/' *

Append both stdout and stderr to a file in Bash

command >> logfile.log 2>&1

Video converting

Basic use of ffmpeg

ffmpeg -i source.extension dest.extension

Resize to 600px wide

ffmpeg -i in.mov -vf scale=600:-1 out.mp4

Crop a video with landscape aspect-ratio to a square from the center

ffmpeg -i in.mov -filter:v "crop=in_h:in_h:((in_w/2)-(in_h/2)):0" out.mp4

Framerate to 24fps

ffmpeg -i in.mov -r 24 out.mp4

Bitrate to 2mb/s

ffmpeg -i in.mov -b:v 2000k out.mp4

Grab a frame from 1 second into the video and convert to jpg (note that .png also works)

ffmpeg -i in.mov -ss 00:00:01 -vframes 1 out.jpg

Make sure it works in Quicktime

ffmpeg -i in.mov -pix_fmt yuv420p out.mp4

Convert from a movie to a series of JPG's (note that -q:v 2 is only needed for JPG)

ffmpeg -i clip.mp4 -q:v 2 "frames/%04d.jpg"

Grab a frame from every second of a clip and save to jpg

ffmpeg -i clip.mp4 -vf fps=1 frames/frame-%d.jpg

Same, but now every minute

ffmpeg -i clip.mp4 -vf fps=1/60 frames/frame-%d.jpg

Convert a bunch of images to a movie clip

ffmpeg -r 24.89 -f image2 -s 1280x720 -i "frames/%04d.jpg" -vcodec libx264 -crf 25 -pix_fmt yuv420p movie.mp4

Convert a MP3 to a MP4 with a single image

ffmpeg -loop 1 -y -i image.jpg -i music.mp3 -shortest -pix_fmt yuv420p output.mp4

Add MP3 audio to a movie and save as MP4 (given the audio is wav, i had less luck with mp3)

ffmpeg -i a.mp4 -i a.wav -c copy -map 0:v:0 -map 1:a:0 -shortest -c:a aac -b:a 192k b.mp4 

Fix out of sync audio/video (note itsoffset parameter)

ffmpeg -i original.avi -itsoffset 0.2 -i original.avi -map 0:0 -map 1:1 -acodec copy -vcodec copy synced.avi

Download movies from YouTube and other video services

Use the brilliant youtube-dl.

youtube-dl -f mp4 https://www.youtube.com/watch?v=bMj3Pc5I078

Directly convert a movie at an URL to mp3

youtube-dl -x --audio-format mp3 https://www.youtube.com/watch?v=bMj3Pc5I078

Batch run a command on all files in a directory

for f in *.ext; do command $f; done

Search for a string appearing in files in directory and all subdirectoris, ignoring case

grep -lir some_string .

The same, but only in files with a certain extension

grep -lir --include \*.py some_string .

Get total size of directories, reporting only on the current level

du -h -d1

Query DNS records

Query the A record for example.com

dig example.com A

Query the MX record(s) for example.com

dig example.com MX

Get the result as just the IP instead of the whole query/answer stuff

dig example.com A +short

Comand line (Linux)

Fix those stupid bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) warnings

$ vim /etc/locale.gen

Uncomment the en_US.UTF-8 line

$ locale-gen

If you're still getting errors, add these two lines to your bashrc or bash_profile on the machine you're connecting from:

# Setting for the new UTF-8 terminal support in Lion
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

Add user to group

usermod -aG group user

Command line (Mac OS X specific)

Pipe outputs to the clipboard

ls | pbcopy

Find stuff on the command line using spotlight

mdfind

Chrome

Remove URLS from autocomplete

Highlight the item using keyboard arrows and press Shift and Delete keys.

Spreadsheets

Use a formula on the output of a condition

=AVERAGE(FILTER(A2:A25, A2:A25>0))

Select a complete row or column in a formula

=AVERAGE(B:B)

Count the number of cells

=COUNTA(B:B)

Only count cells that meet a certain condition

=COUNTIF(B:B, "Yes")

Get the data from a cell with more than one condition (use shift-ctrl-enter to activate formula, note the ampersands)

=INDEX(C:C,MATCH(B2&"value",A:A&B:B,0))

(Excel only): convert seconds to a hh:mm:ss notation

Divide the number of seconds by 86400 (number of seconds in a day) --> format cell --> custom --> enter 'hh:mm:ss'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment