denis-bz

    """ approximate derivatives for non-uniformly spaced data
        deriv1, deriv2( x, t ) with t0 < t1 < ...
        Very sensitive to noise, or delta-t near 0
        Spline, then take derivatives, is much better; see scipy.interpolate.splev( der= )
    """
    # http://google.com/search?q="finite-difference non-uniform site:stackexchange.com"

    from __future__ import division
    import numpy as np

denis-bz

Comparing time series in Python

The task: you have a hundred pairs of time series, say A1 .. A100 and B1 .. B100, e.g. temperature data at 100 cities in two different years. All the time series are say 365 long, but many days are missing (NA or NaN). Some pairs are quite similar, some not. You want to rank them from most similar .. least similar, 0 .. 100, in order to describe and understand how A and B differ.

(The $64 question: does it matter ?

denis-bz

scipy.linalg lsmr with sparse right-hand side

See scipy-sparse-iterative-solvers-no-sparse-right-hand-side-support

linop.py subclasses scipy.sparse.linalg.LinearOperator , then tests lsmr( Linop( A ), b ... ) with random sparse A and b .
Usage, in a shell or IPython:

denis-bz

#!/usr/bin/env python
''' In: a firefox cookies.sqlite file
    	default $HOME/Library/Application\ Support/Firefox/Profiles/*/cookies.sqlite
    Out: urls in the file, one per line on stdout

modified https://linuxfreelancer.com/decoding-firefox-cookies-sqlite-cookies-viewer
'''
	# google python sqlite3 "delete cookies" ? looks mttiw

denis-bz

A 3-term anti-zigzag filter

I want a 3-term FIR filter that meets these 3 objectives:

exact for constants:   1 1 1  -> 1
pretty good for lines: -1 0 1 -> 1.1 say
anti-zigzag:           1 -1 1 -> 0

There are several ways to do this: run a program (that one doesn't understand completely),

denis-bz

# from https://github.com/lyst/lightfm/blob/master/setup.py  2016-02-17 feb 
# compile_args link_args that worked for me on:
# macosx 10.8.3, gcc-4.9, python 2.7.6, no Anaconda; comments welcome
# denis-bz-py t-online.de

import glob
import os
import platform
import subprocess
import sys

denis-bz

1000 beers with PCA

This plot shows 1000 different beers mapped to 2d with PCA:

No surprise: most beers in this database are rated pretty high (red and yellow),

denis-bz

Purpose: generate "print ..." to help follow code.

print_gen.py expands lines like

p var = expr

-->

print( '''>> var = expr''' )

var = expr

denis-bz

#!/usr/bin/env python
""" Gavish + Donoho, Optimal Hard Threshold for Singular Values is 4 / sqrt 3, 2014, 14p
  A = D (Signal, diagonal) + Noise
  Atrunc = truncated SVD( A )
  How well does Atrunc == D + Res approximate D ?
  |Atrunc|, |Res| increase with ntrunc
"""
# what am I missing:
# if one knows that Signal is diagonal, just threshold A ?
# See also

denis-bz

Signal 98 % + Noise 20 % = 100 % ?

Data is often split into "signal" + "noise", with

|data|^2 = |signal|^2 + |noise|^2
         = sum of squares, data1^2 + data2^2 + ...

Sums of squares can be rather non-intuitive. For example,