Example how to read/write large files in incremental way using cPickle. Run `gen.py` to generate a file containing list of integers then use `read.py` to read this list incrementally from the file.

gen.py

import cPickle as pickle
import io

lst = range(16)
with io.open('list.p', 'wb') as f :
    pickler = pickle.Pickler(f)
    for l in lst :
        pickler.dump(l)

read.py

import cPickle as pickle
import io

with io.open('list.p', 'rb') as p :
    unpickler = pickle.Unpickler(p)
    while p.peek(1) :
        print unpickler.load()