Skip to content

Instantly share code, notes, and snippets.

@jakevdp
Created Jun 13, 2011
Embed
What would you like to do?
ARPACK memory error
import numpy as np
from scipy.sparse.linalg import eigs
N = 6
k = 2
# with this random seed, I get a memory error on the third iteration below
np.random.seed(2301)
A = np.random.random((N,N))
for i in range(3):
print i
evals2, evecs2 = eigs(A, k)
@jakevdp

This comment has been minimized.

Copy link
Owner Author

@jakevdp jakevdp commented Jun 13, 2011

valgrind gives me this:

valgrind --tool=memcheck --suppressions=valgrind-python.supp python AR_crash.py

==6569== Memcheck, a memory error detector
==6569== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==6569== Using Valgrind-3.5.0-Debian and LibVEX; rerun with -h for copyright info
==6569== Command: python AR_crash.py
==6569==
0
1
2
==6569== Invalid write of size 1
==6569== at 0x40260BF: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea85f is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 1
==6569== at 0x40260C7: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea85e is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 1
==6569== at 0x40260D0: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea85d is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 1
==6569== at 0x40260D9: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea85c is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA687: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea830 is 0 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA6A1: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea830 is 0 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA6F0: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea838 is 8 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA6F7: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea838 is 8 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA6FB: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea840 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA702: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea840 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA706: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea848 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA70D: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea848 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA711: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea850 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA718: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea850 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x54BA71C: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea858 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid write of size 8
==6569== at 0x54BA720: ATL_dscal_xp0yp0aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea858 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 1
==6569== at 0x4026180: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea830 is 0 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 1
==6569== at 0x4026187: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea831 is 1 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 1
==6569== at 0x4026190: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea832 is 2 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 1
==6569== at 0x4026199: memcpy (mc_replace_strmem.c:482)
==6569== by 0x5477B6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea833 is 3 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x54589B7: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea830 is 0 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x54589C6: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea838 is 8 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x5458EF2: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea840 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x5458F18: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea848 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x5458F24: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea850 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x5458F34: ATL_dJIK0x0x0TN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea858 is not stack'd, malloc'd or (recently) free'd
==6569==
==6569== Invalid read of size 8
==6569== at 0x54494A1: ATL_dJIK0x0x0NN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea830 is 0 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569== Invalid read of size 8
==6569== at 0x54494A8: ATL_dJIK0x0x0NN0x0x0_aX_bX (in /usr/lib/atlas/libblas.so.3gf.0)
==6569== Address 0x5aea838 is 8 bytes after a block of size 240 alloc'd
==6569== at 0x4024C1C: malloc (vg_replace_malloc.c:195)
==6569== by 0x4695ADC: PyArray_NewFromDescr (ctors.c:1058)
==6569== by 0x68B69B4: array_from_pyobj (fortranobject.c:719)
==6569== by 0x68AEB13: f2py_rout__arpack_dnaupd (_arpackmodule.c:2197)
==6569== by 0x68B51E0: fortran_call (fortranobject.c:346)
==6569== by 0x8061209: PyObject_Call (in /usr/bin/python2.6)
==6569== by 0x80DBC3B: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DD383: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DC1B3: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==6569== by 0x80DDDF1: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==6569== by 0x80DDEF6: PyEval_EvalCode (in /usr/bin/python2.6)
==6569==
==6569==
==6569== HEAP SUMMARY:
==6569== in use at exit: 5,053,421 bytes in 1,940 blocks
==6569== total heap usage: 83,899 allocs, 81,959 frees, 35,487,957 bytes allocated
==6569==
==6569== LEAK SUMMARY:
==6569== definitely lost: 0 bytes in 0 blocks
==6569== indirectly lost: 0 bytes in 0 blocks
==6569== possibly lost: 206,770 bytes in 268 blocks
==6569== still reachable: 4,846,635 bytes in 1,671 blocks
==6569== suppressed: 16 bytes in 1 blocks
==6569== Rerun with --leak-check=full to see details of leaked memory
==6569==
==6569== For counts of detected and suppressed errors, rerun with: -v
==6569== ERROR SUMMARY: 144 errors from 28 contexts (suppressed: 3328 from 160)

@jakevdp

This comment has been minimized.

Copy link
Owner Author

@jakevdp jakevdp commented Jun 13, 2011

Given that many of the memory errors are on or after an allocated block of size 240 bytes = 30 doubles, I think this has to do with the array "v" passed to dnaupd. In this case, the shape of v is (n,ncv) where ncv=2*k+1, with n=6 and k=2 in our case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment