Skip to content

Instantly share code, notes, and snippets.

@0leynik
Forked from bearpaw/LMDB_val_patches.txt
Created October 26, 2017 12:47
Show Gist options
  • Save 0leynik/a0cf6118f355c1a8170dd80bd351e885 to your computer and use it in GitHub Desktop.
Save 0leynik/a0cf6118f355c1a8170dd80bd351e885 to your computer and use it in GitHub Desktop.
Read LMDB in python
CAFFE_ROOT=/home/wyang/Data/Code/caffe-wk
$CAFFE_ROOT/build/tools/convert_imageset -backend lmdb /home/wyang/code/pose/chen-nips14-pose/ LMDB_val_patches.txt tmp_LMDB
import caffe
import lmdb
import numpy as np
import matplotlib.pyplot as plt
from caffe.proto import caffe_pb2
# Wei Yang 2015-08-19
# Source
# Read LevelDB/LMDB
# ==================
# http://research.beenfrog.com/code/2015/03/28/read-leveldb-lmdb-for-caffe-with-python.html
# Plot image
# ==================
# http://www.pyimagesearch.com/2014/11/03/display-matplotlib-rgb-image/
# Creating LMDB in python
# ==================
# http://deepdish.io/2015/04/28/creating-lmdb-in-python/
lmdb_file = "/home/wyang/code/pose/chen-nips14-pose/cache/lsp/LMDB_val"
lmdb_env = lmdb.open(lmdb_file)
lmdb_txn = lmdb_env.begin()
lmdb_cursor = lmdb_txn.cursor()
datum = caffe_pb2.Datum()
for key, value in lmdb_cursor:
datum.ParseFromString(value)
label = datum.label
data = caffe.io.datum_to_array(datum)
im = data.astype(np.uint8)
im = np.transpose(im, (2, 1, 0)) # original (dim, col, row)
print "label ", label
plt.imshow(im)
plt.show()
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000001.jpg 1
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000002.jpg 1823
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000003.jpg 2903
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000004.jpg 4470
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000005.jpg 4698
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000006.jpg 4797
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000007.jpg 4917
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000008.jpg 5072
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000009.jpg 5177
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000010.jpg 5271
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000011.jpg 6617
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000012.jpg 7592
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000013.jpg 7754
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000014.jpg 7892
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000015.jpg 7984
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000016.jpg 8109
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000017.jpg 8190
./cache/flic_win/LMDB/LMDB_val_patches/casino-royale-00039871_00000001_000018.jpg 8341
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000001.jpg 1
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000002.jpg 900
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000003.jpg 2585
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000004.jpg 4529
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000005.jpg 4713
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000006.jpg 4819
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000007.jpg 4921
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000008.jpg 5095
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000009.jpg 5160
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000010.jpg 5278
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000011.jpg 5916
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000012.jpg 7624
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000013.jpg 7654
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000014.jpg 7833
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000015.jpg 7985
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000016.jpg 8006
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000017.jpg 8212
./cache/flic_win/LMDB/LMDB_val_patches/million-dollar-baby-disc-00106821_00000002_000018.jpg 8346
@avatar-lavventura
Copy link

avatar-lavventura commented Jan 1, 2021

Thanks seems like I was able to get some key and values but returned value is single character:

for key, value in lmdb_cursor:
    print(key)
    print(value)

# output:
b'Header'
b'\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x06\x00\x00\x00\x00\x00\x00\x00'


Do you think using with caffe might get additional data (because original file size is much larger)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment