Persistent Object Data

.gitignore

__pycache__
myfile.json

README.md

Persistent Object Data

Simple way to keep data through multiple executions of a script.

Designed for multiple executions on a single system. If using on a cluster, consider using something like Redis to hold the data instead of a RAM disk

RAM disk

Check how much RAM you've got:

$ free -gm
              total        used        free      shared  buff/cache   available
Mem:           7833        4871         159        1112        2802        1418
Swap:          8047         506        7541

Create mountpoint

$ mkdir /mnt/ramdisk

Mount and test

# mount and write
$ mount -t tmpfs -o size=2m tmpfs /mnt/ramdisk
$ echo "testing 123" >> /mnt/ramdisk/test.txt
$ cat /mnt/ramdisk/test.txt
testing 123

# unmount
$ umount /mnt/ramdisk

# re-mount and confirm file is gone
$ mount -t tmpfs -o size=2m tmpfs /mnt/ramdisk
$ cat /mnt/ramdisk/test.txt
cat: /mnt/ramdisk/test.txt: No such file or directory

this is correct behaviour, because when we unmount the ramdisk, all the data will vaporise.

To make a permenant fixture, add to /etc/fstab

tmpfs   /mnt/ramdisk   tmpfs   nodev,nosuid,noexec,nodiratime,size=2M   0   0

Mount again. Either re-boot, or run

$ mount /mnt/ramdisk

ref: https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

Persistent Object

For example, with a file called test.py

#!/usr/bin/env python
from persistent import PersistentContent
from random import random

data = PersistentContent('/mnt/ramdisk/myfile.json')

print('before: {!r}'.format(data.count))
data.count = random()
print('after : {!r}'.format(data.count))

Running the first time:

$ python test.py
before: None
after : 0.34227870478577316

Then the 2nd time:

$ python test.py
before: 0.34227870478577316
after : 0.15038667254108706

Note that the before number is the same as the after of the 1st run... hence the persistence of that data through multiple runs.

persistent.py

import json
import os

class PersistentContent(object):
    def __init__(self, filename):
        if os.path.isfile(filename):  # only load if it already exists
            with open(filename, 'r') as fh:
                data = json.load(fh)
                for (k, v) in data.items():
                    self.__dict__[k] = v
        self.__dict__['_file'] = filename  # direct assignment to bypass save

    def _save(self):
        with open(self._file, 'w') as fh:
            json.dump(self.__dict__, fh)

    def __setattr__(self, key, value):
        self.__dict__[key] = value
        self._save()

    def __getattr__(self, key):
        return self.__dict__.get(key, None)

test.py

#!/usr/bin/env python
from persistent import PersistentContent
from random import random

data = PersistentContent('myfile.json')  # change to ramdisk to avoid excessive writes to disk
#data = PersistentContent('/mnt/ramdisk/myfile.json')

print('before: {!r}'.format(data.count))
data.count = random()
print('after : {!r}'.format(data.count))