In one terminal, checkout seaweedfs, build and run.
$ git clone git@github.com:chrislusf/seaweedfs.git
$ cd seaweedfs
$ make
$ weed/weed server -s3 -dir /tmp/weed-benchmark-dir -volume.port 8081
Checkout benchmark snippet.
$ git clone git@gist.github.com:6f3fee974ba82083325c2f24c912b47b.git
$ cd 6f3fee974ba82083325c2f24c912b47b
$ mkvirtualenv 6f3fee974ba82083325c2f24c912b47b
$ pip install -r requirements.txt
$ python s3test.py 2> s3test.log
Tail log file in yet another terminal (worker name, completed per worker, elapsed seconds, docs/second).
$ tail -f s3test.log
worker-0 @0 0.00 0.00 │···························
worker-1 @0 0.00 0.00 │···························
worker-2 @0 0.00 0.00 │···························
worker-3 @0 0.00 0.00 │···························
worker-1 @100 0.99 100.94 │···························
worker-0 @100 1.02 98.25 │···························
worker-3 @100 1.02 98.35 │···························
worker-2 @100 1.05 94.97 │···························
worker-3 @200 1.97 101.57 │···························
worker-1 @200 2.00 99.90 │···························
worker-0 @200 2.02 98.85 │···························
worker-2 @200 2.06 96.98
...
- 100kb docs inserts, about 400 docs/s (with 4 workers)
- 5kb docs inserts, about 540 docs/s (with 4 workers)
botocore.exceptions.ClientError: An error occurred (InternalError) when calling the PutObject operation (reached max retries: 4): We encountered an internal error, please try again.
"fail to allocate volume for /buckets/test/k167733232, collection:test, datacenter:"
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 501022 410 500612 1% /dev
tmpfs 504885 1050 503835 1% /run
/dev/vda1 365613056 59483 365553573 1% /
tmpfs 504885 1 504884 1% /dev/shm
tmpfs 504885 3 504882 1% /run/lock
tmpfs 504885 17 504868 1% /sys/fs/cgroup
nfs-home:/home 478617600 50400103 428217497 11% /home
tmpfs 504885 4 504881 1% /run/user/3211