Delete keys map reduce

gistfile1.md

#Map Reduce Delete Instructions

Set allow_strfun true on all nodes:

echo "rpc:multicall([node() | nodes()], application,set_env,[riak_kv,allow_strfun,true])." | bin/riak attach

Insert 15000 keys:

for i in {1..15000}; do curl -XPUT http://127.0.0.1:9000/buckets/test/keys/test$i -H "content-type: text/plain" -d "Data #$i\n"; done

Count the keys:

for i in {1..15000}; do curl -s -o - -XGET http://127.0.0.1:9000/buckets/test/keys/test$i; echo ""; done | sed -n -e '/not found/D;/^..*$/p' | wc -l

Response: 15000

Iterate the bucket with a delete map phase:

curl -XPOST http://127.0.0.1:9000/mapred \
-H 'Content-Type: application/json' \
-d '{"inputs":"test",
	"query":[{"map":{"language":"erlang",
	"source":"fun(Object, _KD, _A) -> 
	Key = riak_object:key(Object),
	Bucket = riak_object:bucket(Object), 
	{ok, Client} = riak:local_client(), 
	Client:delete(Bucket,Key), 
	[[Bucket,Key]] 
	end."}}]}'

Response: Truncated for space, 15000 pairs are returned...["test","test5322"],["test","test1331"],["test","test1941"],["test","test110"],["test","test8371"],["test","test3231"],["test","test11717"],["test","test11643"],["test","test13546"]]

Repeat with a get reduce phase to clear out riak_kv level tombstones:

curl -XPOST http://127.0.0.1:9000/mapred \
-H 'Content-Type: application/json' \
-d '{"inputs":"test",
	"query":[{"map":{"language":"erlang",
	"source":"fun(Object, _KD, _A) -> 
		Key = riak_object:key(Object),
		Bucket = riak_object:bucket(Object), 
		{ok, Client} = riak:local_client(), 
		Client:delete(Bucket,Key), 
		[[Bucket,Key]] 
	end."}},
{"reduce":{"language":"erlang", 
	"source":"fun(Objects, _A) -> 
		lists:map(fun([B,K]) -> 
			{ok,Client} = riak:local_client(), 
			Client:get(B,K), 
			[B,K] 
		end, 
			Objects) 
	end."}}]}'

Response: [] A few errors may return if delete was inconsistent. Reduce phase will reconcile those inconsistencies.

See how many keys are left (Expect 0):

curl http://127.0.0.1:9000/buckets/test/keys/?keys=true

Response: {"keys":[]}