Skip to content

Instantly share code, notes, and snippets.

@methane

methane/gist:1894166

Created February 23, 2012 18:09
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save methane/1894166 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save methane/1894166 to your computer and use it in GitHub Desktop.
Download ZIP
Redis を試した時のメモ
Raw
gistfile1.py
inada-n@valkryie:~/work/redis$ mkvirtualenv redis # 実験用の独立した環境を容易
Using real prefix '/usr'
New python executable in redis/bin/python
Installing setuptools............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /home/inada-n/.virtualenvs/redis/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/inada-n/.virtualenvs/redis/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/inada-n/.virtualenvs/redis/bin/preactivate
virtualenvwrapper.user_scripts creating /home/inada-n/.virtualenvs/redis/bin/postactivate
virtualenvwrapper.user_scripts creating /home/inada-n/.virtualenvs/redis/bin/get_env_details
(redis)inada-n@valkryie:~/work/redis$ pip install redis # Redis クライアントライブラリをインストール
Downloading/unpacking redis
Downloading redis-2.4.11.tar.gz
Storing download in cache at /home/inada-n/.pip_cache/http%3A%2F%2Fcloud.github.com%2Fdownloads%2Fandymccurdy%2Fredis-py%2Fredis-2.4.11.tar.gz
Running setup.py egg_info for package redis
Installing collected packages: redis
Running setup.py install for redis
Successfully installed redis
Cleaning up...
(redis)inada-n@valkryie:~/work/redis$ pip install ipython # IPythonをインストール
Downloading/unpacking ipython
Using download cache from /home/inada-n/.pip_cache/http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fi%2Fipython%2Fipython-0.12.tar.gz
Running setup.py egg_info for package ipython
no previously-included directories found matching 'IPython/deathrow'
no previously-included directories found matching 'IPython/frontend/html/notebook/static/mathjax'
warning: no previously-included files found matching 'docs/#*'
warning: no previously-included files found matching 'docs/man/*.1'
no previously-included directories found matching 'docs/attic'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'docs/gh-pages'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.flc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.dircopy.log' found anywhere in distribution
Installing collected packages: ipython
Running setup.py install for ipython
no previously-included directories found matching 'IPython/deathrow'
no previously-included directories found matching 'IPython/frontend/html/notebook/static/mathjax'
warning: no previously-included files found matching 'docs/#*'
warning: no previously-included files found matching 'docs/man/*.1'
no previously-included directories found matching 'docs/attic'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'docs/gh-pages'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.flc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.dircopy.log' found anywhere in distribution
Installing ipcontroller script to /home/inada-n/.virtualenvs/redis/bin
Installing iptest script to /home/inada-n/.virtualenvs/redis/bin
Installing ipcluster script to /home/inada-n/.virtualenvs/redis/bin
Installing ipython script to /home/inada-n/.virtualenvs/redis/bin
Installing pycolor script to /home/inada-n/.virtualenvs/redis/bin
Installing iplogger script to /home/inada-n/.virtualenvs/redis/bin
Installing irunner script to /home/inada-n/.virtualenvs/redis/bin
Installing ipengine script to /home/inada-n/.virtualenvs/redis/bin
Installing ipython-qtconsole script to /home/inada-n/.virtualenvs/redis/bin
Successfully installed ipython
Cleaning up...
(redis)inada-n@valkryie:~/work/redis$ ipython # IPython を起動。以下は本当はカラーです。
Python 2.7.2+ (default, Oct 4 2011, 20:06:09)
Type "copyright", "credits" or "license" for more information.
IPython 0.12 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import redis
In [2]: client = redis.Redis()
In [3]: client. # タブキーを押してどんなメソッドがあるのか眺める
Display all 117 possibilities? (y or n)
client.RESPONSE_CALLBACKS client.hdel client.lrem client.scard client.transaction
client.append client.hexists client.lset client.sdiff client.ttl
client.bgrewriteaof client.hget client.ltrim client.sdiffstore client.type
client.bgsave client.hgetall client.mget client.set client.unwatch
client.blpop client.hincrby client.move client.set_response_callback client.watch
client.brpop client.hkeys client.mset client.setbit client.zadd
client.brpoplpush client.hlen client.msetnx client.setex client.zcard
client.config_get client.hmget client.object client.setnx client.zcount
client.config_set client.hmset client.parse_response client.setrange client.zincrby
client.connection_pool client.hset client.persist client.shutdown client.zinterstore
client.dbsize client.hsetnx client.ping client.sinter client.zrange
client.debug_object client.hvals client.pipeline client.sinterstore client.zrangebyscore
client.decr client.incr client.publish client.sismember client.zrank
client.delete client.info client.pubsub client.slaveof client.zrem
client.echo client.keys client.randomkey client.smembers client.zremrangebyrank
client.execute_command client.lastsave client.rename client.smove client.zremrangebyscore
client.exists client.lindex client.renamenx client.sort client.zrevrange
client.expire client.linsert client.response_callbacks client.spop client.zrevrangebyscore
client.expireat client.llen client.rpop client.srandmember client.zrevrank
client.flushall client.lock client.rpoplpush client.srem client.zscore
client.flushdb client.lpop client.rpush client.strlen client.zunionstore
client.get client.lpush client.rpushx client.substr
client.getbit client.lpushx client.sadd client.sunion
client.getset client.lrange client.save client.sunionstore
In [3]: client.keys() # 適当に実行してみる。まだ何も入れてないので空。
Out[3]: []
In [4]: import random # テストデータ作るために乱数使おうか。
In [5]: for i in xrange(1000000):
...: client.zadd('foo'+str(i), random.randint(0,10000) # 閉じカッコ一個足りんかった。。。
...:
...:
...: )
...:
---------------------------------------------------------------------------
RedisError Traceback (most recent call last)
/home/inada-n/work/redis/<ipython-input-5-50f1e21cdc1f> in <module>()
1 for i in xrange(1000000):
----> 2 client.zadd('foo'+str(i), random.randint(0,10000)
3 )
/home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.pyc in zadd(self, name, *args, **kwargs)
1177 if args:
1178 if len(args) % 2 != 0:
-> 1179 raise RedisError("ZADD requires an equal number of "
1180 "values and scores")
1181 pieces.extend(reversed(args))
RedisError: ZADD requires an equal number of values and scores # おもむろに実行したら使い方ミスってた。
In [6]: client.zadd? # ヘルプを読む
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zadd of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zadd(self, name, *args, **kwargs)
Docstring:
NOTE: The order of arguments differs from that of the official ZADD
command. For backwards compatability, this method accepts arguments
in the form of name1, score1, name2, score2, while the official Redis
documents expects score1, name1, score2, name2.
If you're looking to use the standard syntax, consider using the
StrictRedis class. See the API Reference section of the docs for more
information.
Set any number of element-name, score pairs to the key ``name``. Pairs
can be specified in two ways:
As *args, in the form of: name1, score1, name2, score2, ...
or as **kwargs, in the form of: name1=score1, name2=score2, ...
The following example would add four values to the 'my-key' key:
redis.zadd('my-key', 'name1', 1.1, 'name2', 2.2, name3=3.3, name4=4.4)
In [7]: test_data = {'foo'+str(i): random.randrange(10000) for i in xrange(1000000)}
In [8]: client.zadd('hogeranking', **test_data) # テストデータ一気に作ってぶち込む
ERROR: An unexpected error occurred while tokenizing input # 一気にぶち込みすぎたかな。。。
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (323, 0))
---------------------------------------------------------------------------
ConnectionError Traceback (most recent call last)
/home/inada-n/work/redis/<ipython-input-8-856237c9c705> in <module>()
----> 1 client.zadd('hogeranking', **test_data)
/home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.pyc in zadd(self, name, *args, **kwargs)
1183 pieces.append(pair[1])
1184 pieces.append(pair[0])
-> 1185 return self.execute_command('ZADD', name, *pieces)
1186
1187
/home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.pyc in execute_command(self, *args, **options)
280 except ConnectionError:
281 connection.disconnect()
--> 282 connection.send_command(*args)
283 return self.parse_response(connection, command_name, **options)
284 finally:
/home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/connection.pyc in send_command(self, *args)
256 def send_command(self, *args):
257 "Pack and send a command to the Redis server"
--> 258 self.send_packed_command(self.pack_command(*args))
259
260 def read_response(self):
/home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/connection.pyc in send_packed_command(self, command)
249 _errno, errmsg = e.args
250 raise ConnectionError("Error %s while writing to socket. %s." % \
--> 251 (_errno, errmsg))
252 except:
253 self.disconnect()
ConnectionError: Error 104 while writing to socket. Connection reset by peer.
In [9]: for i in xrange(1000000): # ループで1個づつやろう.
...: client.zadd('hogeranking', 'hoge'+str(i), random.randrange(10000)
...: )
...:
In [10]: client.zran
client.zrange client.zrangebyscore client.zrank
In [10]: client.zrank?
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zrank of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zrank(self, name, value)
Docstring:
Returns a 0-based value indicating the rank of ``value`` in sorted set
``name``
In [11]: client.zrank('hogeranking', 32)
In [12]: client.zrank('hogeranking', 'hoge32')
Out[12]: 179354
In [13]: client.zrank('hogeranking', 'hoge33')
Out[13]: 915375
In [14]: client.z
client.zadd client.zincrby client.zrangebyscore client.zremrangebyrank client.zrevrangebyscore client.zunionstore
client.zcard client.zinterstore client.zrank client.zremrangebyscore client.zrevrank
client.zcount client.zrange client.zrem client.zrevrange client.zscore
In [14]: client.zremrangebyrank? # ちょっと脇道。どんな関数だろう？
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zremrangebyrank of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zremrangebyrank(self, name, min, max)
Docstring:
Remove all elements in the sorted set ``name`` with ranks between
``min`` and ``max``. Values are 0-based, ordered from smallest score
to largest. Values can be negative indicating the highest scores.
Returns the number of elements removed
In [15]: client.zcard?
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zcard of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zcard(self, name)
Docstring: Return the number of elements in the sorted set ``name``
In [16]: client.zrange??
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zrange of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zrange(self, name, start, end, desc=False, withscores=False, score_cast_func=<type 'float'>)
Source:
def zrange(self, name, start, end, desc=False, withscores=False,
score_cast_func=float):
"""
Return a range of values from sorted set ``name`` between
``start`` and ``end`` sorted in ascending order.
``start`` and ``end`` can be negative, indicating the end of the range.
``desc`` a boolean indicating whether to sort the results descendingly
``withscores`` indicates to return the scores along with the values.
The return type is a list of (value, score) pairs
``score_cast_func`` a callable used to cast the score return value
"""
if desc:
return self.zrevrange(name, start, end, withscores)
pieces = ['ZRANGE', name, start, end]
if withscores:
pieces.append('withscores')
options = {'withscores': withscores, 'score_cast_func': score_cast_func}
return self.execute_command(*pieces, **options)
In [17]: client.zrange?
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zrange of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zrange(self, name, start, end, desc=False, withscores=False, score_cast_func=<type 'float'>)
Docstring:
Return a range of values from sorted set ``name`` between
``start`` and ``end`` sorted in ascending order.
``start`` and ``end`` can be negative, indicating the end of the range.
``desc`` a boolean indicating whether to sort the results descendingly
``withscores`` indicates to return the scores along with the values.
The return type is a list of (value, score) pairs
``score_cast_func`` a callable used to cast the score return value
# zrange を使ってみる
In [18]: client.zrange('hogeranking', 10, 20, withscores=True, score_cast_func=int)
Out[18]:
[('hoge263623', 0),
('hoge264840', 0),
('hoge273321', 0),
('hoge285067', 0),
('hoge287277', 0),
('hoge304667', 0),
('hoge306361', 0),
('hoge313299', 0),
('hoge332746', 0),
('hoge345214', 0),
('hoge367376', 0)]
In [19]: client.zrange('hogeranking', 1000, 1010, withscores=True, score_cast_func=int)
Out[19]:
[('hoge168585', 10),
('hoge173047', 10),
('hoge184281', 10),
('hoge199988', 10),
('hoge202798', 10),
('hoge203601', 10),
('hoge204865', 10),
('hoge222837', 10),
('hoge225557', 10),
('hoge239789', 10),
('hoge244020', 10)]
In [20]: def get_near_ranking(name): # 自分の付近のランキングを作ってみる.
....: rank = client.zrank('hogeranking', name)
....: min_ = max(rank-5, 0)
....: max_ = rank+5
....: ranking = client.zrange('hogeranking, min_, max_, withscores=True, score_cast_func=int)
....: rank = client.zcount('hogeranking', 0, ranking[0][1]-1)
....: prev_score = -1
....: for name, score in ranking:
....: if prev_score != score:
....: rank += 1
....: prev_score = score
....: yield rank, name, score
....:
File "<ipython-input-20-6c846048682e>", line 5
ranking = client.zrange('hogeranking, min_, max_, withscores=True, score_cast_func=int)
^
SyntaxError: EOL while scanning string literal # 一箇所文字列閉じるの忘れてる！
In [21]: def get_near_ranking(name): # ↑キーでヒストリを戻って再編集.
rank = client.zrank('hogeranking', name)
min_ = max(rank-5, 0)
max_ = rank+5
ranking = client.zrange('hogeranking', min_, max_, withscores=True, score_cast_func=int)
rank = client.zcount('hogeranking', 0, ranking[0][1]-1)
prev_score = -1
for name, score in ranking:
if prev_score != score:
rank += 1
prev_score = score
yield rank, name, score
....:
In [22]: get_near_ranking('hoge3333')
Out[22]: <generator object get_near_ranking at 0x60e5730> # ジェネレータ返したから見にくいな。
In [23]: list(_) # リスト化しよう. (最後に実行した文の結果は _ という変数に入ってる)
Out[23]:
[(36731L, 'hoge295941', 368),
(36731L, 'hoge30828', 368),
(36731L, 'hoge309143', 368),
(36731L, 'hoge310052', 368),
(36731L, 'hoge331654', 368),
(36731L, 'hoge3333', 368),
(36731L, 'hoge333991', 368),
(36731L, 'hoge335628', 368),
(36731L, 'hoge340924', 368),
(36731L, 'hoge348571', 368),
(36731L, 'hoge360139', 368)]
In [24]: %timeit get_near_ranking('hoge3333') # 速度は、と。。。
1000000 loops, best of 3: 572 ns per loop # 1000000 回実行して、3番目に良い奴が 572ns か。
In [25]: while 1: # 無限ループして、topでCPU使用率を確認。
....: get_near_ranking('hoge3333') # Pure Python だとやっぱり Python 側がほとんどCPU使ってた。
....:
^C---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
/home/inada-n/work/redis/<ipython-input-25-0ee564ea142f> in <module>()
1 while 1:
----> 2 get_near_ranking('hoge3333')
KeyboardInterrupt:
In [26]: client.zcount? # Redis のサイトを見ると、 count とかの範囲で開区間が使えるらしいぞ
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zcount of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zcount(self, name, min, max)
Docstring: <no docstring>
In [27]: client.zcount?? # ヘルプには書いてないから、ソース読んでみよう。
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Redis.zcount of <redis.client.Redis object at 0x1989510>>
Namespace: Interactive
File: /home/inada-n/.virtualenvs/redis/local/lib/python2.7/site-packages/redis/client.py
Definition: client.zcount(self, name, min, max)
Source:
def zcount(self, name, min, max):
return self.execute_command('ZCOUNT', name, min, max)
In [28]: client.zcount('hogeranking', 0, 9) # min, max はそのまま送ってるぽいから、、、
Out[28]: 992L
In [29]: client.zcount('hogeranking', 0, '(10') # これでいけた！
Out[29]: 992L
In [30]: H = 'hogeranking' # そろそろ毎回入力するのが面倒だ。
In [31]: client.zran
client.zrange client.zrangebyscore client.zrank
In [31]: client.zrange(H, 0, 1, withscores=True) # トップのやつを調べて、
Out[31]: [('hoge102401', 0.0), ('hoge14880', 0.0)]
In [32]: list(get_near_ranking('hoge102401')) # トップのやつの周辺を出せることを確認.
Out[32]:
[(1L, 'hoge102401', 0),
(1L, 'hoge14880', 0),
(1L, 'hoge15907', 0),
(1L, 'hoge169749', 0),
(1L, 'hoge201812', 0),
(1L, 'hoge211861', 0)]
In [33]: client.zrevra
client.zrevrange client.zrevrangebyscore client.zrevrank
In [33]: client.zrevrange(H, 0, 1) # ボトムのやつもチェックしとこ。
Out[33]: ['hoge997016', 'hoge993117']
In [34]: list(get_near_ranking('hoge997016')) # よし
Out[34]:
[(999880L, 'hoge984518', 9999),
(999880L, 'hoge989631', 9999),
(999880L, 'hoge991434', 9999),
(999880L, 'hoge991465', 9999),
(999880L, 'hoge993117', 9999),
(999880L, 'hoge997016', 9999)]
In [35]:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment