README.rst 16.6 KB
Newer Older
A
andy 已提交
1 2
redis-py
========
A
andy 已提交
3 4 5

The Python interface to the Redis key-value store.

A
andy 已提交
6 7
.. image:: https://secure.travis-ci.org/andymccurdy/redis-py.png?branch=master
        :target: http://travis-ci.org/andymccurdy/redis-py
A
andy 已提交
8

A
andy 已提交
9 10
Installation
------------
A
andy 已提交
11

A
andy 已提交
12 13 14 15 16 17
redis-py requires a running Redis server. See `Redis's quickstart
<http://redis.io/topics/quickstart>`_ for installation instructions.

To install redis-py, simply:

.. code-block:: bash
A
andy 已提交
18 19 20 21 22

    $ sudo pip install redis

or alternatively (you really should be using pip though):

A
andy 已提交
23 24
.. code-block:: bash

A
andy 已提交
25 26
    $ sudo easy_install redis

A
andy 已提交
27 28 29
or from source:

.. code-block:: bash
A
andy 已提交
30 31 32 33

    $ sudo python setup.py install


A
andy 已提交
34 35 36 37
Getting Started
---------------

.. code-block:: pycon
A
andy 已提交
38 39 40 41 42 43 44 45

    >>> import redis
    >>> r = redis.StrictRedis(host='localhost', port=6379, db=0)
    >>> r.set('foo', 'bar')
    True
    >>> r.get('foo')
    'bar'

A
andy 已提交
46 47
API Reference
-------------
A
andy 已提交
48

A
andy 已提交
49 50 51 52
The `official Redis command documentation <http://redis.io/commands>`_ does a
great job of explaining each command in detail. redis-py exposes two client
classes that implement these commands. The StrictRedis class attempts to adhere
to the official command syntax. There are a few exceptions:
A
andy 已提交
53

A
andy 已提交
54
* **SELECT**: Not implemented. See the explanation in the Thread Safety section
A
andy 已提交
55
  below.
A
andy 已提交
56
* **DEL**: 'del' is a reserved keyword in the Python syntax. Therefore redis-py
A
andy 已提交
57
  uses 'delete' instead.
A
andy 已提交
58 59
* **CONFIG GET|SET**: These are implemented separately as config_get or config_set.
* **MULTI/EXEC**: These are implemented as part of the Pipeline class. The
A
andy 已提交
60 61 62
  pipeline is wrapped with the MULTI and EXEC statements by default when it
  is executed, which can be disabled by specifying transaction=False.
  See more about Pipelines below.
A
andy 已提交
63
* **SUBSCRIBE/LISTEN**: Similar to pipelines, PubSub is implemented as a separate
A
andy 已提交
64 65 66
  class as it places the underlying connection in a state where it can't
  execute non-pubsub commands. Calling the pubsub method from the Redis client
  will return a PubSub instance where you can subscribe to channels and listen
A
andy 已提交
67 68 69 70
  for messages. You can only call PUBLISH from the Redis client (see
  `this comment on issue #151
  <https://github.com/andymccurdy/redis-py/issues/151#issuecomment-1545015>`_
  for details).
A
andy 已提交
71 72 73 74 75

In addition to the changes above, the Redis class, a subclass of StrictRedis,
overrides several other commands to provide backwards compatibility with older
versions of redis-py:

A
andy 已提交
76
* **LREM**: Order of 'num' and 'value' arguments reversed such that 'num' can
A
andy 已提交
77
  provide a default value of zero.
A
andy 已提交
78
* **ZADD**: Redis specifies the 'score' argument before 'value'. These were swapped
A
andy 已提交
79
  accidentally when being implemented and not discovered until after people
A
andy 已提交
80
  were already using it. The Redis class expects \*args in the form of:
A
andy 已提交
81
  `name1, score1, name2, score2, ...`
A
andy 已提交
82
* **SETEX**: Order of 'time' and 'value' arguments reversed.
A
andy 已提交
83 84


A
andy 已提交
85 86
More Detail
-----------
A
andy 已提交
87

A
andy 已提交
88 89
Connection Pools
^^^^^^^^^^^^^^^^
A
andy 已提交
90 91 92 93 94 95

Behind the scenes, redis-py uses a connection pool to manage connections to
a Redis server. By default, each Redis instance you create will in turn create
its own connection pool. You can override this behavior and use an existing
connection pool by passing an already created connection pool instance to the
connection_pool argument of the Redis class. You may choose to do this in order
A
andy 已提交
96 97 98 99
to implement client side sharding or have finer grain control of how
connections are managed.

.. code-block:: pycon
A
andy 已提交
100 101 102 103

    >>> pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
    >>> r = redis.Redis(connection_pool=pool)

A
andy 已提交
104 105
Connections
^^^^^^^^^^^
A
andy 已提交
106 107 108 109 110 111 112 113 114 115

ConnectionPools manage a set of Connection instances. redis-py ships with two
types of Connections. The default, Connection, is a normal TCP socket based
connection. The UnixDomainSocketConnection allows for clients running on the
same device as the server to connect via a unix domain socket. To use a
UnixDomainSocketConnection connection, simply pass the unix_socket_path
argument, which is a string to the unix domain socket file. Additionally, make
sure the unixsocket parameter is defined in your redis.conf file. It's
commented out by default.

A
andy 已提交
116 117
.. code-block:: pycon

A
andy 已提交
118 119 120 121 122 123 124 125 126
    >>> r = redis.Redis(unix_socket_path='/tmp/redis.sock')

You can create your own Connection subclasses as well. This may be useful if
you want to control the socket behavior within an async framework. To
instantiate a client class using your own connection, you need to create
a connection pool, passing your class to the connection_class argument.
Other keyword parameters your pass to the pool will be passed to the class
specified during initialization.

A
andy 已提交
127 128
.. code-block:: pycon

A
andy 已提交
129 130 131
    >>> pool = redis.ConnectionPool(connection_class=YourConnectionClass,
                                    your_arg='...', ...)

A
andy 已提交
132 133
Parsers
^^^^^^^
A
andy 已提交
134 135 136 137 138 139 140 141 142 143 144 145 146

Parser classes provide a way to control how responses from the Redis server
are parsed. redis-py ships with two parser classes, the PythonParser and the
HiredisParser. By default, redis-py will attempt to use the HiredisParser if
you have the hiredis module installed and will fallback to the PythonParser
otherwise.

Hiredis is a C library maintained by the core Redis team. Pieter Noordhuis was
kind enough to create Python bindings. Using Hiredis can provide up to a
10x speed improvement in parsing responses from the Redis server. The
performance increase is most noticeable when retrieving many pieces of data,
such as from LRANGE or SMEMBERS operations.

A
andy 已提交
147
Hiredis is available on PyPI, and can be installed via pip or easy_install
A
andy 已提交
148 149
just like redis-py.

A
andy 已提交
150 151
.. code-block:: bash

A
andy 已提交
152 153 154 155
    $ pip install hiredis

or

A
andy 已提交
156 157
.. code-block:: bash

A
andy 已提交
158 159
    $ easy_install hiredis

A
andy 已提交
160 161
Response Callbacks
^^^^^^^^^^^^^^^^^^
A
andy 已提交
162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179

The client class uses a set of callbacks to cast Redis responses to the
appropriate Python type. There are a number of these callbacks defined on
the Redis client class in a dictionary called RESPONSE_CALLBACKS.

Custom callbacks can be added on a per-instance basis using the
set_response_callback method. This method accepts two arguments: a command
name and the callback. Callbacks added in this manner are only valid on the
instance the callback is added to. If you want to define or override a callback
globally, you should make a subclass of the Redis client and add your callback
to its REDIS_CALLBACKS class dictionary.

Response callbacks take at least one parameter: the response from the Redis
server. Keyword arguments may also be accepted in order to further control
how to interpret the response. These keyword arguments are specified during the
command's call to execute_command. The ZRANGE implementation demonstrates the
use of response callback keyword arguments with its "withscores" argument.

A
andy 已提交
180 181
Thread Safety
^^^^^^^^^^^^^
A
andy 已提交
182 183 184 185 186 187 188 189 190 191 192 193

Redis client instances can safely be shared between threads. Internally,
connection instances are only retrieved from the connection pool during
command execution, and returned to the pool directly after. Command execution
never modifies state on the client instance.

However, there is one caveat: the Redis SELECT command. The SELECT command
allows you to switch the database currently in use by the connection. That
database remains selected until another is selected or until the connection is
closed. This creates an issue in that connections could be returned to the pool
that are connected to a different database.

A
andy 已提交
194 195 196 197
As a result, redis-py does not implement the SELECT command on client
instances. If you use multiple Redis databases within the same application, you
should create a separate client instance (and possibly a separate connection
pool) for each database.
A
andy 已提交
198 199 200

It is not safe to pass PubSub or Pipeline objects between threads.

A
andy 已提交
201 202
Pipelines
^^^^^^^^^
A
andy 已提交
203 204 205 206 207 208 209 210

Pipelines are a subclass of the base Redis class that provide support for
buffering multiple commands to the server in a single request. They can be used
to dramatically increase the performance of groups of commands by reducing the
number of back-and-forth TCP packets between the client and server.

Pipelines are quite simple to use:

A
andy 已提交
211 212
.. code-block:: pycon

A
andy 已提交
213 214 215 216 217 218 219 220 221 222 223 224 225 226 227
    >>> r = redis.Redis(...)
    >>> r.set('bing', 'baz')
    >>> # Use the pipeline() method to create a pipeline instance
    >>> pipe = r.pipeline()
    >>> # The following SET commands are buffered
    >>> pipe.set('foo', 'bar')
    >>> pipe.get('bing')
    >>> # the EXECUTE call sends all buffered commands to the server, returning
    >>> # a list of responses, one for each command.
    >>> pipe.execute()
    [True, 'baz']

For ease of use, all commands being buffered into the pipeline return the
pipeline object itself. Therefore calls can be chained like:

A
andy 已提交
228 229
.. code-block:: pycon

A
andy 已提交
230 231 232 233 234 235 236 237
    >>> pipe.set('foo', 'bar').sadd('faz', 'baz').incr('auto_number').execute()
    [True, True, 6]

In addition, pipelines can also ensure the buffered commands are executed
atomically as a group. This happens by default. If you want to disable the
atomic nature of a pipeline but still want to buffer commands, you can turn
off transactions.

A
andy 已提交
238 239
.. code-block:: pycon

A
andy 已提交
240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257
    >>> pipe = r.pipeline(transaction=False)

A common issue occurs when requiring atomic transactions but needing to
retrieve values in Redis prior for use within the transaction. For instance,
let's assume that the INCR command didn't exist and we need to build an atomic
version of INCR in Python.

The completely naive implementation could GET the value, increment it in
Python, and SET the new value back. However, this is not atomic because
multiple clients could be doing this at the same time, each getting the same
value from GET.

Enter the WATCH command. WATCH provides the ability to monitor one or more keys
prior to starting a transaction. If any of those keys change prior the
execution of that transaction, the entire transaction will be canceled and a
WatchError will be raised. To implement our own client-side INCR command, we
could do something like this:

A
andy 已提交
258 259
.. code-block:: pycon

A
andy 已提交
260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290
    >>> with r.pipeline() as pipe:
    ...     while 1:
    ...         try:
    ...             # put a WATCH on the key that holds our sequence value
    ...             pipe.watch('OUR-SEQUENCE-KEY')
    ...             # after WATCHing, the pipeline is put into immediate execution
    ...             # mode until we tell it to start buffering commands again.
    ...             # this allows us to get the current value of our sequence
    ...             current_value = pipe.get('OUR-SEQUENCE-KEY')
    ...             next_value = int(current_value) + 1
    ...             # now we can put the pipeline back into buffered mode with MULTI
    ...             pipe.multi()
    ...             pipe.set('OUR-SEQUENCE-KEY', next_value)
    ...             # and finally, execute the pipeline (the set command)
    ...             pipe.execute()
    ...             # if a WatchError wasn't raised during execution, everything
    ...             # we just did happened atomically.
    ...             break
    ...        except WatchError:
    ...             # another client must have changed 'OUR-SEQUENCE-KEY' between
    ...             # the time we started WATCHing it and the pipeline's execution.
    ...             # our best bet is to just retry.
    ...             continue

Note that, because the Pipeline must bind to a single connection for the
duration of a WATCH, care must be taken to ensure that the connection is
returned to the connection pool by calling the reset() method. If the
Pipeline is used as a context manager (as in the example above) reset()
will be called automatically. Of course you can do this the manual way by
explicity calling reset():

A
andy 已提交
291 292
.. code-block:: pycon

A
andy 已提交
293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310
    >>> pipe = r.pipeline()
    >>> while 1:
    ...     try:
    ...         pipe.watch('OUR-SEQUENCE-KEY')
    ...         ...
    ...         pipe.execute()
    ...         break
    ...     except WatchError:
    ...         continue
    ...     finally:
    ...         pipe.reset()

A convenience method named "transaction" exists for handling all the
boilerplate of handling and retrying watch errors. It takes a callable that
should expect a single parameter, a pipeline object, and any number of keys to
be WATCHed. Our client-side INCR command above can be written like this,
which is much easier to read:

A
andy 已提交
311 312
.. code-block:: pycon

A
andy 已提交
313 314 315 316 317 318 319 320 321
    >>> def client_side_incr(pipe):
    ...     current_value = pipe.get('OUR-SEQUENCE-KEY')
    ...     next_value = int(current_value) + 1
    ...     pipe.multi()
    ...     pipe.set('OUR-SEQUENCE-KEY', next_value)
    >>>
    >>> r.transaction(client_side_incr, 'OUR-SEQUENCE-KEY')
    [True]

A
andy 已提交
322 323
LUA Scripting
^^^^^^^^^^^^^
A
andy 已提交
324 325 326 327 328 329 330 331 332 333

redis-py supports the EVAL, EVALSHA, and SCRIPT commands. However, there are
a number of edge cases that make these commands tedious to use in real world
scenarios. Therefore, redis-py exposes a Script object that makes scripting
much easier to use.

To create a Script instance, use the `register_script` function on a client
instance passing the LUA code as the first argument. `register_script` returns
a Script instance that you can use throughout your code.

A
andy 已提交
334 335
The following trivial LUA script accepts two parameters: the name of a key and
a multiplier value. The script fetches the value stored in the key, multiplies
A
andy 已提交
336 337
it with the multiplier value and returns the result.

A
andy 已提交
338 339
.. code-block:: pycon

A
andy 已提交
340 341 342 343 344 345 346 347 348 349
    >>> r = redis.StrictRedis()
    >>> lua = """
    ... local value = redis.call('GET', KEYS[1])
    ... value = tonumber(value)
    ... return value * ARGV[1]"""
    >>> multiply = r.register_script(lua)

`multiply` is now a Script instance that is invoked by calling it like a
function. Script instances accept the following optional arguments:

A
andy 已提交
350 351 352 353 354 355 356
* **keys**: A list of key names that the script will access. This becomes the
  KEYS list in LUA.
* **args**: A list of argument values. This becomes the ARGV list in LUA.
* **client**: A redis-py Client or Pipeline instance that will invoke the
  script. If client isn't specified, the client that intiially
  created the Script instance (the one that `register_script` was
  invoked from) will be used.
A
andy 已提交
357 358 359

Continuing the example from above:

A
andy 已提交
360 361
.. code-block:: pycon

A
andy 已提交
362 363 364 365 366 367 368 369 370 371 372
    >>> r.set('foo', 2)
    >>> multiply(keys=['foo'], args=[5])
    10

The value of key 'foo' is set to 2. When multiply is invoked, the 'foo' key is
passed to the script along with the multiplier value of 5. LUA executes the
script and returns the result, 10.

Script instances can be executed using a different client instance, even one
that points to a completely different Redis server.

A
andy 已提交
373 374
.. code-block:: pycon

A
andy 已提交
375 376 377 378 379 380 381 382 383 384 385 386 387 388
    >>> r2 = redis.StrictRedis('redis2.example.com')
    >>> r2.set('foo', 3)
    >>> multiply(keys=['foo'], args=[5], client=r2)
    15

The Script object ensures that the LUA script is loaded into Redis's script
cache. In the event of a NOSCRIPT error, it will load the script and retry
executing it.

Script objects can also be used in pipelines. The pipeline instance should be
passed as the client argument when calling the script. Care is taken to ensure
that the script is registered in Redis's script cache just prior to pipeline
execution.

A
andy 已提交
389 390
.. code-block:: pycon

A
andy 已提交
391 392 393 394 395 396
    >>> pipe = r.pipeline()
    >>> pipe.set('foo', 5)
    >>> multiply(keys=['foo'], args=[5], client=pipe)
    >>> pipe.execute()
    [True, 25]

397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432
Sentinel support
^^^^^^^^^^^^^^^^

redis-py can be used together with `Redis Sentinel <http://redis.io/topics/sentinel>`_
to discover redis nodes on the fly using service name instead of address.
First of all you need to define sentinel cluster:

.. code-block:: pycon

    >>> from redis.sentinel import Sentinel
    >>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)
    >>> sentinel.discover_master('mymaster')
    ('127.0.0.1', 6379)
    >>> sentinel.discover_slave('mymaster')
    ('127.0.0.1', 6380)

Then you can use sentinel instance to acquire master or slave client for the
corresponding service name:

.. code-block:: pycon

    >>> master = sentinel.master_for('mymaster', socket_timeout=0.1)
    >>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
    >>> master.set('foo', 'bar')
    >>> slave.get('foo')
    'bar'

master and slave objects are normal StrictRedis instances with their connection
pool bound to the sentinel instance. To establish connection with a redis server
sentinel servers will be asked for the address of 'mymaster' service. If no
server is found MasterNotFoundError or SlaveNotFoundError is raised both
exceptions are subclasses of ConnectionError.

See `Guidelines for Redis clients with support for Redis Sentinel
<http://redis.io/topics/sentinel-clients>`_ to learn more abour Redis Sentinel.

A
andy 已提交
433
Author
A
andy 已提交
434
^^^^^^
A
andy 已提交
435 436 437 438 439 440 441 442 443 444 445

redis-py is developed and maintained by Andy McCurdy (sedrik@gmail.com).
It can be found here: http://github.com/andymccurdy/redis-py

Special thanks to:

* Ludovico Magnocavallo, author of the original Python Redis client, from
  which some of the socket code is still used.
* Alexander Solovyov for ideas on the generic response callback system.
* Paul Hubbard for initial packaging support.