---
title: Redis Pub/Sub... How Does it Work?
teaser:
tags: web,redis
author: Nick Quaranto
published_on: 2011-06-08
---

Redis is a [key/value store](http://redis.io), but it's jam-packed with a ton of
other little utilities that make it a joy to explore and implement. Two of these
are the PUBLISH and SUBSCRIBE commands, which enable you to do quick messaging
and communication between processes. Granted, there's plenty of other messaging
systems out there
([AMQP](http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol) and
[ØMQ](http://www.zeromq.org/) come to mind), but Redis is worth a look too.

The way it works is simple:

* [SUBSCRIBE](http://redis.io/commands/subscribe) will listen to a channel
* [PUBLISH](http://redis.io/commands/publish) allows you to push a message into
  a channel

Those two commands are all you need to build a messaging system with Redis. But,
what should you build?

## Demo

I tend to build quick little games to learn new ideas, frameworks, languages,
etc. For Redis pub/sub I chose to emulate IRC, since "channels" are essentially
the same concept for an IRC server. A user connects, talks into a channel, and
if others are there, they get the message. This is the basic concept, we're not
going to re-implement the IRC protocol here.

I scraped together two tiny little Ruby scripts for this. The repo is [on
GitHub](http://github.com/qrush/pubsub), if you want to play with it. Make sure
you're running Redis locally first! Install Redis via
[redis.io](http://redis.io/download) if not.

## PUBLISH

First up, `pub.rb` publishes messages to a channel. We're going to bring in the
[Redis](http://rubygems.org/gems/redis) gem to use for making a client, and
[JSON](http://rubygems.org/gems/json) in order to have an easy transport format.
We could have used Ruby's Marshal class instead to serialize, but this works
fine and is human readable.

    # usage:
    # ruby pub.rb channel username

    require 'rubygems'
    require 'redis'
    require 'json'

    $redis = Redis.new

    data = {"user" => ARGV[1]}

    loop do
      msg = STDIN.gets
      $redis.publish ARGV[0], data.merge('msg' => msg.strip).to_json
    end

This script will run interactively, once provided a channel and username from
the command line arguments. It then fires off the `PUBLISH` command every time
the user types a message and hits Enter. It publishes the message to a channel
(in `ARGV[0]`, the first command line argument.

So if we were to run:

    % ruby pub.rb rubyonrails qrush
    Hello world

Our Redis client would then send a PUBLISH command down the "rubyonrails"
channel with the given message. The message itself is <abbr title="JavaScript
Object Notation">JSON</abbr> and looks like:

    {
      "msg": "Hello world",
      "user": "qrush"
    }

We can actually verify this with the [MONITOR](http://redis.io/commands/monitor)
command, which will spit out all commands the Redis server has processed. If we
had MONITOR running before sending the above hello world snippet, it shows:

    % redis-cli
    redis> MONITOR
    OK
    1306462616.036890 "MONITOR"
    1306462620.482914 "publish" "rubyonrails" "{\"user\":\"qrush\",\"msg\":\"Hello world\"}"

Currently this simple script doesn't support publishing under more than one
channel. Opening up more than one `pub.rb` process will let you do that.

## SUBSCRIBE

Woot! Now that we have messages being sent we have to listen to them. Enter `sub.rb`:

    require 'rubygems'
    require 'redis'
    require 'json'

    $redis = Redis.new(:timeout => 0)

    $redis.subscribe('rubyonrails', 'ruby-lang') do |on|
      on.message do |channel, msg|
        data = JSON.parse(msg)
        puts "##{channel} - [#{data['user']}]: #{data['msg']}"
      end
    end

Once again we'll need Redis and <abbr title="JavaScript Object
Notation">JSON</abbr> to connect and parse messages. The initialization process
for Redis is different this time: it's using a new `:timeout` option. This will
force the Redis client to never timeout when waiting a response, so we'll wait
forever for messages to come in. Perfect!

This script subscribes to two different channels: `rubyonrails` and `ruby-lang`.
Basically, once the interpreter reaches the subscribe block, it will never exit
and continue to wait for messages.

When a message comes in, the `message` block is fired, yielding two arguments:
the channel the message was on, and the actual data sent down the pipe. Parsing
that <abbr title="JavaScript Object Notation">JSON</abbr> chunk then allows us
to spit out who said it, where they said it, and what was actually said. Here's
what it looks like if some other clients are publishing messages after we run
our `sub.rb` file. (The published messages arrive in the order they are sent,
but that's hard to display in text):

    % ruby pub.rb rubyonrails qrush
    Whoa!
    `rake routes` right?

    % ruby pub.rb rubyonrails turbage
    How do I list routes?
    Oh, duh. thanks bro.

    % ruby pub.rb ruby-lang qrush
    I think it's Array#include? you really want.

    % ruby sub.rb
    #rubyonrails - [qrush]: Whoa!
    #rubyonrails - [turbage]: How do I list routes?
    #ruby-lang - [qrush]: I think it's Array#include? you really want.
    #rubyonrails - [qrush]: `rake routes` right?
    #rubyonrails - [turbage]: Oh, duh. thanks bro.

Whoa! How does it work!? Under the hood in the redis-rb client, the `subscribe`
block is [actually
stuck](https://github.com/ezmobius/redis-rb/blob/c41bc94ea6e6eebb7184/lib/redis/client.rb#L71)
in a Ruby `loop` (called from
[Redis::SubscribedClient#subscription](https://github.com/ezmobius/redis-rb/blob/c41bc94ea6e6eebb718428d74d22c84b6d2513a4/lib/redis/subscribe.rb#L48)).
The client is going to continually attempt to read from the socket for messages,
until there's an error of some kind (but not a Timeout!). The redis-server then
[keeps a list of channels and
patterns](https://github.com/antirez/redis/blob/efc3408748061a08ec5ff18e392c30f31f5094a2/src/redis.h#L346-347)
for each connected client, and [publishes messages to
them](https://github.com/antirez/redis/blob/efc3408748061a08ec5ff18e392c30f31f5094a2/src/pubsub.c#L173-220)
when a PUBLISH command is sent.

## Usage

Although this is a simple example of how Redis pub/sub works, it's pretty cool
to see what others have done with it. Some examples include
[Convore](http://nosql.mypopescu.com/post/3351430579/convore-usage-of-redis-pub-sub),
which is used as a pretty central part of their infrastructure, and
[Realie](https://github.com/laktek/realie), a real-time code editor like
Etherpad. Another good link to check out is [Salvatore
Sanfilippo](http://twitter.com/antirez)'s recent interview on [The Changelog
(around
~24:00-27:00)](http://thechangelog.com/post/2801342864/episode-0-4-5-redis-with-salvatore-sanfilippo)
where he discusses that developers are switching from other MQs to Redis due to
its simplicity and performance.

If you're using Redis' Pub/Sub within your infrastructure, we'd love to hear
your feedback on how [Radish](http://radishapp.com) can provide more visibility
to what your messaging system is up to.