Ruby wrapper for Twitter Search API

Twitter Search (formerly Summize) is sweet. I use it every day to find first-person commentary on thoughtbot’s work, Boston sports, national politics, and anything else that catches my fancy.

The Twitter Search API is also sweet. I wanted to use it a while back for Politweets. In combination with a simple cron job, the effect is commentary on the U.S. Presidential Election from around the world, seen within minutes of being tweeted.

Dustin Sallings wrote a great Ruby wrapper for the Summize API that he put on github. I forked it and added a shoulda test suite and documentation.

Introducing the Twitter Search gem.

Usage

Install the gem.

gem install twitter-search

Require the gem.

require 'twitter_search'

Set up a TwitterSearch::Client. Name your client (a.k.a. ‘user agent’) to something meaningful, such as your app’s name. This helps Twitter Search answer any questions about your use of the API.

@client = TwitterSearch::Client.new 'politweets'

Request tweets by calling the query method of your client.

@tweets = @client.query :q => 'twitter search'

Search Operators

The following operator examples find tweets…

:q => \‘twitter search\’ – containing both twitter and search. This is the default operator.
:q => \‘happy hour\’ – containing the exact phrase happy hour.
:q => \‘obama OR hillary\’ – containing either obama or hillary (or both).
:q => \‘beer -root\’ – containing beer but not root.
:q => \‘#haiku\’ – containing the hashtag haiku.
:q => \‘from:alexiskold\’ – sent from person alexiskold.
:q => \‘to:techcrunch\’ – sent to person techcrunch.
:q => \‘@mashable\’ – referencing person mashable.
:q => \‘superhero since:2008-05-01\’ – containing superhero and sent since date 2008-05-01 (year-month-day).
:q => \‘ftw until:2008-05-03\’ – containing ftw and sent up to date 2008-05-03.
:q => \‘movie -scary :)\’ – containing movie, but not scary, and with a positive attitude.
:q => \‘flight :(\’ – containing flight and with a negative attitude.
:q => \‘traffic ?\’ – containing traffic and asking a question.
:q => \‘hilarious filter:links\’ – containing hilarious and linking to URLs.

Foreign Languages

The Twitter Search API supports foreign languages, accessible via the :lang key. Use the ISO 639-1 codes as the value:

@tweets = @client.query :q => 'programmé', :lang => 'fr'

Result pagination

Alter the number of Tweets returned per page with the :rpp key. Stick with 10, 15, 20, 25, 30, or 50.

@tweets = @client.query :q => 'Boston Celtics', :rpp => '30'

Gotchas

Searching for a positive attitude :) returns tweets containing the text :), =), :D, and :-)
Searches are case-insenstive.
The near operator available in the Twitter Search web interface is not available via the API. You must geocode before making your Twitter Search API call, and use the :geocode key in your request using the pattern lat,lngmi or lat,lngkm: @tweets = @client.query :q => 'Pearl Jam', :geocode => '43.4411,-70.9846mi'

Usage With ActiveRecord and Cron

You can get fancier with your setup, using queuing or another approach, but here’s a simple example using cron.

Schema:

create_table "tweets", :force => true do |t|
  t.string   "user_name",          :limit => 20,  :default => "", :null => false
  t.string   "body",               :limit => 140, :default => "", :null => false
  t.datetime "created_at",                                        :null => false
  t.datetime "updated_at",                                        :null => false
  t.integer  "twitter_id",         :limit => 11
end

add_index "tweets", ["created_at"], :name => "index_tweets_on_created_at"
add_index "tweets", ["twitter_id"], :name => "index_tweets_on_twitter_id"

Hit the API in your rake task (write the code in a class so it can be easily tested):

class Twitter
  def self.import!
    # the implementation is up to you
  end
end

Since I don’t have the cron API memorized, I like to create cron jobs with a handy little tool called crondle:

require 'lib/crondle'

Crondle.define_jobs do |builder|
  rails_root = '/var/www/apps/politweets/current'

  [3, 9, 15, 21, 27, 33, 39, 45, 51, 57].each do |minute|
    builder.desc "Import tweets at #{minute} past the hour"
    builder.job "#{rails_root}/script/runner Twitter.import!", :minute => minute
  end
end

Run the crondle script to get the text that you’ll put in crontab -e:

# Import tweets at 3 past the hour
3 * * * * /var/www/apps/politweets/current/script/runner Twitter.import!

# Import tweets at 9 past the hour
9 * * * * /var/www/apps/politweets/current/script/runner Twitter.import!

And you’re on your way.

Thanks to Doug, Gabe, Jason, Min, and Dustin for their work on Politweets and the Twitter Search gem.