Intelligent Navigation With ctags

Video

Want to see the full-length video right now for free?

Notes

You can download a cheat sheet and install instructions for all of the tools shown in this video.

Transcript

ctags is an external program that scans through your codebase, producing an index of keywords. Vim has built-in ctags integration, which lets you quickly navigate any codebase that has been indexed by ctags.

In the first half of this tutorial, we'll see what Vim can do when properly configured to work with ctags. In the second half, I'll demonstrate how to setup ctags to automatically index the code in your project, as well as any bundled gems, and ruby's standard library.

Ctags Usage

Here I have the source code for appraisal, which is an open source project by thoughtbot. Let's open up the lib/appraisal.rb file:

vim lib/appraisal.rb

Jumping to a definition (and getting back to where you started)

Watch this: I'll position my cursor on the word Task, then use the <C-]> "control-close-bracket" mapping to jump to the definition of the current word. Boom! With one command, we've jumped to the Task class.

This happens to be a subclass of the rake TaskLib class. Let's try invoking the jump-to-definition command on the superclass. Awesome! That works too! Note the path of this file: it's not part of the appraisal project, but I've configured Vim to understand the project's dependencies.

Let's try that one more time, for good measure: we'll jump to the DSL definition.

Well that was three jumps, and it feels as though we're a long way from where we started. The good news is that Vim maintains a history of jumps made using the go to definition command. We can inspect the stack by running the tags Ex command:

:tags
  # TO tag         FROM line  in file/text
  1  1 Task                4  lib/appraisal.rb
  2  1 TaskLib             7  ~/appraisal/lib/appraisal/task.rb
  3  1 DSL                 8  /usr/local/rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/rake/tasklib.rb

The most recently visited locations appear last.

You can think of the go to definition command as being like clicking a hyperlink in a web browser. Vim also provides the equivalent of a back button, which can be invoked by pressing <C-t> in Normal mode. Let's try that a couple of times, then inspect the tag list again:

:tags
  # TO tag         FROM line  in file/text
  1  1 Task                4  lib/appraisal.rb
> 2  1 TaskLib             7  ~/appraisal/lib/appraisal/task.rb
  3  1 DSL                 8  /usr/local/rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/rake/tasklib.rb

The > "greater than sign" points to the current position within the tag stack. If I use the back button a couple more times, Vim eventually issues the warning:

at bottom of tag stack

Sure enough, we are back where we started.

:help tagstack

Choosing between multiple matches

Let's see what happens when we invoke the go to definition command on Appraisal. That takes us to the definition of a class called Appraisal, but look at this: the class is defined within a module of the same name. So why did Vim choose to take us to the class definition instead of the module definition?

Truth is, both definitions have been indexed by ctags. When Vim is instructed to go to a tag for which there are multiple matches, it ranks the tags according to rules of precedence, then jumps to the match with the highest priority. If you want to know more, you can read up on the rules in Vim's help:

:help tag-priority

Let's back up with the <C-t> mapping and try something different. Instead of using "control close bracket", I'll use the same command prefixed with the g key. This time, we get to see a list of all the tags that match the word Appraisal.

# pri kind tag               file
1 F   c    Appraisal         /home/vagrant/appraisal/lib/appraisal/appraisal.rb
             class:Appraisal
             class Appraisal
2 F   m    Appraisal         /home/vagrant/appraisal/lib/appraisal/appraisal.rb
             module Appraisal
3 F   m    Appraisal         /home/vagrant/appraisal/lib/appraisal/command.rb
             module Appraisal
4 F   m    Appraisal         /home/vagrant/appraisal/lib/appraisal/dependency.rb
             module Appraisal

Each match is numbered, with various annotations including the filepath. We can jump to any of these tags by entering their number and pressing return. Let's try number 2 - and that turns out to be the module that contains a class of the same name.

Vim provides a few commands to help us interact with the list of matches. I can view the full list by running the :tselect Ex command:

:tselect
  # pri kind tag               file
  1 F   c    Appraisal         /home/vagrant/appraisal/lib/appraisal/appraisal.rb
               class:Appraisal
               class Appraisal
> 2 F   m    Appraisal         /home/vagrant/appraisal/lib/appraisal/appraisal.rb
               module Appraisal
  3 F   m    Appraisal         /home/vagrant/appraisal/lib/appraisal/command.rb
               module Appraisal

Note the "greater than sign", which marks the tag we selected. Having run :tselect, we can choose any item from the list by number, as we did before. But we can also traverse the list of matches using the tnext, tprev, tfirst, and tlast commands:

Ex cmd     Unimpaired mapping
:tfirst    [T
:tprev     [t
:tnext     ]t
:tlast     ]T

Tim Pope's unimpaired plugin supplies a few handy mappings to make it easier to run these commands. I recommend installing it if you haven't already done so.

Classes, Modules, and methods are all definitions

So far, I've shown that the go to definition command can jump to Module names and Class names. It also works on method definitions. Let's jump to the Task class and try that out. I'll enter a pattern that lets me jump to method invocations:

/\.\zs\w

Let's try jumping to the definition of File.each: that works. What about appraisal.write_gemfile? That works too.

These methods are defined within the Appraisal project, but here we're using the fileutils module from Ruby's standard library. Sure enough, the go to definition command works fine on the FileUtils module, and we can also use it to jump directly to the definition of the rm_f "force remove" method.

Ex commands

Vim has a couple of Ex commands that complement the go to definition mappings:

<C-]>   :tag {keyword}     - go to the first match for {keyword}
g<C-]>  :tjump {keyword}   - prompt user to select from multiple matches for {keyword}

The :tag command accepts a keyword as an argument, and jumps directly to the first match (this behavior is similar to the "control close bracket" command). The :tjump command also goes directly to the first match if there only is one match, but if there are multiple matches it shows a menu prompting the user to choose where to go (just like the g<C-]> mapping).

Also, :tag and :tjump can accept a pattern!

If the method you want to look up is right there in front of you, it's surely quickest to position your cursor on the keyword and use one of the go to definition mappings. But in many cases, you might want to look up the definition of a method that you haven't used yet. That's where these Ex commands come into their own, particularly because they hook into Vim's tab-completion behavior.

Suppose that we're working with the minitest library, and we'd like to know which assertions are available. Let's use the :tag Ex command to look up the definition of assert

:tag assert

Press enter, and boom! We go straight to the source of the method.

We could scroll through the rest of this file to browse the other assertions, but here's another way. Enter the same Ex command, but instead of pressing enter, I'll use "control dee":

:tag assert<C-d>
assert                          assert_block
assert_block                    assert_equal
assert_empty                    assert_headers_equal
assert_equal                    assert_no_match
assert_in_delta                 assert_not_equal
assert_in_epsilon               assert_not_nil
assert_includes                 assert_not_same
assert_instance_of              assert_not_send
assert_kind_of                  assert_nothing_raised
assert_match                    assert_nothing_thrown
assert_nil                      assert_path_exists
assert_operator                 assert_performance
assert_output                   assert_performance_constant
assert_raises                   assert_performance_exponential
assert_respond_to               assert_performance_linear
assert_same                     assert_performance_power
assert_send                     assert_raise
assert_silent                   assert_respond_to
assert_throws                   assert_send

That reveals all of the matching tags, giving us a concise list of all the assertions provided by minitest. Pressing the tab key cycles forward through the list.

Ctags configuration

We've seen what Vim is capable of when properly configured to work with ctags. Now let's focus on how to set up your own environment the same way.

Install ctags

First and foremost, we have to install exuberant ctags.

Mac users: beware that OS X ships with a BSD program that goes by the name of ctags.

man -M /usr/share/man ctags

That's not the program you want! You can get exuberant ctags via homebrew:

brew install ctags

Make sure that the exuberant ctags executable appears in your path before the BSD program.

which -a ctags
/usr/local/bin/ctags
/usr/bin/ctags
/usr/local/bin/ctags

If you've set it up right, you should get a message like this when you run ctags dash-dash version:

$ ctags --version
Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
Compiled: Feb  1 2013, 15:58:37
Addresses: <dhiebert@users.sourceforge.net>, http://ctags.sourceforge.net
Optional compiled features: +wildcards, +regex

This virtual machine is running Ubuntu, and I can get the program by running:

sudo apt-get install exuberant-ctags

Basic usage

make Vim aware of the location of the tags file

Here I've got a copy of the appraisal project that has not been indexed yet. If I attempt to use Vim's go to definition mapping on the word Appraisal, Vim reports an error:

tag not found

Let's switch to the shell and invoke ctags to index this codebase:

ctags -R .

The "minus big are" flag tells ctags to recurse through sub directories. Now there's a tags file in our project directory:

ls -l | grep tags

Out of curiosity, let's open that up:

vim tags

Each line of the file represents a tag, which consists of a name, filepath, and a search pattern that Vim can use to locate the tag within the specified file. Note too that the entries are sorted, which lets Vim search the index quickly.

If we open up our source code now, the go to definition mapping should work for any modules, classes or methods that are defined in this project.

Vim automatically looks for a file called tags in the current working directory, so in this case we didn't have to do any more configuration. But if we had saved the tags file to some other location, then we could inform Vim of where to look for it by setting the tags option:

:help 'tags'

We can specify multiple tags files, and Vim will search each one in turn until it finds a match.

To summarize, two steps are required to make Vim's tag functionality work:

the codebase must be indexed using ctags
Vim must be able to locate the tags index file.

Instead of running these by hand, it would be ideal if we could set up our environment to handle them automatically.

In the next few sections, we'll look at how to make this work for your project, for your bundled gems, and finally for the ruby standard library.

Credit goes to Tim Pope for devising these strategies and writing each of the required plugins.

Installing Vim plugins

For this demo, I'm using a Vim plugin manager called Vundle. We'll start off with a minimal setup, then we'll install each of these plugins and see how they help.

I've also created a custom command:

:TagFiles

That inspects Vim's tags option, showing one entry per line.

ctags for your project

Tim Pope has devised a strategy that uses git hooks to automatically run ctags on your project codebase whenever you checkout, commit, merge or rebase. The setup is a little bit fiddly: you have to create a global git template containing a few different shell scripts. But the beauty of this solution is that you only have to set it up once on your machine, then all of your projects that are managed by git will be automatically indexed by ctags.

I've run through Tim Pope's instructions off camera. Now I can run:

git init

to apply the template to this repository, and

git ctags

to index the appraisal codebase.

Instead of generating a tags file in the project root, which would require you to add an extra line to your .gitignore file, Tim Pope recommends saving the tags file in the .git directory.

Install the fugitive plugin:

uncomment
save the change
source the file, and run
bundle install

The fugitive plugin will configure Vim to look for a tags file in the .git directory. And now, the go to definition command will work for modules, classes and methods defined in this project.

The tags file will go stale as we make changes to the codebase. For example, if I rename the Task class to something else then attempt to go to the new definition, Vim tells us it can't find the tag.

Now if I commit those changes:

git add lib
git commit -m 's/Task/Quest'

the post-commit hook kicks in, re-indexing our codebase with ctags. Now I can jump to the definition of the Quest class. The tags file will also be regenerated following a checkout, merge or rebase, which means your project is practically always indexed.

Note that these git hooks will kick in automatically for all repositories that you initialize or clone after creating the template, but for git repositories that were created before the template, the hooks won't apply. You can fix that by running:

git init

in the project root of each of your pre-existing git repositories.

ctags for bundled gems

We still can't use the go to definition mapping on bundled gems: when I use it on the Rake module I get an error. Let's fix that.

[gem-ctags][] is a plugin for Rubygems that automatically invokes ctags on gems as they are installed. The plugin itself is a rubygem, so you can install it by running:

gem install gem-ctags

Now we can run:

gem ctags

to index all the rubygems that are already installed on the system. You won't have to run that command again though. From now on, each time you install a rubygem it will be automatically indexed by ctags.

Now all we have to do is make sure that Vim can find those tags index files, and we'll be able to use Vim's ctags navigation features on installed rubygems. The vim-bundler plugin tells Vim where to find the tags index file for each gem listed in a project's Gemfile.

Let's enable the vim-bundler plugin, then restart Vim and inspect the list of tags files:

:TagFiles

Sure enough, it references each of the bundled gems.

Now it works when I use the go to definition mapping on rake.

ctags for ruby standard library

We still can't use the go to definition mapping on Ruby's standard library. For example, nothing happens when I use it on the force remove method from the FileUtils module.

If you're using rbenv to manage your ruby environment, you should install Tim Pope's rbenv-ctags plugin.

I'll just copy and paste the instructions direct from the README... that's all there is to it.

This plugin provides a convenient command for running ctags to index rubies that are managed by rbenv:

rbenv ctags

If you use ruby-build to install versions of ruby, then this plugin will automatically index your rubies as they are installed.

Now we just have to instruct Vim on where to find the tags index files for our ruby standard library. Recent versions of the vim-ruby plugin will handle this. Even though this plugin comes bundled with the standard Vim distribution, I recommend installing it by hand to get the latest version.

Now if we inspect the tags option, it references the tags file in each directory from Vim's load path:

:TagFiles

And sure enough, the go to definition command now works on methods defined in Ruby's standard library.

Outro

To recap: we've set things up so that ctags automatically indexes our project, bundled gems, and Ruby's standard library. Thanks to the fugitive, bundler, and ruby plugins, Vim knows where to find the generated tag files. It takes a bit of effort to set all of this up, but it's so worthwhile. You'll get a terrific boost from being able to use ctags in all your ruby projects.

Navigating Ruby Files with Vim

16 minutes