Back in 2016, I wrote about the ways we’re using GNU Readline for tab
completion in gitsh. The examples in that post, like the
gitsh tab completion code, were written in C. I’d have preferred to use Ruby,
but the complex tab completion in gitsh needed Readline features beyond those
provided by the Ruby
Having built on Ruby’s
Readline module to add complex tab
completion to gitsh, and then contributed those changes back to Ruby, I can
finally write the post I wanted to write two years ago. What follows is an
edited version of my original post, replacing all of that complex C with lovely
GNU Readline is a powerful line editor with support for fancy editing commands, history, and tab completion. Even if you’re not familiar with the name Readline you might still be using it: it’s integrated into all kinds of tools including GNU Bash, various language REPLs, and our own gitsh project.
This post will talk you through the more advanced Readline tab completion features gitsh uses and show you how to use them in your own programs.
To avoid getting lost in the details of the gitsh code, we’ll use a simplified example application for this post.
Basic tab completion
To get us started, here’s the simplest Readline program I can think of. It uses Readline to get input from the user, echoes that input back, and then exits.
Readline.readline method prompts the user for input,
with all of Readline’s power behind it.
This includes tab completion for file system paths.
If you don’t want to complete anything more than filenames
you don’t need to go any further than this.
It also includes history support:
if you pass
true as the second argument
user’s will be able to quickly repeat their previous inputs.
Custom completion options
In gitsh—and many other programs that use Readline—it’s useful to be able to complete things other than paths. In gitsh, we’re interested in completing things like Git commands, branch names, and remotes. For the purpose of this example, let’s say we’re only interested in completing values from a fixed list of the names of some characters from The Hitchiker’s Guide to the Galaxy.
Here’s our expanded program with custom tab completion:
We’re making use of a new Readline feature here:
we’ve set a
to provide custom completion options.
When the user hits their tab key
Readline will invoke the proc we’ve assigned to
The partial argument we’re completing
will be passed as an argument to the proc.
If we modify our
to print its argument,
we’d see something like this:
Note that we’re only passed
and not the whole input.
Given this information,
we need to return an
Array of possible completions.
Quoting and escaping
Our current implementation works well enough when the user is entering the name of a single character. But what would happen if they needed to enter a list of characters, separated by spaces? How would we know if we were seeing a space between a character’s first name and last name, or a space between two different characters?
Shells like bash, zsh, and gitsh get around this with quoting and escaping.
We could quote each character’s name:
"Arthur Dent" "Ford Prefect"
Or we could escape the spaces that don’t indicate the start of a new character’s name:
Arthur\ Dent Ford\ Prefect
Quoting and escaping are important for tab completion. As we’ve seen, Readline passes only the last argument of the user’s input to our completion proc. If we want to support quoting and escaping we need some way of telling Readline if the space separating two words counts as the start of a new argument. We also need to make sure that when we complete an argument containing a space that it is appropriately escaped.
The cases we need to cover are:
|"Arthur D⇥||"Arthur Dent"|
|Arthur\ D⇥||Arthur\ Dent|
Adding quoting support
Quoting is easier than escaping, so let’s tackle that first.
All we need to do
is tell Readline
which characters our program uses
as delimiters for quoted strings,
Now, when we press tab within a single- or double-quoted string, Readline will pass everything after the opening quote to our completion proc.
It’ll even close the quotes for us if there’s only one possible completion, or leave them open if there are several to choose from.
Adding escaping support
The first thing we need to do to support escaping is to make sure that the completion options we return are properly escaped.
We’d expect unquoted input to produce escaped output, and quoted input to produce unescaped but quoted output:
we’ve already set
so Readline is aware of whether or not we are
completing a quoted string.
We can modify our
which will tell us what the argument we’re completing was quoted with.
we’re completing an unquoted argument,
so we need to
produce escaped character names:
If Readline has seen an un-closed quote
will return the appropriate quote character
(in our case
since those are the characters we listed in
we know we’re not completing a quoted argument.
Detecting escaped word breaks
This is getting pretty good, but we’re still left with one case we can’t handle. If the user input contains a space that’s escaped:
|Arthur\ D⇥||Arthur\ Dent|
Readline will still see the space
as an argument boundary.
Our completion function will be passed
when we want it to be passed
To handle this,
we need to give Readline
a proc that it can call to discover
if the space between words is escaped,
which we can do with the
takes the whole line of input
and the index of the space
that might indicate a break between arguments,
or a quote character that might indicate the start of a quoted string.
It should return
if the character is quoted,
false if it is quoted:
It’s worth noting that this implementation is recursive.
In many shells,
it’s possible to escape the
\ character with another
\\ represents a literal
and doesn’t escape the character that follows it.
The recursion makes sure we handle
any number of
\ characters before a space,
and always do the right thing.
Readline won’t make use of
unless it believes some kind of quoting or escaping
is being used in the user’s input.
There are several practical implications of this:
quoting_detection_procis only ever called if
quoting_detection_procis only ever called if the input contains an unclosed quoted string, or a literal
\character. This limits the kind of escaping schemes we can implement to those that use a
\in some way.
Which characters separate arguments?
Readline will only invoke
with characters that would,
indicate a break between arguments.
quoting_detection_proc implementation to work,
we need to customise the list of word break characters:
Notice that we’ve been happily completing space-separated arguments from the very first example, so why do we need to explicitly specify this now?
The default value of
which we use for escaping.
If encountering a
\ indicated a word break,
we wouldn’t get very far with escaped spaces;
Readline would include the space in the value
passed to our completion proc,
but stop at the
An alternative solution to this problem
would be to decrement
but since we don’t need
to act as word breaks,
we can happily remove them
Running the example code
The code in this blog post won’t run correctly everywhere. The version of Ruby is important, and so is the line editing library that Ruby was compiled against.
GNU Readline vs. libedit
Ruby can use two different line editors: GNU Readline, which has more features but a more restrictive license; and libedit, which has fewer features and more bugs but a more permissive license. Everything in this post beyond basic tab completion requires GNU Readline.
You can determine which line editor your version of Ruby is compiled against using the following command:
ruby -r readline -e Readline.vi_editing_mode?
If the command exits silently, then your Ruby is built with GNU Readline. If it
fails with a
NotImplementedError, then it is built with libedit.
Most of these features have been available for a long time, but there are two more recent additions, which were extracted from gitsh and contributed upstream.
Readline.quoting_detection_procwas added in Ruby 2.4 (see Ruby patch #12659).
Readline.completion_quote_characterwas merged into Ruby trunk in September 2018, and should be in a production version of Ruby soon (see Ruby patch #13050).