3 Mental Models For Ruby Enumerators

I love to collect mental models. These give me a lens (sometimes several!) for thinking about a given type of problem. For example, thinking of reduce as a tool to scale a method that combines 2 items to a method that combines n items helps me simplify complex reduce functions.

Ruby’s Enumerator (the class, NOT the Enumerable module!) can be a bit tricky to wrap your head around. Here are 3 different mental models I have for it:

1. An Enumerator is a cursor over a list
2. An Enumerator is a lazy list
3. An Enumerator is a series generator

Cursor

Ruby’s Enumerator can be thought of as a cursor within a collection. Unlike an array that has no sense of position, an enumerator remembers where it is in the list and can give you the next item(s). This persists even if you pass the enumerator around. In this way, it behaves kind of like Ruby’s IO objects.

array = [1, 2, 3, 4]
cursor = array.to_enum

cursor.next #=> 1
cursor.next #=> 2

In addition to the #next method which reads the next value and moves the cursor, you can use #peek to read the next value without moving the cursor. While you can’t step the cursor backwards, you can jump back to the beginning using #rewind.

If a cursor that you can move forward, pause, pass the cursor to another object, and then keep moving forward sounds like a familar concept, that might be because cursors are a form of the external iterator pattern.

Lazy List

Another way to think of Enumerator is as a lazy list. It’s a chain of Schrödinger’s boxes. You don’t know what each value will be (and neither does Ruby!) until you attempt to read it. Among other things, this allows us to construct infinite lists such as the list of all integers below.

integers = Enumerator.produce(1) { |n| n + 1 }
integers.take(3) #=> [1, 2, 3]
integers.next #=> 4
integers.next #=> 5

Note that if you try to traverse such an infinite list using a method like #map, it will attempt to evaluate every item in your (infinite) list. Regular enumerators allow us to lazily read items, but to get lazy traversals you need to call #lazy first.

integers.map { |n| n * 2 } #=> will execute infinitely

evens = integers.lazy.map { |n| n * 2 } #=> Enumerator::Lazy
evens.take(3) #=> [2, 4, 6]

This mental model is helpful when you only need to interact with the first part of a collection that might be infinite, just too big to fit in memory (e.g. loading 5 million property ids), or where each item is expensive to compute.

Series Generator

Enumerator can also be thought of as a series generator. In this mental model, think of an enumerator as a robot you can ask to generate new values for you according to a pattern. This pattern can even be random, as can be seen in the example below. Here we use an enumerator to construct a generator that will give us random numbers between 1 and 100.

generator = Enumerator.produce { rand(100) }
generator.take(5) #=> [92, 64, 99, 77, 71]
generator.next #=> 73
generator.next #=> 94

This perspective is helpful when you aren’t operating on static data but each value in the array needs to be calculated. For example, if you are fetching data from a paginated API, each page fetch generates a new set of records that will get yielded one at a time by the enumerator so only every $PAGE_SIZE fetch is expensive.

Teaching Ruby to Count

Want to dig more into these ideas? The 3 mental models were shared in my talk Teaching Ruby to Count. The talk explores how to make the most of out ranges, enumerators, and enumerables as well as how to make your custom objects play just as nicely with them as built-in Ruby primitives.