The Case of The Enigmatic Enumerable Module

By Jade McGough / Developer
Mar 20, 2017

As it sometimes happens, the problem started with updating a Ruby gem.

Dawn had been working for SaaS.me for the past year, developing a localized social network app for dogs, which made heavy use of social media APIs. Their Facetwit gem’s version had been locked down some time ago and was starting to cause dependency conflicts, meaning that it was time to upgrade it.

The gem update seemed to go smoothly at first, but she soon realized the application dashboard was missing its list of recent tweets by users in nearby dog parks. Checking the response from the facetwit client revealed an annoying 429: Too Many Requests. Dawn frowned, waited several minutes for her limit to refresh, only to be greeted by the 429 again. This one had a new expiration time, suggesting that she’d somehow exhausted her queries again.

Poking around the controller didn’t immediately reveal anything out of the ordinary:

class FacetwitController < ApplicationController
def search
client = Facetwit::Client.new(api_key: 'fuzzy_pickles')
@posts = client.search(params[:query])
respond_to do |format|
format.json { render json: @posts }
end
end
end

She stepped through with a debugger after that, and noticed something interesting when she examined @posts. She’d expected it to be an array of responses returned from the search, but it was actually a Facetwit::SearchResults object. Even more surprisingly, client.search wasn’t actually making an API call at all. So how was her query limit being exhausted?

The Enumerable Module

Let’s step back for a moment. One of Ruby’s great strengths is its powerful collection manipulation methods, such as inject, sort_by and map. These are made available in both Array and Hash by Ruby’s Enumerable module.

In fact, you can gain access to all of these methods in any class. You simply need to include the module and create an each definition which yields members to a block. Ruby is smart enough to extrapolate all of the other methods from your each definition (though you’ll also need to define <=> for sorting methods).

class Pack
include Enumerable
attr_accessor :dogs

def initialize
@dogs = []
end

def each(&block)
@dogs.each{ |dog| block.call(dog) }
end
end

Dog = Struct.new(:name, :breed)

> pack = Pack.new
> pack.dogs << Dog.new("Fido", "Pug")
> pack.dogs << Dog.new("Sparky", "Beagle")
> pack.map(&:breed)
=> ["Pug", "Beagle"]

So how does this relate to Dawn’s problem? Well, what if each wasn’t just used for iterating through a collection, but requesting the actual collection as well?

Module Facetwit
class SearchResults
include Enumerable

def each(start = 0)
@posts.each{ |item| yield(item) }
unless finished?
start = [@posts.size, start].max
get_next
each(start, &Proc.new)
end
end
end
end

Dawn saw this and realized that, given how popular a platform Facetwit is, iterating through a single Facetwit::SearchResults object would continue to fetch results until it exhausted her available queries. The searching wasn’t being kicked off in her controller, but here in her view:

<% @posts.each do |post| %>

In order to limit the number of external requests, Dawn made use of another Enumerable module method, take.

<% @posts.take(10).each do |post| %>

With that simple change she reloaded the page again and was greeted by ten posts from Dolores Park.

Don't Be Enigmatic

Dawn immediately had to debug her code because an external library was behaving unexpectedly. The programmer who made the change violated the principle of least astonishment. This is an important concept that many Rubyists try to adhere to which states that code should not behave in a way that’s unnecessarily surprising.

Using Enumerable module for pagination can be an elegant pattern to use when writing an external API gem, as long as it’s clear to the user that use of Enumerable methods can make multiple external requests. In a single-threaded Rails application, actively waiting for several HTTP responses in a controller could lock up the application for an unacceptably long period of time.

After all, you should be able to appreciate good code without needing to solve a mystery.