A Warning Sign
Los Angeles is famous for its complicated parking signs:
Sunny totems of rules and exceptions, and exceptions to the exceptions. Often, when we code, we forget a lesson that’s obvious in these preposterous signs: Humans understand simple, consistent rules, but fail on special cases.
A trivial example
Say you’re given an array of integers, and you want to calculate the sum of each element’s neighbors. Try it:
1 3 4 5 0 1 8 --- | 3 --+
Before going on, consider how you’d solve this yourself.
A straightforward approach might look like this:
input = [1, 3, 4, 5, 0, 1, 8]
result = input.map.with_index do |_, i|
if i == 0 # <-- LEFT ENDPOINT
input[i+1]
elsif i == input.size - 1 # <-- RIGHT ENDPOINT
input[i-1]
else
input[i-1] + input[i+1] # <-- GENERAL CASE
end
end.to_a
Notice how we treat the endpoints as special cases, complicating the simple rule “sum both neighbors of every element.” A better approach is to first transform the input so the special cases vanish, leaving only the general case. Let’s pad the input with zeros, but loop over the original elements:
0 1 3 4 5 0 1 8 0 --- --- | | +-- 3 --+
Every item now has both a left and right neighbor. The special cases are gone, and the simple algorithm shines through.
input = [1, 3, 4, 5, 0, 1, 8]
padded = [0] + input + [0]
result = (1...padded.size-1).map do |i|
padded[i-1] + padded[i+1]
end.to_a
Another simple example
You start at the bottom (step A) of a 4 step staircase, labeled as follows:
____ | D ____| O | C |< ____| / \ | B ____| A
You take n
steps, turning around at the top and bottom. Where do you end up?
If you took 4 steps, you’d finish on step C – 3 steps to reach the top, turn around, 1 step down. Naively, you could simulate this stair pacing: iterate from 1
to n
, ping-ponging a current index inside an array.
But this is inefficient for large n
. Most programmers notice that every 6 steps you return to step A, so the “n
answer” must equal the “n % 6
answer” (the remainder when n
is divided by 6
).
They might solve it like this:
STEP_LABELS = 'ABCD'.chars
STAIRCASE_HEIGHT = STEP_LABELS.size - 1
def final_step(n)
n = n % (2 * STAIRCASE_HEIGHT)
index = if n <= STAIRCASE_HEIGHT
n # never turn around
else
2 * STAIRCASE_HEIGHT - n # reach top and turn around
end
STEP_LABELS[index]
end
This solution is fast, but still contains a needless special case: reaching the top and turning around. Instead, we can “complete the staircase,” viewing the problem like this:
____ | D | ____| |____ O | C C | |< ____| |____ / \ | B B | --> wrap ____| |____ around A
and walk in a single direction, forever. The “top and bottom” special cases disappear. Once again, the code is simpler:
STEP_LABELS = 'ABCD'.chars
CIRCULAR_LABELS = STEP_LABELS + (STEP_LABELS[1...-1]).reverse
def final_step(n)
n = n % CIRCULAR_LABELS.size
CIRCULAR_LABELS[n]
end
This example is prototypical: Both modular arithmetic and symmetry completion are common techniques when eliminating special cases.
Down with evens!
Say you’re given an array of arrays, and wish to eliminate even-sized arrays by slicing between the first and second element, like so:
# odd length, even, slice odd length,
# unchanged into two unchanged
# V V V
input = [ [1], [1, 2, 3, 4], [1, 2, 3] ]
desired_output = [ [1], [1], [2, 3, 4], [1, 2, 3] ]
A good first instinct is to use map
:
input = [ [1], [1, 2, 3, 4], [1, 2, 3] ]
input.map { |x| x.size.even? ? [[x.first], [x.drop(1)]] : x }
#=> [[1], [[1], [[2, 3, 4]]], [1, 2, 3]]
But the nesting isn’t right: we want the split to produce two elements, not an array containing two elements. You might give up here and use reduce
:
input = [ [1], [1, 2, 3, 4], [1, 2, 3] ]
result = input.reduce([]) do |m, x|
x.size.even? ? (m << [x.first] << x.drop(1)) : m << x
end
#=> [[1], [1], [2, 3, 4], [1, 2, 3]]
Which would be a shame, because conceptually map
is a better fit. Our map
attempt fails because it special cases the even elements, wrapping only those in an array. Instead, let’s wrap odd elements too, and then remove the wrapping from all elements:
input = [ [1], [1, 2, 3, 4], [1, 2, 3] ]
result = input.map do |x|
x.size.even? ? [[x.first], x.drop(1)] : [x]
end.flatten(1)
#=> [[1], [1], [2, 3, 4], [1, 2, 3]]
This trick takes many forms. Ultimately, avoiding special cases is about making things identical. When you can’t make the special case look general, see if you can make the general case look special.
A real world example
In a system I worked on, merchant hours of operation were stored in the database as bit arrays, and each day was broken into 24 * 12 = 288
5 minute intervals. Here, for simplicity, we’ll use 24 one hour intervals – it won’t change anything relevant.
If a store was open Monday 8am to 1pm, and then again from 2pm to 7pm, our Monday bit array would be:
12am 8am 1pm 7pm 11pm
v v v v v
0 0 0 0 0 0 0 0 1 1 1 1 1 0 1 1 1 1 1 0 0 0 0 0
The goal is to display these bit arrays in human friendly form on a web page, the way Yelp does:
Since 01
marks an open time and 10
marks a close time, you might try a solution like this, and think you’re done:
# NOTE: The utility methods `all_indexes(arr, subarr)`, which returns
# the indexes of all matches of subarr within arr, and `to_12h(hour)`,
# which converts 24 hour time to 12 hour time, are defined in the link
# below, and at the end of the article.
#
bits = '000000001111101111100000'.chars.map(&:to_i)
open_times = all_indexes(bits, [0, 1]).map(&:succ).map(&to_12h)
close_times = all_indexes(bits, [1, 0]).map(&:succ).map(&to_12h)
hours = open_times.zip(close_times)
puts hours.map{ |x| x.join(' - ') }.join("\n")
#
# Output:
#
# 8am - 1pm
# 2pm - 7pm
But what if a merchant – a club, say – opens at midnight? The 01
will be missing. Likewise, past-midnight closing times force you to look at the next day:
To display Friday’s hours, for example, we must look at Thursday and Saturday, so we can ignore Thursday’s late night overflow and correctly report Friday’s 2am closing time.
5pm 1am 5pm 1am 5pm
v v v v v
000000000000000001111111110000000000000001111111110000000000000001111111
\______________________/\______________________/\______________________/
Thurs Friday Saturday
Again, think how you’d approach this.
For simplicity, ignore the case where a merchant is open 24 hours straight, but handle all other possibilities. It’s tempting to make special cases for midnight openings and late night closings. But can we can avoid them?
Let’s take a full week schedule, and a merchant with both edge cases:
# A full week of daily hours of operation
# This is what would be returned by a database query
#
hours = [
'000000000000111000001111', # Mon: 12pm - 3pm, 8pm - 12am
'000000000000111000001111', # Tue: 12pm - 3pm, 8pm - 12am
'000000000000111000001111', # Wed: 12pm - 3pm, 8pm - 12am
'000000000000111000000000', # Thu: 12pm - 3pm
'111100000000000000000011', # Fri: 12am - 4am, 10pm - 4am
'111100000000000000000011', # Sat: 10pm - 4am
'111100000000000000000000' # Sun: Closed
].map { |h| h.chars.map(&:to_i) }
The crucial observation – the rule with no exceptions – is this: the time intervals we display on a given day are exactly those whose left endpoint is contained in that day.
Our algorithm, then, will be:
- Flatten the input so we don’t have to worry about individual days.
- Pad the input so we don’t have to worry about midnight or overflows.
- Zip all valid opening times with all valid closing times.
- Group the results by day, to recover the “day” view.
That’s it. Just a couple small adjustments to make our original idea work despite the special cases.
Taking all valid closing times works because ruby’s zip
method ignores extra items by default:
Using the hours
we defined above, we’ll combine the tricks we’ve learned so far: modular arithmetic to capture each day’s 24-hour cycle; “padding” the endpoints of the week to make it circular; and modular arithmetic again to handle 24 to 12 hour time conversions.
# make the hours circular, to avoid special casing Monday
# at midnight or late night Sunday overlowing into Monday
circular_hours = hours.last + hours.flatten + hours.first
# this makes two adjustments:
# 1. adds 1 to found indexes, because searching for '01'
# or '10' returns the index before the one we want
# 2. subtracts 24 to remove the extra "circular" Sunday
# we added to the beginning in "circular_hours"
def all_adjusted_indexes(arr, subarr)
all_indexes(arr, subarr).map { |x| x + 1 - 24 }
end
# all valid open times for a week
#
# this filters out our extra "padding" days
open_times = all_adjusted_indexes(circular_hours, [0,1])
.select { |x| (0...24*7).cover?(x) }
# all valid close times for a week
close_times = all_adjusted_indexes(circular_hours, [1,0])
.select { |x| x > open_times.first }
# the heart of algorithm
#
# we zip open and close times, group the "left endpoints"
# by day of week, and apply human friendly formatting
hours_by_day = open_times.zip(close_times)
.group_by { |x| (x[0] / 24).floor }
.transform_values { |x| x.map(&human_hours).join(', ') }
# all the rest is just printing our results
days = %w(Mon Tue Wed Thu Fri Sat Sun)
hours_of_operation = days.map.with_index do |day, i|
"#{day}: #{hours_by_day[i] || 'Closed'}"
end
puts hours_of_operation.join("\n")
See the whole thing in action here: Try it online!
Let’s recap what we’ve done. We’ve translated a problem that would require special cases – treating each day’s hours, and the overlaps between them, individually – to one in which all opening and closing times in a week are treated uniformly. In this uniform world, the solution to our problem became trivial: we just zipped together all the opening and closing times.
When we were done, we translated back to our original world of individual days using group_by
. This trick of translating a problem to a simpler “world”, solving it there, and translating it back is common in mathematics, and is related to the concept of duality.
Take Aways
Once you start looking for special cases, you’ll see them everywhere. Eliminating them isn’t a minor matter of tidiness, either. It will make your code easier to read, simpler to maintain, and, most crucially, less prone to bugs.
Think of it this way: To make non-trivial code reliable, there are two basic strategies. One is to meticulously enumerate and test all special cases. If you’ve missed any, or if any of your tests are incorrect or incomplete, your code will fail. The other is to recast your problem to make the special cases vanish. There are many fewer ways to fail with the latter strategy. And much less code to maintain.
It’s not always possible to simplify a problem by recasting it. But often it is, if only partially. So when you find yourself typing if
… stop, and pause for a moment. Look for another solution. It might be simpler than you were thinking.
Additional Notes
Here are the utility functions we used in the final merchant hours example:
# NOTE: Utility function definitions
# utility to find all indexes of subarr in arr
def all_indexes(arr, subarr)
arr.each_cons(subarr.size).map.with_index do |x, i|
x == subarr ? i : nil
end.compact
end
# utility to convert 24 hour time to 12 hour time
#
# using odd? generalizes this, so it works for
# 36 hours, 48 hours, etc
to_12h = ->(military) do
pm, h = military.divmod(12)
"#{h == 0 ? 12 : h}#{pm.odd? ? 'pm' : 'am'}"
end
# converts an array of 24h [open, close] intervals to:
# a hyphenated string range in 12h format
# eg: [12, 16] becomes "12pm - 4pm"
human_hours = ->(arr) { arr.map(&to_12h).join(' - ') }
Staff Writer
CyberArk uses a collection of staff writers and practitioners to support the DevOps Security.