Why Crossword Construction is Still Driven by Humans

This is an original crossword I wrote which hasn’t appeared anywhere else. It gets spoiled within the post, so try solving before reading on!

So why do you need humans at all?

In our technology-driven world where computers are doing more and more of the tedious tasks once relegated to humans (and doing them more quickly and, in many ways, much better than humans ever did), this is a question I get a lot. And, I get this question both about trading and about another passion of mine – constructing crossword puzzles. Both are endeavors that it’s easy to imagine a computer doing much better than a human. Technology has absolutely revolutionized both, but I believe humans are, and will continue to be, paramount to the highest level of performance.

The first step to writing (most) crossword puzzles is theme development. This is a step which clearly highlights why computers will (likely) never fully replace people in cruciverbalism. Take, for example, my theme from the sample puzzle (spoilers below!):

The theme here is BREXIT – four people whose names begin with BR are clued in reference to a famous exit they’ve made (BRAD PITT from his relationship with Jennifer Aniston, BRIAN JOHNSON from AC/DC, etc.). And since BREXIT is so-named because it was BRitain’s EXIT from the EU, each of these constitutes its own BREXIT. This isn’t the sort of idea that current – or even easily-imagined – software is capable of coming up with, and is representative of the kinds of creative re-imaginings of language that are very common in crosswords right now.

Once a theme has been fleshed out, the next step is designing and filling in the rest of the grid. This is the stage where computers, for the most part, do outshine humans. It’s pretty simple to write an algorithm which, given an incomplete grid and a list of words (as well as a score for each word denoting how “good” it is), can provide a number of filled grid options (and can also optimize for total score). Indeed, there’s a number of commercially-available pieces of software which do just that, and good scored word lists are out there as well. The advent of this software has made modern crosswords significantly better than their counterparts of even five or ten years ago. Obscure/illegitimate words with common letter sequences like ETUI and IIII and AEAEA no longer dominate grids because the algorithms can find fills which stick primarily to in-the-language words and phrases. Solving a crossword no longer feels like wading through a mire of foreign words and abbreviations, which can be scored appropriately low on constructors’ word lists (or removed altogether!), and that’s entirely due to the advent of software.

The best crosswords typically aren’t filled just by computers, though. Indeed, no word list contains every phrase which might be good to put into a crossword – a constructor’s own vocabulary and sense of what’s in-the-language will always be more complete. Take, for instance, the BREXIT puzzle. In particular, look at the entry FASHIONISTA at 12-Down. At the time I constructed the puzzle, FASHIONISTA did not appear in my word list, for whatever reason. I needed to fill the pattern ???????I??? (the I being fixed by a theme entry), and the fill that I was most satisfied by ended up being this one, dependent on an entry I put into the grid myself. Adding phrases like this which my word list (and likely other people’s, as well, since mine is in-part based on a database of entries used in previous puzzles) does not contain, represents one of the biggest human value-adds in filling grids. I also added the entries DIBS ON, FAV, and BAE manually – all examples of very modern slang which haven’t been used in mainstream puzzles before – as well as fun longer entries DON’T RUB IT IN and NAME GAME.

There’s also a certain level of “supervision” that I do when filling grids. After all, I have a better idea of what makes a good entry than the software, which only has a calculated score for each entry. I trust my decisions about entry quality much more than I trust the computer’s calculated judgments.

Indeed, my software presented the following as its best fill for the BREXIT puzzle, after fixing the theme entries in place (with the original for comparison):

This computer-generated grid strikes me as, overall, somewhat less lively, and filled with significantly more questionable entries than my eventual grid. This result is, in large part, because the software values entries like APIARIAN (19-Across) equally with ENDEAVOR (16-Across), DIBBER (48-Down) equally with DIBS ON (48-Down) – something, I think, very few solvers would agree with. This grid also doesn’t do a good enough job separating short entries like AYS, ERS, IRINA, ROO, SLR out as poor fill choices (though I’m always trying to improve – I lowered their scores immediately after I generated this!). When I’m filling a grid, I see the software’s fills like this, choose elements I like, and fix them (for instance, I might try fixing the nice BLADE II / LIVE A LIE / TEN PINS set), then re-fill around those entries, seeing if I can find something I like better. And I also try to plug in my own entries (like FASHIONISTA) and see if I can do better with those. My process is one of continual iteration, filled with interesting, challenging decisions.

Perhaps the biggest reason humans are still necessary in filling is the actual grid design (i.e. where to place black squares). Grid design is still an entirely manual process, and executing well on grid design can turn a mediocre grid into a great one (a simple example is not forcing yourself into spots where you need to fill the pattern ??K?V, for instance – but it gets much more subtle and more difficult than that).

Overall, I’d liken filling the grid to the actual day-to-day, computer-aided trading we do at SIG – we depend heavily on our models and algorithms to guide us, but we also place extremely high value on human decision making. There are lots of unique problems that are better solved by humans than computers. For instance, pattern recognition is key to both fitting entries into a grid and figuring out trends in a trading market, and it’s a skill that computers haven’t fully mastered and do not (yet) match humans in performing.

Once the grid has been created, the final step is writing clues. A computer can, of course, generate dry, dictionary definition-based clues for any entry. There also exist databases of clues for entries which have been used in previous puzzles. But fun, wordplay-based clues can’t be written by a computer, and neither can other less conventional clues – like those for ID TAG, ONION, and FASHIONISTA in the BREXIT puzzle. Here are some examples of wordplay-based clues I’ve written which aim to misdirect the solver:

• Org. imposing a travel ban?: ???

  Click to show answer

  NBA – this is a play on the typical term “travel ban,” and actually refers to the rule in basketball prohibiting traveling.
• Reason your company might fold?: ?????

  Click to show answer

  FUTON – this clue isn’t talking about a business, but rather houseguests, and “fold” refers to a guest needing to fold out a futon before sleeping on it.
• Member of the Browns who often misses the ball?: ???????

  Click to show answer

  CHARLIE – The Browns aren’t the football team here, but rather the cartoon family. This is a reference to the “Charlie Brown” gag where Lucy pulls the ball away as Charlie’s about to kick it.

Today, none of these could be generated by a computer – they require a person’s grasp of the rules of language (and how those rules can be strategically broken). A computer can recognize signals it’s specifically trained to look out for, but often new, unexpected situations develop – and that’s where human understanding comes in.

The constant innovation that exists in crossword construction (from this brilliant puzzle to new takes on what a crossword can be) signals to me that the role of humans in making quality crosswords – just as in successful trading – will never disappear.

Note: If you enjoyed this crossword, you might be also be interested in several puzzles I wrote for The New York Times here and here. A subscription to the crossword is needed to solve, but you can get a month free at the links.