First-gen matrix for evaluating software options. Harvard Law School, circa 1985.

Oh, the Humanity!  We can choose to choose better.

My first serious experience choosing law-related technology was in early 1985. Personal computers had just been introduced in the Harvard Law School clinics (as part of Project Pericles) and we had to decide which software to use for word processing. (WordPerfect was around, but we somehow missed it.) So I typed up a chart on an electric typewriter and added lines by pencil. See above graphic.  We wanted to be sure our choice did things like automatically centering text.

Such charts are familiar to product choosers everywhere. Options on one axis; features or considerations on the other. Ideally incorporating some sense of the relative importance of the latter. (One defect of the above chart is that there’s a “How desirable?” column for each option. Perceived importance of factors may vary across decision-makers, but shouldn’t differ by option.)

We ended up for a while with XyWrite, not on the list. And went through more elaborate exercises to choose case management, document management, and computer conferencing systems.

Putting choices on the table

Some years later I found myself in consulting roles, advising clients on software choices, like the below one for a different law school. Fortunately, word processors by then let you create nice-looking tables without the need for a pencil! And if you did this kind of thing in a digital file, rather than, say, on a whiteboard, it was less likely to be lost or erased between meetings.

Spreadsheets also come in handy, like this one we did as part of an effort to select a document assembly tool for the national legal aid community:

Spreadsheets of course make it easier to implement weights, thresholds, and automatically calculated scores and ranks. They are now among the most common tools used in procurement processes. You can even add sheets for different decision participants, in a quasi 3D mode.

My colleagues and I have gone on to help dozens of client organizations choose technologies (and other things). Along the way I went off the deep end around another approach, which involves interactive visualizations like the one to the left of this paragraph.

(For a narrated version of the above points, check out this 13-minute talk I gave at a gathering of legal knowledge management folks. And if you’re a glutton for punishment, you can read my Introduction to Choiceboxing.)

Too much information?

To the extent that decision-makers use digital tools, they are mostly familiar ones from the 80s. Email. Word, Excel, and their online cousins. Our processes also haven’t changed much. But at least we have more information than ever.

Choosers often face a blizzard of options and considerations. And it keeps snowing.

In the legal tech world, promising new value-adding directories of products and services have arisen, like Lawnext and the Legaltech Hub. There are category-specific ones, like Reynen Court (for cloud apps) and Catherine Bamford’s database of 250+ (!) document automation products. You can find and filter with abandon. Comparing and sorting isn’t as well supported. Nor is there yet much “social production” of actionable know-how.

This embarrassment of informational riches underscores how overwhelming choices can be, and not just for newbies. Even frequent fliers can be flummoxed by the quantities and complexities in play. Iterative rounds of deliberation, research, and conversation can peter out in decision fatigue, if not exhaustion. “Let’s get this over with” isn’t a productive attitude.

None of this is peculiar to legal tech. Similar behavior happens in most procurement contexts. And in many other decision contexts.

Methods and madness

Most things we do on a regular basis stimulate us to develop informal methods that are reasonably reliable and efficient.

Less so with choices. We seem to repeat the same old mistakes.

In my experience, we often choose erratically. We don’t see choosing as a “job.”  There’s a lot of improvisation; many strong opinions, some ill-founded. Our methods are at best artisanal, not systematic.

After having weathered through a few product selection processes you may start to notice patterns and anti-patterns. Some of us can’t help writing about them.

I came up with Twelve Mantras for Technology Decisions for the Technolawyer Newsletter and went on to write these other short pieces for Attorney at Work:

A second trio delved into one specific category:

I also enjoy learning about these things with students. This fall will see the 9th instance of my Suffolk Law School course on Decision Making and Choice Management.

Good techniques can de-complexify choices. Like eliminating possibilities that lack must-haves or feature must-not-haves before addressing the nice-to-haves. Reasons can be roughly quantified and made commensurable when they point in different directions. You can usefully separate how much you care about things from how well given options “do” on the things you care about. It helps to externalize, even “reify,” your thinking. Get out of your head.

Human, all too human

Choice making of course is not just a cognitive activity. It’s inherently emotional and social. Unalgorithmic.

Decisions almost always involve unspoken factors, conscious and unconscious incentives, and rationalizations. Our reasoning is “motivated.”

Choosing well is hard enough for individuals; it’s often even harder for groups. Deciders benefit from guidance, given the treacherous terrain.

One strategy is to supplement SMEs (subject matter experts) with CPAs (choice process advisers). They represent different disciplines, and the practitioners rarely overlap. But hiring a sherpa can be a luxury.

Another strategy is to build out a new social medium, like the “choice space” I’ve been proselytizing. A collaborative online world of interactive visualization for thinking together through the comparative goodness of options across multiple factors, perspectives, and tradeoffs. For the sake of better decisions, with less stress, rancor, and regret.

Many of us know more than any of us.

 *       *      *

Some tech projects ascend to greatness. Some crash and burn. See, e.g., “77% of Inhouse Lawyers Experience Failed Legal Tech Projects,” Artificial Lawyer, May 23, 2022.  Many bounce along, avoiding both disaster and triumph. The choices we make along the way make a difference.

Let’s choose to choose better.