Without effective communication principles, advanced statistics are useless. Some of my key lessons from the field.
The graphic above provides a breakdown of 2018 law school graduates with diverse race/ethnicity backgrounds. Each hand represents 100 JDs. The colors represent four different categories in the U.S. News law school rankings. Thus, the Tier 3/4 schools have the largest number of diverse race/ethnicity graduates—4,500 JDs, or about 45% of all diverse 2018 JD grads. Likewise, only 1,300, or 13%, attended elite T-14 schools, which is clear, useful information for legal employers who have urgency regarding diversity.
I would not have produced a graphic like this during my time as a political science professor because of its basic descriptive information and iconography. Where are the high-powered quantitative methods, the novel estimators, the measures of uncertainty?
That I post it proudly today reflects three lessons I’ve learned from presenting data to legal audiences: (a) simple analytics and visualizations are powerful because they are easy to understand; (b) quantitative methods and rigor are important, but the details belong in the background; and (c) it’s important to check the “quant jock” ego.
I didn’t have these views in my early days working with law firms. Like a lot of aspiring legal analysts, I learned how to think about, analyze, and present data-driven results from social scientists in the academy. And for reasons that will become clear, the standards and norms that breathe life into scientific analysis and writing are dead-on-arrival in law.
This post describes some of my observations about what works when presenting data to legal audiences. I like to follow three guiding principles, and I think my readers who work with data will benefit from incorporating them in their own work:
- Understand what data benefits legal professionals and lawyers.
- Default to conceptual illustrations and intuition.
- Less is more.
Working with law firms and legal departments, I’ve seen data presentations spark and propel progress on issues ranging from employee engagement and diversity to merger strategy and practice focus. In adopting principles that have worked for me in the past, I think aspiring analysts will gain the influence they need to drive systems that benefit their organizations and the profession.
Orienting yourself to legal audiences
Data communicated effectively can wield considerable influence over decisions in law departments and firms. Yet, the potential for missteps in preparing and presenting results are numerous and varied. Qualified analysts will start out doing what they were trained to do: First theorize about a process and generate predictions, then explain data and methods in depth, and finally report results and speculate about meaning—in that order every time. Analysts with research chops are taught to build suspense before unleashing knockout findings.
Yet as the entrepreneur Jerry Neumann noted in a recent tweet, effective practices in business writing are markedly different. “Get to the point very quickly,” Neumann writes, and “use fewer words.” No time for theory and hypotheses, reviews of past research, discussion of data and methods. This is probably five times more important in legal settings, for at least two reasons.
First, in law firms the billable hour reigns supreme. If you’ve managed to get yourself in the room with high-powered partners to discuss your analysis, they are literally paying to listen to you in six minute increments. This makes the environment very different from an academic setting. Professors seem to have considerable time for, um, conversation.
Second, as a friend who manages the Labor and Employment practice in a large global law firm recently pointed out, lengthy discussions about what you did with data raise suspicion in legal audiences. “Why’s it taking so long for Parker to explain what he did? What’s he trying to hide?” Rightly or wrongly, lengthy presentations about data beget lawyer cynicism.
You can’t influence people with data if you can’t gain their confidence. You can avoid a lot of potential pitfalls, however, by understanding or reminding yourself what sorts of data can benefit legal professionals and lawyers. (Preview: it’s a pretty low bar.)
Understand what data benefits legal professionals and lawyers
Law firms have flat management structures. For this reason they are well-suited to learning from data. It’s difficult to know what’s happening in a large global organization with diffuse power centers. Also, law firms generate and archive massive amounts of internal data. Analysts can use data to fill the information void effectively as long as they understand what their colleagues need to know to inform decisions. In time, you’ll recognize this is not necessarily what your colleagues want to know. And the need-to-know data is sometimes far more basic than what would pass muster in the academy (recall the graphic above).
My first experience using basic statistics to positive ends came soon after I began working with a large law firm on their Engagement Survey data. I was asked to produce averages and other summary statistics like standard deviations overall and for particular firm groups. I’d spent more than ten years working with data, running regressions, beefing up models with the latest, greatest estimators and technology. This was the first time, however, I had to manipulate data to create averages, sample sizes, and so on for lots of groups. And it was really hard for me! (As data science has gone mainstream, techniques for calculating and reporting such statistics have proliferated—see, e.g., the dplyr package in R.)
The other lesson learned from this project was the power of visualization for legal professionals and lawyers. Too often, they receive reams of data in clunky tables and spreadsheets. As an educational tool in large organizations, visualizations like those we produced for the Engagement Survey project were powerful. Figure 2 offers a stylized example based on a composite of law firm surveys. It reports the averages calculated across 10 distinct topic areas (1 to 5 scale, from least to most satisfied).
Ever the analyst, I produced and reported scores like those in Figure 2 and wanted to know more. Attorneys were more satisfied with Client Service and Community than Communications? In addition to observing some areas had high and others low scores, I wanted to know why!
In many legal settings, getting into the why is a bridge too far. To their credit, the aforementioned firm was willing to explore methods like regression analysis, and in time they understood how these results provided opportunities to focus on areas most likely to increase engagement at the firm.
It would not have worked, however, to present such results using scientific reporting practices. We needed the rigor of the regression approach and a presentation that was intuitive.
Default to conceptual illustrations and intuition
So, another lesson learned emerged from the need to report model-based results to non-experts. It was common practice in academic writing to report results in a table like the below. This snapshot is from Table 3 of a paper I published in 2013 on the dual roles of motivated reasoning and real-world information in economic perception. See Evan Parker, “Tides of Disagreement: How Reality Facilitates (and Inhibits) Partisan Public Opinion.” 75 J. Politics 1077 (2013). If you put results like this on a slide and present them to a legal audience, you’ll lose them quickly. Today I think you’d lose me too.
Table 3 might reflect standard practices, and the model itself was pretty advanced at the time. But none of that would matter to legal audiences, and rightly so. A table reporting estimates as modes, standard deviations, etc. creates massive confusion. Let’s be serious, the table references model “Deviance” at the bottom!
Lawyers and legal professionals understand the value of a regression-style analysis. To understand an outcome we care about, a regression lets us isolate factors and get a more robust sense of what matters and what doesn’t. The challenge lies in communicating the results in a way that is conceptual and intuitive. Everyone should have a reasonable opportunity to follow along on some level. For me, the way forward is to visualize the results of the statistical model and speak as conceptually as possible.
A regression analysis is a model for representing the world. The common form uses a straight line to represent a relationship between an outcome of interest, like law firm profitability, and potential explanatory factors, or “predictors.” The power of the model is that it allows us to derive what I’ll call “appropriate comparisons” that assign different weights to different predictors. The larger the weight, the more strongly the factor differentiates on the outcome in the positive or negative direction.
To make it concrete, let’s say we are interested in using a regression model to understand law firm profitability. The regression will help us generate accurate and consistent predictions about different firms’ expected profits when they have a certain attribute versus not. It accomplishes this by isolating the importance of predictive factors under “all else equal” conditions.
Figure 3 below shows how these results get reported in visual form. This orange line has proven very important for me as a means of orienting legal audiences. It reflects a very specific quantity: in this case, the expected profit for a law firm whose attributes are average on all the predictive factors.
I have had a lot of success getting people to understand the conceptual meaning of regression with reference to this orange line:
“Here’s the profit you’d expect from a typical law firm. Factors to the right coincide with higher profits, and factors to the left lower profits. So here, prestige, leverage, and geographic concentration are the biggest positive differentiators. Racial/ethnic diversity is also a strong differentiator.”
Another example of using intuitive illustrations came with work on using algorithms to predict successful lawyer hires. In a final sales meeting to close a deal with a large firm, a member of the Executive Committee asked me pointedly how he could get comfortable using a model developed from historical data. He was concerned that the business system that produced the data could have biases baked in. In my on-the-spot answer I fell back on my academic training, referencing model estimates, “controlling” for variables, making adjustments to predictions, and other jargon. I learned later this partner’s reaction to my answer went something like this: “I could tell Parker knew what he was talking about, but I just didn’t get it.”
He was correct, and I had failed to provide an explanation that was conceptual and intuitive. In response, we wrote a memo that used plain language and culminated with the visualizations in Figures 4a and 4b. The figures compare distributions of predictive scores generated by an algorithm that ignores the potential for historical bias based on race/ethnicity factors (4a) vs. one that adjusts for this likely possibility (4b). In 4a, the predictive scores for diverse race/ethnicity lawyers are consistently lower on average. In 4b, the asymmetry is reduced and the scores are statistically equivalent.
My colleague Kathleen Fredriksen said this graphic showed that our “de-biasing” approach could “move mountains.” Bad joke aside, the intuitive presentation was effective, and our ongoing work with this firm has been some of the most rewarding of my career.
Less is more
A third lesson comes from a principle that is at once more abstract and all-encompassing. When you’re working with data to help people solve problems, abiding by the phrase made famous by the renowned architect Mies van der Rohe in 1947—“less is more”—is a winning strategy, always.
I have my wonderful wife, a New York architect, to thank for educating me on the meaning and application of minimalism in architecture and design. A “less is more” philosophy turns out to be immensely valuable when preparing and presenting data, perhaps for the same reason Mies van der Rohe’s famous Farnsworth House (see image below), located in Northwest Illinois, is stunning in person. A less is more mindset forces creators to decide what’s most important and stick to it, like the Farnsworth House’s simple horizontal planes sitting in nature.
When working on data projects, the temptation to go big, showcase your knowledge, go full-on “quant jock” is always there. This won’t help your case to gain influence. From what I gather, lawyers are smart too!
I remind myself less is more, and good things happen. For one, I replace jargon-laden text and terms with plain language. As a recent study demonstrated, jargon-heavy text is not only bad for learning, it kills people’s interest in topics like science and politics (H/T John Grant).
I also end up reducing the use of ink in data visualizations. This increases what Edward Tufte calls the “data-to-ink ratio,” a feature that helps an audience focus on the data and facilitates interpretation. It’s worth pointing out that bar charts have very low data-to-ink ratios. I won’t comment on bar charts with “3-D” effects.
Less is more reminds me that, when I’m using complex statistical models, there’s no need to announce that fact. I work to convey the meaning in these models in the simplest terms possible. This gives the audience an opportunity to understand the findings intuitively. In my experience, lawyers are not shy about raising questions and probing more deeply.
The less is more philosophy is a strong close because it forces data analysts to make good decisions. Yet, there’s one more item to raise pertaining to this post’s title. It’s a riff on the title of the autobiography of Lenny Bruce, “How to Talk Dirty and Influence People.” Bruce was known for his willingness to call out societal conventions he viewed as hypocritical and wrong-headed, and he used his own sorts of data and commentary to make the case convincingly.
In an era where the legal profession is increasingly under pressure to adapt on several fronts—increasing diversity and equity, access to justice and more—it’s worth recognizing the role data can play in pushing past convention. If you buy what I’m selling, you can craft winning arguments and gain influence. So, let’s use data more persuasively to highlight inequities, force tough conversations, and be a force for change.
 One reason for this process is that science values reproducibility—others should be able to understand how you developed your thinking and performed your analysis so that they can accurately critique it and perhaps execute the same study.