The Difficult Problem Framework is a simple tool that requires continuous learning and objectivity. Part II of a two-part series.


The framework above was developed to solve very difficult problems related to organizational change, particularly those now facing the legal field. I realize the framework looks laughably simple. That said, it’s harder to apply than you might think.

Part I (056) summarized my Deliberative Leadership course at Indiana Law, which gave me the opportunity to learn about and reflect on these topics.  In Part II, I explain Difficult Problem Framework (DPF), starting first with the basic mechanics of how it works and then providing examples drawn from Deliberative Leadership and other materials.

One caveat: Initially, this post will seem more focused on decision-making than leadership.  The goal, however, is effective leadership that has a chance at solving difficult problems. As you will see, the leadership part is already within our grasp. Effectiveness, however, requires more of a set-up.

Box 1: accurate assessments and root causes

Box 1 is the space where someone — a leader / innovator / change agent — seeks to understand a difficult problem and identify its root causes. Thus, Box 1 is primarily about fact-gathering and reasoning. To do it right, we observe the problem, locate relevant data and research, ask questions, listen, reflect over a period of days/weeks/months/years, write out our analysis in a clear and ordered way, and remain on the lookout for disconfirming evidence that reveals faulty assumptions or conclusions. In this respect, feedback loops are especially valuable. Cf. Chris Arygris, “Teaching Smart People How to Learn,” Harv. Bus. Rev., May-June 1991 (discussing double-loop learning).

Box 1 has two potential failure points. First, our assessment of the present is inaccurate due to insufficient fact-gathering, faulty unstated assumptions, lack of rigor, or something else (1A failure). Second, after constructing what we believe is an accurate analysis, we never leave Box 1, as we believe the hard part, or important part, is done (1B failure).

Readers may laugh when I say that Box 1 activities are similar to writing a peer-reviewed academic article. As we all know, however, academic journals are filled with symposia articles on problems that remain unsolved. Obviously, something is missing. That’s a 1B failure, the most common type for those of us in the academic crowd.

In contrast, practicing lawyers band together into firms that place a heavy emphasis on revenue generation. This leaves little time to read, reflect, and understand difficult problems that are one or more steps removed from the immediate demands of client work.  Despite being liberated from timesheets, in-house lawyers are not much better. In both contexts, daily responsibilities thwart deep systems thinking. This dynamic keeps the entire profession stuck in a rut of 1A failure, as we are trying to solve our most systemic problems with after-hours resources.

The image below captures our dilemma (H/T Casey Flaherty).

 Box 2: change strategy

Box 2 is about effective change strategy. Here, the operant conditioning of law school gets in the way, as we spend most of our time learning how to construct arguments that could prevail in a court of law. If we do it well, our reward is law review membership and a high-paying job. Yet, it’s also a problem-solving approach based purely on legal authority.  That’s a big limitation.

Even if a client’s life or business problem might turn on question of law, most clients can’t afford to engage the wheels of justice. And even they can, few leave the courthouse feeling good about the experience. Thus, lawyers with large practices eventually build out a toolbox of nonlegal skills. In fact, Shultz & Zedeck documented 26 different tools. See “Identification, Development, and Validation of Predictors for Successful Lawyering,” LSAC Final Report, Sept. 2006.

If we move on to organizational problems — e.g., law firms, law schools, legal departments, court systems or regulators struggling to adapt to changing time — and we are paying any attention at all, we soon observe that stakeholders are seldom won over by reasoned arguments. In fact, they may not even show up for the meeting. If they do, their head may be elsewhere. For those who show up and listen, they’ll likely want to modify our ideas with some of their own. Suffice it to say, no one leaves these meetings with a quorum for change.

Several years ago, I learned this lesson the hard way, as I was part of a team building and selling evidence-based tools to lawyers. See Post 004 (discussing Lawyer Metrics); Post 016 (same). Straightforward presentation of data, even when connected to bottomline results, is not effective to win over a group of well-credentialed professionals. Cf. Daniel Kahneman, Thinking, Fast & Slow 227-29 (2011) (discussing hostility to algorithms, particularly when they demystify professional judgement). These experiences eventually drove me to diffusion theory and the insight that innovation adoption occurs through a social system where innovators and early adopters go first. When these two groups benefit, the rest of the social system follows. See Post 004 (presenting theory); Post 007 (providing detail).

Diffusion theory, however, is but one of many theories and frameworks that can improve our odds of desirable change. I’ll give a few examples below, many of which are not only connected with leadership, but also meaning, purpose and fairness. But for now, the core point is that Box 2 requires us to continuously learn new ideas and reflect on how they might connect to our difficult problems. This is no less time-consuming than the Box 1 analysis.

Difficult problems and decisionmaking

To summarize: We can only solve a difficult problem if we can accurately assess its root causes. This requires a major investment of time and resources to get right (Box 1). Thereafter, we need to formulate an effective change strategy that goes well beyond explaining/publishing our analysis (Box 2). The purpose of the DPF is to keep these two activities separate and analytically distinct.


Correct root causes + right change strategy = chance at success

Going a bit deeper, solving difficult problems is a thinking person’s game where the biggest risk factors are (a) self-deception that causes us to underinvest in learning, fact-gathering and reflection, and (b) bias and distortion in how we evaluate information. Through work ethic and mental training, we can mitigate these risk factors, but never completely. On this score, I’d recommend four “applied” resources: Charlie Munger, 24 Cause of Human Misjudgment (1995) (75-minute audio); Daniel Kahneman, Thinking, Fast & Slow (2011); Randall Kiser, How Leading Lawyers Think (2011); Ray Dalio, Principles: Life and Work (2017).

Let’s now move from the abstract to the concrete.

Box 1: what are my assumptions?

The example below is based on an assignment in Week 1 of my Deliberative Leadership class.

Imagine you are an executive at General Motors in 1984. For reasons of cost and quality, the company has been losing marketshare to the Japanese. You’ve given this a lot of thought and concluded that the root cause of GM’s woes is an old, expensive and undisciplined workforce protected by overly generous union contracts.  Until that gets solved, the company cannot effectively compete with companies like Toyota, GM’s most formidable Japanese competitor.

This problem set is based on New United Motor Manufacturing, Inc. (NUMMI), which was a joint venture between GM and Toyota launched in the mid-1980s at an old GM auto plant in Fremont, California. Basically, this came about because Toyota was making great inroads in the U.S. market. To preempt a protectionist backlash, Toyota needed a plan to shift some of its production to the U.S. and learn how to adapted to U.S. workers. For GM, it was an opportunity to learn Toyota’s lean production methods, which combined world-class quality with world-class efficiency. This story is expertly told in a 1-hour podcast produced by This American Life. See NUMMI 2015, episode 561 (July 17, 2015).

The first part of the episode details the problems that existed at the GM Fremont plant prior to its closure in 1983 — pretty much non-stop drinking, drug use, absenteeism, and antagonism toward management.  According to Bruce Lee, who ran the western region for the UAW, “It was considered the worst workforce in the automobile industry in the United States. And it was a reputation that was well earned. Everything was a fight. They spent more time on grievances and on things like that than they did on producing cars. They had strikes all the time. It was just chaos constantly.”

In negotiating the re-opening of the Fremont for the NUMMI joint-venture, the UAW demanded that GM and Toyota rehire a substantial portion of the old Fremont facility workforce (it would turn out to be 85%).  Remarkably, Toyota was willing to go along.  According to NPR automotive correspondent, Frank Langfitt, ” Toyota execs believed their system would turn bad workers into good ones.”

The rest of the episode tells the story of how, under the Toyota production system, the Fremont facility went on produce world-class quality on par with the rest of the Toyota system.  What changed everything was the inclusion of workers in a team-based process of continuous improvement. For the first time on their careers, these old, tired workers were asked for their ideas on how the cars could be made better and more efficiently.


1A Failure

NUMMI is a vivid example of a 1A failure. The root cause of the problem was not the people; it was the system. For me, this lesson hit close to home because during the early 1980s, I was a college student living in Cleveland, Ohio. Pretty much the entire region blamed the workers and union for the decline of the industry.

The lessons of NUMMI are supported by others materials assigned during Weeks 1 and 2, such as Batman, This American Life, Episode 544 (Jan. 9, 2015) (expectations in our head have a profound effect on the physical and social world); Viktor Frankl, Why Believe in Others, Ted.com (video) (Jewish-Austrian neurologist who survived Nazi concentration camp and wrote Man’s Search for Meaning exhorting group of Americans to elevate their expectations of others and thus enable them to reach their full potential); Carl F. Braun, Management and Leadership (1948) (leader of C.F. Braun & Co., an international engineering and construction company, outlining the principles of human respect, dignity, and collaboration that underlie the company’s financial and technical success).

As these excerpts suggest, perhaps the root causes of organizational and institutional malaise are not exclusively gaps in logic or analytical rigor. Rather, a major root cause could be lack of clarity around purpose and, until that gets resolved, worry over status, hierarchy, and security.

Box 2: the missing link

I’ll admit that it wasn’t until that fourth year of Deliberative Leadership that I realized that there was a second box.  The turning point was this spring when Alli Gerkman, Director of Educating Tomorrow’s Lawyers, visited by class.  One of her selected readings was an article on an internal study by Google’s People Analytics group, dubbed Project Aristotle, that attempted to identify the attributes of high performing teams. See Charles Duhigg, “What Google Learned From Its Quest to Build the Perfect Team,” N.Y. Times Magazine, Feb. 25, 2016.

Google had long observed wide variations in team performance.  If it could isolate the factors consistently associated with high performance, perhaps they could be scaled across the entire organization.  Yet there were many false starts. In particular, Project Aristotle invested a lot of time and resources looking at how team compositions based on personality, skills or background affected team performance.  “No matter how researchers arranged the data,” wrote Duhigg, “it was almost impossible to find patterns — or any evidence that the composition of a team made any difference.”

Eventually this led the Aristotle team to the social science on group norms. One line of this research suggested that norms within groups may produce a “collective IQ” that is distinct from the intelligence of any single team member.  This hypothesis proved to be the missing link in Google’s research.

So, what is the cultural factor that explains high-performing teams at Google?  Psychological safety.


“Google’s data indicated that psychological safety, more than anything else, was critical to making a team work.”

According to Professor Amy Edmondson of Harvard Business School, who conducted much of the group norm research relied upon by Project Aristotle, psychological safety is a ‘‘shared belief held by members of a team that the team is safe for interpersonal risk-taking. … [and] a sense of confidence that the team will not embarrass, reject or punish someone for speaking up.” Edmundson, “Psychological Safety and Learning Behavior in Work Teams”, 44 Admin. Sci. Quarterly 350-383 (Dec. 1999). From the outside, a psychologically safe group might appear free-flowing and chaotic. Yet, because of group norms, the members are very good at allocating airtime equally and truly listening to one another.

This past spring, I gave two talks on leadership, one to group of young lawyers and another to a group of students from several law schools. In both talks, I explained the Google research, presented the definition of psychological safety, and asked audience members to anonymously complete a notecard that said whether their workplace or law school was psychologically safe (“yes”, “no”, “something in between”).  I then collected passed along the basket of notecards, letting each person draw a random card. Finally, I polled the results by having audience members raise their hands.  In both cases, less than 1/3 reported feeling psychologically safe. That’s a problem.

Connecting it together

For all four years of Deliberative Leadership, I have assigned a well-known article on authentic leadership. See Bill George, et al., “Discovering Your Authentic Leadership,” Harv. Bus. Rev. (Feb. 2007).  It’s an attractive thesis — that the most effective leaders “demonstrate a passion for their purpose, practice their values consistently, and lead with their hearts as well as their heads.”  Yet, the Google article got me to think that perhaps the authentic leader’s effectiveness flows from the group norms they foster, especially psychological safety.

One of the repeat readings in my class (picked by more than one guest lawyer) is True North: Discover Your Authentic Leadership, which is basically the book version of the HBR article.  It’s primary author is Bill George, former CEO of Medtronics who now teaches at Harvard Business School.  True North was picked again this year, in close proximity to Alli’s article. Thus, I took the opportunity to to better understand the book’s methodology.

Below is a graph from True North that, the authors claim, fits the pattern of many of the 125 leaders in the study.

Often, according to the authors, authentic leaders are forged during a period of extreme hardship. Through the “crucible,” leaders finally develop the courage and confidence to live by their own values.  Perhaps one way to establish a psychologically safe workplace is for the boss to explain difficult decisions in the context of their own learning, including painful failures and setbacks.  I’ve had my own crucible moments in life. At age 55, I can say that crucibles really do burn away our allegiance to things that are stupid and really don’t matter.

In terms of Box 2 change strategy, the Google research and Bill George’s authentic leadership are connected together with the work of Chris Arygris, the late, great HBS professor who focused on organizational behavior, organizational learning, and change management. In a 1991 article in the Harvard Business Review that was later republished as a HBR Classic, see “Teaching Smart People How to Learn,” Arygris discusses his work with elite management consultants. The primary theme is engrained defensive reasoning that keeps very smart professionals from learning why many of their engagements continually fall short of desired results for them and their clients.  Arygris reports many heroic efforts to change this dynamic that all end in failure.

Arygris then relates the story of a CEO of a large organizational-development firm who was so disgusted with the pattern that instead of preparing for an upcoming meeting, he decided to script out the failure in advance.  He divided his work into two columns. On the right side, he wrote out the likely dialogue that would take place.  On the left side, he wrote the thoughts and feelings that he would likely have during the meeting “but that he wouldn’t express for fear they would derail the discussion.”  Then, instead of having the meeting, he used the time to analyze the scenario with his direct reports.

What happened next was an honest dialogue in which the CEO became privy to the honest but unspoken views of his entire team. Then they could finally hear him with a new set of ears. Finally, real progress could occur.

Below is a stylized version of Aygris’s recommended approach, which I use in my Deliberative Leadership class:

This is a potentially useful Box 2 tool. Do you have the courage to give it a try?

As legal organizations and institutions struggle mightily to adapt to rapidly changing times, there is renewed and growing interest in the topic of leadership. I am confident great things are going to happen as a result.


This is a two-part series on leadership.  For lawyers and legal educators, the big test is now.


The first time I heard “smooth seas make poor sailors” was from Fred Bartlit, one of the founding partners of Bartlit Beck.  I thought Fred was providing a guidance on how to become a great trial lawyer, i.e., through experience.  But Fred corrected me and said he was making a larger point.  Fred had been a U.S. Army Ranger and had led a platoon of soldiers in the early days of Vietnam.  He was talking about the value of perspective, emotional control, making choices with consequences, and filtering out noise. His Army experience had given Fred a very valuable general tool that could be applied to anything, including a career in law.

That conversation took place a decade ago when the legal profession and legal education were still riding high.  After the financial crisis in 2008-09, bleak job numbers and high debt loads gave rise to the scam blog movement followed by relentless negative coverage in the New York Times and Wall Street Journal.  With so much bad press and a weak entry-level job market, applications went into a free-fall. In the fall of 2012, Brian Tamanaha published Failing Law Schoolsfollowed by Steve Harper’s The Lawyer Bubble in the spring of 2013.

Law professors and law school deans were unprepared for the depth and magnitude of the change. Moreover, there was evidence that things might get worst, as lawyers were now discussing the likelihood of a permanent market shift in how law was being practiced. Through decades of prosperity and growth, we had been conditioned to believe that an endurable normal would eventually return.  But what if that wasn’t true? How would we know? Could the old guard be counted upon to make the call? If not, what then?

These questions were very much on my mind. Thus, in the fall of 2014, I convened a small, diverse group of Indiana Law alumni to discuss the topic of leadership. There was broad consensus that the legal profession/industry was entering a period of transformation and that these challenges were a microcosm of broader issues affecting our social, economic, and political institutions. I repeated the quote I heard from Bartlit and asked, “Where will the leaders come from?”

I also asked the group for their help in creating a course on leadership at Indiana Law. Everyone agreed to pitch in, but they scuttled the proposed name. “How about ‘Deliberative Leadership?,'” offered one seasoned alum who was CEO of a large company. “Before anyone agrees to lead,” he explained, “they should reflect on leadership in a deep and deliberate way.”  That seemed like advice designed to win over a group of lawyers. And it did.

You know more about leadership than you think

During the course of those meetings with alumni, I conducted an exercise that mimicked the only formal leadership training I ever had. The exercise asks two sets of questions:

  1. Identify a person who has had a major positive influence on your life. What did you learn from them? How did you learn it (e.g., through words or behaviors or some combination)?
  2. Identify a leader from your past who you decided to follow. Why did you decide to follow them? What were their sources of authority (based on job title, work experience, moral character)?

The purpose of this exercise is to surface the fact that we already possess keen insights on leadership by virtue of our life experience.  In modern times, leaders don’t have power or influence without the benefit of followers. Thus, who have we decided to follow? Invariably, the answer has a strong overlap with who has had a positive influence on our lives.

As an acknowledgement of the alumni who participated in my working group, I wrote up the results and shared it with them. See Summary of Leadership Exercise Conducted with Indiana Law Alumni, Indianapolis, IN, Oct. 23, 2014. I continue to use this exercise. And each year, I get essentially the same results.

Deliberative Leadership at Indiana Law

After I wrote up a detailed course proposal, I submitted it to the Indiana Law’s Educational Policy Committee.  On the eve the committee vote, I asked my colleague, the committee chair, if he thought the course was a good idea. He said, “Over at the business school, they have faculty who are experts on this topic.” After a slight pause that gently pointed to my lack of qualifications, my colleague commented on the thoroughness of the proposal and the customary deference given to tenured faculty. “Let’s see how it goes.”

One element of a high quality course is the quality of teaching.  A second, more subtle element is course design — i.e., how the classroom is run and the mechanisms for student learning.  In this case, I believed it was crucial to avoid the familiar role of professor as subject matter expert.  I didn’t know very much about leadership, and my students knew it.  But I had sincere curiosity and a few ideas on how to make the class work courtesy of the alumni working group and a few other resources.

Designing the course

Consulting with practicing lawyers, including former students, in your course design can be both a rewarding and humbling experience.

Perhaps the most humbling was an observation make by a former student who was the youngest member of the alumni group. She had just started a clerkship on the Indiana Supreme Court.  Prior to law school, she was a grade school teacher through Teach for America.  In reviewing my detailed course proposal, my former student remarked, “have your thought about learning objectives?”  After I wrote them up, she edited them, making them active and specific.

Another source of valuable input came from a former student who was still at the law school during his summer bar prep. Over lunch with a group of fellow grads, I explained that I was worried that students in my new class would sit back and become spectators. That very act, I said, would undermine the entire endeavor.

My former student replied, “why don’t you use a Harkness diagram.”  Having no idea what he was talking about it, he told the story of how, as a poor kid from Chicago, he ended up at the elite Phillip Exeter Academy in New Hampshire.  He also explained that most of the pedagogy at Exeter requires students to “lead the class.” The instructor’s role is to formulate the reading assignments and track the discussion as it takes place around a oval-shaped Harkness table.  Below is an example of a Harkness diagram:

In tracking the class discussion, the balance of airtime is immediately apparent. My student told me that the Exeter format had forced him into critical thinking at a very young age and that nothing in his law school experience had come close to a similar level of classroom engagement.  Suffice it to say, I adopted the method.

Students in charge

As I continued to ruminate on how to design this new leadership class, I was struck by my student’s comment that students at Exeter “lead the class.” This echoed one of the observations in the Carnegie Foundation’s Educating Lawyers report that law school stunts student development by elongating the process of passive classroom learning and delaying the act of applying the knowledge in context.

Reflecting upon these insights, I decided that a cornerstone of Deliberative Leadership would be student-led classes.  Students would be divided into five teams of four.  After the first two weeks, which I would lead (and fully exhaust my then-limited knowledge of leadership), the student teams would be in charge. Each team would be responsible for two classes. One based on readings selected by lawyers I would invite to class.  And a second based on a topic and readings selected by each team.

This course design has many benefits and very few downsides. Among the benefits is the use of crowdsourcing to identify leadership materials worthy of inclusion in class.  Thanks in substantial part to the many lawyers who have been invited to class, I have built up quite a library of articles on leadership and, much to my surprise, a handful of especially valuable resources on followership. (Gary LeClair of Post 053 was a guest in 2015. His handwritten annotations on a followership article left a big impression on students.  In 2018, an alum of the 2015 class returned and quoted LeClair: “Don’t manage your time; manage your energy.”)

A second benefit of the student-led crowdsourcing method is the opportunity to observe patterns. When the same resources get selected over multiple years, or when the same themes get drawn from disparate readings, some of the best working tools are revealed.

A third benefit is to get inside the heads of my students, who are now typically more than a quarter century my junior. Over the years, topics selected multiple times include overcoming fear (invariably one of the best classes), stress management, creativity and innovation, diversity in the profession, work-life balance, saying no, and the impact of the billable hour culture. How can I influence what I don’t understand?  When I want to learn about my students — a very powerful future demographic — all I need to do is show up and listen.

Assessment

This post is not long enough to fully explain the course’s assessment system. If you’re curious, see Deliberative Leadership syllabus.  However, there is one fairly unusual assessment method that consistently advances the course’s learning objectives.

The class meets once a week for two hours over the course of a 13-week semester.  Starting with Class 1, I circulate a half-sheet assessment rubric that is loosely based on the “hotwash” debriefing method I learned from Jeff Carr, a well-known general counsel who no shortage of opinions on leadership (albeit backed up by a track record of impressive department results). The rubric is pictured to the right [click to enlarge].

For the first two weeks, the student are grading me, albeit with useful formative feedback that I can reflect on and apply in the future. The complete feedback is collated and posted for everyone to see.  Starting in week 3, students are assessing the class organized by the student teams. This is designed to feed the “double-loop learning” method pioneered by Chris Arygris and Donald Schon.  Double-loop learning is the road to practice mastery. And, as I demonstrate to my students, it can be retooled for leadership.

It is somewhat comical and sobering to see the initial reluctance of students to read and digest feedback on how peers perceive them and others. Students are reluctant because processing feedback is difficult emotional labor. It can also be rationalized away as a “soft skill” that can be put off to a day that never comes.  Yet all day long, the perceptions of others is what determines our fate, including our fitness to lead.  Through this iterative process, I am trying to show my students that their future success is largely within their own control.  The major limitation is not intelligence, which they in abundance.  It’s a willingness to continuously observe and learn. Cf. Kiser, How Leading Lawyers Think ch. 8 (2011) (discussion of “perpetual learning,” often through feedback loops, as key to trial lawyers who consistently outperform their peers).

Call to Action speeches

The last week of class, students are required to deliver five-minute “call to action” speeches on any topic of their choosing.  The speech has to be written out in advance.  Five minutes is roughly 750 words.

This is a course requirement that almost didn’t make the cut, as some of the younger members of the alumni group voiced concern that a speech in front of peers would be a source of major student stress.  Yet, that objection was shot down by a mid-career female partner at a large law firm who remarked, “Last week, I had to give a speech to my fellow partners on the need of the firm to put substantial resources into its diversity efforts. I can tell you, I wish someone back in law school had forced me to give a call to action speech.”

In the four years I have been teaching Deliberative Leadership, one of the most startling aspects has been the evolution of each class into a community of professionals who have learned to respect and trust one another, even when they differ widely on important issues. Prior to the class, the students tend to have opinions of one another, albeit developed from a distance. But after listening to their peers over the course of 12 weeks, they learn that their fellow students are much deeper and more interesting than they ever imagined.  Thus, during the Call to Action speeches, the room is filled with energy, as students root for each other.

Although it was not my intention, Deliberative Leadership may have become one of the few law school classes where someone’s earnest pre-law personal statement can be taken out and re-read as something real, vital, and important. Occasionally I get lucky. That was the case here.

Part II

Through four years of Deliberative Leadership combined with additional life experience, I have developed a framework to aid leaders in solving very difficult problems — the kind that now confront legal education and the legal profession.  I will discuss that framework next week in Part II.


The legal industry wants more innovation. The missing ingredient is strong leadership.


Several years ago, a good friend threw me to the lions, though that was not his intent.

My friend, who works in legaltech, asked me to show up at the headquarters of a Fortune 100 company to present some prototypes I had developed on giving feedback to law firms.  Cost pressures were rolling downhill to the legal department.  Thus, in an effort to better manage costs, the senior leadership winnowed their outside law firms to a panel of preferred providers.  In theory, the firms were supposed to work cooperatively with each other to deliver world-class quality within a large predefined budget.

From a distance, this all sounded innovative. But up close, implementation was a challenge. The only management tool was an annual rating system that measured law firms on a 1 to 5 scale (1 = poor, 5 = excellent). Because performance was aggregated across dozens of lawyers and dozens of matters, the narrative comments were too general and lacking in context to be helpful. Further, all the quantitative scores were clustered in the 4.8 to 4.9 range, making them useless for making merit-based adjustments.  Indeed, if in-house lawyers gave scores any lower, they’d be tacitly admitting a problem with their own oversight.

I had approximately 90 minutes to present my prototype to a room full of BigLaw relationship partners.  Basically, my proposal was to have in-house counsel complete a monthly survey tool for each significant matter they were managing (a 10 to 30-minute commitment per lawyer who managed outside counsel). In turn, the results would roll up to a centralized knowledge management system that would generate practice group, firm, and legal department-level reports.

Although the proposed prototype required the in-house lawyers to do all the work to generate the feedback, the law firm partners disliked everything they heard, arguing that the work to review the feedback would be burdensome and counterproductive. One especially vociferous partner remarked, “If there’s a problem, I’d rather have a phone call.” He would not concede that there was any value to timely bucketing specific examples of good and bad behaviors, nor that the resulting data could provide a roadmap to help the client and create a factual basis for higher fees.

As I was getting pummeled by the BigLaw partners, the in-house lawyers looked on in silence.  And in hindsight, I really don’t blame them.  They, like me, were learning the depth of the opposition to systematic measurement of performance.  It would have been a different dynamic if the general counsel, who operated at a level above these lawyers and was not supervising this initiative, had communicated that the company was going to use a feedback system to better manage millions in legal spend and that the purpose of this meeting was not to question the premise, but collaborate on implementation.

At this juncture in my career, I had not witnessed many examples of strong and decisive leadership among lawyers and thus did not appreciate how essential it was to organizational progress.  Over the next several years, however, I began to see the pattern.

Who should run the feedback process?

A few years later, in December of 2014, I spent the afternoon with two law firm insiders who were in charge of strategic initiatives at their respective firms.  Both believed in the importance of client feedback to not only enhance the quality of service but also deepen relationships with clients and build a path to more meaningful and sustainable growth.  Yet, they expressed frustration at its limited value to drive firm-wide or industry-level change.

Here’s why.  Imagine a large corporate client that uses 20 outside law firms.  In most cases, that means that there are nearly 20 different ways that the client provides feedback. One firm sends the managing partner for an annual dinner with the general counsel. Another sends the relationship partner. A third sends the Chief Value Officer. A fourth has an annual client survey system, albeit only 30% of the in-house lawyers reply. Several other firms use a third-party service, such as Acritas, Wicker Park Group, BTI, or PP&C Consulting.  And a surprising number of firms are content with feedback in the form of paid bills and continued work.

Virtually all of these feedback mechanisms are fragmented and lacking in context, making it easy for lawyers to rationalize away negative information. Under the best case scenario, only 20-30% of the total feedback time will result in significantly better performance.  That means that 70-80% of feedback has zero ROI. That’s an enormous amount of waste.

Yet, what if clients took control of the feedback process? As my colleagues pointed out, if clients rigorously evaluated their outside counsel, the information would be too direct and specific to be ignored. Then we laughed at our Panglossian idea, “This is never going to happen.”

Sometimes it’s good to be wrong

One of my law firm friends in the December 2014 meeting was John Fernandez, who was at the time was the US Chief Innovation Officer at Dentons (now Global Chief Innovation Officer).  One of John’s projects was the launching of NextLaw Ventures and NextLaw Labs, which identified promising new legal technologies for investment and piloting within the firm.

In June of 2015, John fielded an inquiry from a corporate GC who had, over the course of eight years and two different companies, developed a feedback system for managing his outside law firms.  The general counsel, Mark Smolik of DHL Supply Chain Americas, was looking for guidance on whether this idea had commercial application. John asked if I wanted to join a meeting with Mark to help vet the opportunity for NextLaw.  I said “sure.”

That meeting was very fateful because (a) John and I had already identified that this was a problem worth solving, and (b) Mark Smolik had years worth of data showing that his system worked.   Miscommunication and derailments were going down, value per dollar spent was going up, and Mark had more bandwidth to focus on other company priorities.

Borrowing from HR

I think readers will benefit from understanding the origins of Smolik’s system, as it reveals the power of simple ideas and insights.

The first insight occurred to Mark over a decade ago when he was general counsel of Safelite Auto Glass, a national company doing on-site windshield repair.  In addition to running the legal function, Mark was also in charge of HR. One day, Mark became a Safelite customer when the windshield on his wife’s car got damaged.  While at work, Mark took a call from his wife, who told him that a somewhat frightening looking guy claiming to be with Safelite showed up at the house to repair the damage  “I have no idea who this person is. Why should I open the door?” Wanting to reassure his wife, Mark contacted the Columbus service center and asked them to send their best technician to perform the work. “Please tell me his name and at least what he looks like.”

That incident gave Mark an opportunity to experience Safelite through the eyes of the customer.  Shortly thereafter, Safelite developed a standard practice of sending a technician profile email to all its mobile customers that included name, photo and credentials of the auto glass technician.  Safelite also implemented a client feedback tool to track the quality of each service call.  By the time Smolik left Safelite in 2009 (two years after its successful sale to Belron), Safelite was planning a national ad campaign that would make the quality and friendliness of their glass technicians the centerpiece of the company’s branding.

The systematic tracking of the customer-facing personnel at Safelite created a desire in Mark to apply the same logic to the many law firms that he was managing.  “If the company is going to spend a few hours each year reviewing the performance of each of its employees, then why aren’t we devoting at least that much attention to the large sums we spend on law firms?”

Thus, Mark applied basic HR principles to his outside counsel, developing performance criteria, applying it to firms, sharing results, and collaborating on a plan for improvement.  Mark used this methodology to winnow and consolidate the number of firms he worked with. This reduced his overall communication overhead while increasing the value of each dollar Safelite spent on legal.

Building a company around scorecards

By the spring of 2016, Mark Smolik’s outside counsel scorecarding system became the basis for Qualmet, one of the first companies in NextLaw Ventures investment portfolio.

Along with John Fernandez, the other law firm insider at my December 2014 meeting was Jim Beckett, who at the time was Chief Business Development Officer at Frost Brown Todd.

Beckett started his legal career as a Frost Brown Todd associate before going in-house at RJ Reynolds. A few years later, he moved to the business side, running an RJ Reynolds operating unit in Puerto Rico.  Jim came back to the firm partially because it enabled him to raise his family in his hometown of Louisville.  But having spent eight years inside a large company, he felt he had a roadmap in his head for how a law firm could grow market share. Jim and the firm’s chairman, John Crockett, had worked together when Jim was an associate and John was a young partner.  John wanted to give Jim’s ideas a try.

Jim’s business development strategy was very simple.  Spend time with your clients and listen to what’s on their mind.  Then make their problems your problems, using all your creative energies to identify, anticipate, and solve what’s happening in their world. This may sound obvious, but many lawyers struggle to get out of their comfort zone and then blame the lack of immediate returns on client resistance.

At his core, Jim is an impatient person who wants to change the industry.  Thus, in December of 2014, when we discussed the possibility of the client owning the feedback process, Jim couldn’t get it out of his head.  By the time Fernandez and I met with Smolik, Jim was sketching out a business plan.  Thus, during the June 2015 meeting, I told Mark, “There is a guy, Jim Beckett, who you’ll want to talk to. He has been on both the buy and sell side and is already fixated on this idea.”  John nodded in agreement, “I can’t think of a better guy to run with this.”  After several months of additional vetting, Qualmet was formed and Jim was named CEO.

CEO in legaltech may sound glamorous, but in reality it’s just more stress, a pay cut,  a chaotic mix of product, marketing, and sales, 6 am flights, bad airport food, and guilt over how your career decision is affecting your family. But if you think this is your big opportunity to make a difference, you’re willing to pay that price.


Disclosure: Through NextLaw Lab, I gave input to Qualmet during its formation, including sitting on its Board. Qualmet also became a client of Lawyer Metrics, where I served as Chief Strategy Officer.   When I left Lawyer Metrics in late 2016, and before I started Legal Evolution, I resigned from Qualmet’s Board, as I viewed fiduciary obligations to any legal industry business as incompatible with my role as editor. In addition, I have no financial or investment interest in Qualmet or any legal industry company.


We’re entering the management age for lawyers

Leadership and management are not part of the legal education canon.  Yet, that is bound to change as more lawyers stumble forward into these disciplines to cope with the relentless growth in complexity we face on a daily basis. In the meantime, however, we are at risk for misinterpreting the tides of change.

For example, many lawyers and law firms (and initially this professor) are quick to conclude that the goal of scorecards is to save money.  Yet, in most cases, the motivation is scarcity of internal bandwidth. An important task done well and efficiently frees up time and mental energy to tackle other strategic priorities. Saving money, or getting more value per dollar spent, is a by-product of a more disciplined approach to one’s job as lawyer-manager.

The first step in this more disciplined approach is formulating the evaluation criteria.  Initially at Safelite and DHL, Mark Smolik focused on seven criteria:  (1) understands our objectives / expectations, (2) expertise, (3) responsiveness / communications, (4) efficiency / process management, (5) cost / budgeting skill, (6) results delivered / execution, and (7) compatibility with company values.  Each criteria, in turn, is defined by a set of specific behaviors.

What managing law firms looks like

For ideas like scorecards, lawyers need examples rather than abstract descriptions. In 2016, I ran some focus groups for what would later become Qualmet. Below are some of the graphics from those sessions (credit: Evan Parker from LawyerMetrix).

These data reflect the performance of actual law firms, including the AmLaw 200 firm of Conroy & Alexander (a pseudonym). The scores for each criterion are averages of in-house lawyers who used the firm. Obviously, between 2011 and 2015, things moved in the right direction. Conroy & Alexander now exceeds expectations on six of seven criteria and has a clear priority on where it needs to improve.

Below is the trendline of Conroy & Alexander’s average annual performance. This is the ROI that flows back to the in-house lawyers who are providing the feedback — they’re expending less time and attention to get better results.

Below is a picture of how the top seven firms are doing. Conroy & Alexander is firm E.

One takeaway is that expertise — which lawyers routinely fall back on to sell themselves, are table stakes.  Another takeaway is that no firm really stands out on efficiency / process management. Thus, perhaps this is an area where a firm could seek to differentiate itself over the next one to two years. A third takeaway is that firm F is in trouble.  During our focus groups, several leaders of AmLaw 200 firms said they would like this data as a management tool for partners who are all-too-ready to blame the client.

These scorecard graphics above are basic management tools applied to the work of lawyers.

Progress will require leadership

As a profession, have we accepted the premise that working within a well-designed management system would make our work more valuable to clients?

Few of us would debate the general premise, particularly in front of our clients. Yet, we struggle to accept it because, in our own little zones, we fear losing control.  As a profession, we need a handful of lawyers in positions of authority who will make the decision for us.  They will be subject to a lot of blowback and pleas for special treatments.  However, in the long-run they will win our trust and respect.  We will view them as leaders.

I came to this conclusion in December of 2017 during a design workshop in Chicago.  After more than a year in business, the Qualmet team is coming to grips with a common innovator mistake: they had confused why they loved their product with why a client might buy it.  Cf. Post 008 (“[The innovator is] often deeply immersed in the technical workings of the project … [and thus] at grave risk of falling in love with features that are of little practical value to the target end user.”). Fortunately, the Qualmet team includes professionals with expertise in marketing and design thinking. I secured them meeting space at Northwestern Law.  In exchange, I got to observe the workshop.

The key goal of the daylong session was to work backwards from the daily lives of legal department professionals.  A wide variety of legal professionals–not just general counsel–were invited in for 60- to 90-minute conversations.  The Qualmet team wanted to know how they spent their time, their biggest frustrations, what they wanted most out of their jobs, etc.  Yet, very rarely were these questions asked directly. Instead, they were asked for their reactions to a series of crude prototypes (the vast majority that had nothing to do with outside counsel scorecards).

For me, the most surprising revelation was that in legal departments with several lawyers, the general counsel spends less than half of his or her time managing the department.  Instead, they are focused on being a fully contributing member of a C-suite management team.  One GC of a publicly traded company put the percentage at 70%, with less than 15% that touched on anything related to outside counsel.  Among the department professionals, the common theme was lack of time and budget to operate at a strategic level.

Indeed, I did not realize it until later, but Qualmet was running the design work shop to test their thesis that scorecards were a tool to put the general counsel into alignment with the CEO, as the performance data could be used to show how decisions regarding outside counsel were being made. The use of quantified performance puts the GC in more of a business place than a “legal place.”

One question to a general counsel that I especially enjoyed was, “Do you want to be a CEO some day?” Reply, “yes.”

Follow-up, “What about your general counsel friends — do they want to be get promoted?” Reply, “Probably. Otherwise, why do this job? Once you become a general counsel, you are more a manager and leader than a practicing lawyer. Thus, you have to develop those skills to excel at your job. Why not embrace the career path?”

We need to talk more about leadership

Leadership in legal departments is different than leadership in law firms.  Unlike a law firm leader, a general counsel can make an unpopular but necessary decision and not worry about losing revenue and triggering a proverbial run on the bank.  This reality is what is driving the consolidation of law firms into global giants. The hope is that global reach and the support services that a large firm can afford — technology, project management, process improvement, data analytics, etc — will wed the client to the firm.

I would like to see more general counsel collaborate with law firm leaders. Scorecards are just the start.  The goal should be to bring out the best in the lawyers and legal professionals they lead and manage.

What’s next?  See Confusing conversations about clients (048)