How legal services will be evaluated in 2021 and beyond

NewLaw is not what you think it is.  It is not a label to be applied only to new companies with trendy names.  It is a business model that any legal services provider can, in theory, adopt.  Cf. Post 055 (discussing clear evidence that “legal operations is a discipline” for buyers and sellers of legal services and thus not just a role within a legal department). But, while new companies built for it, others have to overcome how they created themselves in the first place.

Of the elements that define the new business law model—tech-intensive defined processes, lawyers and business professionals blended on teams, incentives for efficiency—it is the use of metrics that can differentiate how the results look to a corporate legal department.  And no provider wants to look like “Old Law” on a legal department’s dashboard.

Metrics affect buying behavior for only a minority of the market today.  That’s not a reason to ignore them.  It means there is still time to understand and affect metrics before they drive the majority.  The NewLaw model and its metrics are like the automotive industry’s transition to electric—a minority of the market but clearly where investments and learning need to happen for all.

Mary O’Carroll

The pace of change is equally significant in the legal departments.  Thousands of people with legal operations responsibilities in these departments have been driving the evolution of measured results.  Mary O’Carroll, Director of Legal Operations for Google and President of CLOC, described the role of legal ops leadership as not just selecting the “right resource; whether it is internal, external, outside counsel, alternative legal service provider” but also evaluating “if it could be automated” or “do we have to do it at all?” LTV Podcast, “Mary O’Carroll: The Legal Operations Role and Its Growing Importance,” Law Technology Today, Mar. 3, 2020 (hosted by Ralph Baxter).

An Trotter

Further, economic pressures on corporations and the proliferation of scalable Big Four accounting firms and law companies (fka alternative legal service providers) may change who is leading legal department operations.  An Trotter, Senior Director of Operations, Office of the GC of Hearst, recently envisioned that “the golden age of legal operations is over.”  Trotter, “Golden Age of the Corp. Legal Department is Done – The Evidence,” Ops in a Box Blog, Oct. 1, 2020. While some legal services are strategic, “running an advanced in-house legal department may no longer be seen as strategic to some companies.”  Id. If a department were to hand operations and procurement to a third party, this would surely accelerate the focus on metrics-driven performance analysis.

In any case, metrics are the language of the NewLaw model because they provide the context for modern General Counsels to make decisions.

Common Metrics

Firms and departments frequently calculate financial and diversity metrics, and both affect legal buying decisions.

Example 1: Financial

Financial metrics are so core that they are the only type of metric on most law firm partner dashboards.  These dashboards likely show fees, write-offs and write-downs, expenses, pre-bills for review, and, hopefully, phased budget status and projections.  These are Metrics 101.  Legal departments are often tracking even more, including those below.

  • Spend analysis
    • Blended rates
    • Percentage of matters that are fixed / alternative fee
    • Year-on-year spend by matter type
  • Budget analysis
    • Accuracy
    • Scope changes
  • Efficiency/effectiveness analysis
    • Complexity or the significance of matters (sometimes by task)
    • Unit or task costs
  • Billing analysis
    • Compliance, per outside counsel guidelines
    • Timeliness, sometimes with penalties
  • Time and staffing analysis
    • Staffing ratios
    • Timekeeper churn ratio
    • Block billing as a percentage of total spend
    • Timekeeper mental health: % hours over 8 per day, % hours on a weekend
    • Re-categorization of time by work done to validate rate appropriateness
    • Spend % by gender and ethnicity (more on this in the next section)
David Cambria

David Cambria, former Director of Global Operations – Law Compliance and Government Relations for Archer Daniels Midland and now Chief Services Officer at Baker McKenzie, was among the pioneers for calculating such information and sharing it with outside counsel to clarify expectations and support improvements in each firm.  And, now, such analysis is more commonly undertaken thanks to data and AI systems like Persuit, Digitory Legal, Bodhala, and Counsel Command, and matter management software like Mitratech’s Team Connect.

Example 2: Diversity, Inclusion, Equity, and Belonging

Diversity metrics have exploded in their depth, breadth, and scope.  It is the third most likely type of metric, after financial and expertise, to affect buying and panel selection decisions in larger departments.  Individual programs by departments like Facebook and Microsoft and groups like the global financial service leaders, Buying Legal Council, and AdvanceLaw define the new expectations.

Across the industry, core metrics include:

  • Diversity of a firm. Some programs, like The Intel Rule and the American Lawyer Diversity Scorecard, focus on the diversity of lawyers in the firm.
  • Diversity of client teams and leaders. Other programs focus on the diversity of the teams working on their matters. Novartis, for example, considers the lawyers used over the lifetime of each matter, while Microsoft and Facebook also consider matter leadership roles.
  • Talent pipeline. The Diversity Lab’s Mansfield Rule and Rare’s Race Fairness Commitment programs challenge firms and departments to consider diverse candidates when they are hiring and promoting.
  • Belonging. Evaluating if a person feels they are a welcome and equal member of a firm isn’t data tracked in HR systems but firms may undertake periodic surveys.  The closest published metric is Vault’s Best Law Firms for Diversity, as the results stem from an annual survey of associates’ perceptions.

However, law firms realize that most departments still ask about diversity but do not actually let it drive their decisions.  So, it is early days, but the pace of expectations and progress is not linear.  It is common for firms and teams to be 30+% diverse today but Novartis is adjusting its target toward parity for its 22 panel firms and Orrick has a 5-year Move the Needle Fund goal of 55% diversity across 40 key clients.

Likewise, the diversity metrics themselves are becoming more intense and comprehensive, including:

  • UK Diversity Focus. In the recent past, diversity in the UK was primarily a focus on supporting the female gender, but the UK is evolving in the trajectory of the US with the aforementioned RFC. In addition, the UK InterLaw Diversity Forum is extending the ABA Model Diversity Survey in the UK market. See Announcement of MOU.
  • Julie Savarino

    Law firm staff.  As noted by American Lawyer in describing Calibrate Legal’s CalibrateID program and the advocacy of Julie Savarino, Chief Client Experience, Value & Development Officer of Business Development, Inc., there are initiatives to recognize that law firm professionals aren’t simply “non-lawyers,” but part of the delivery of legal services and, therefore, their diversity should equally matter.

  • Legal department lawyers.  Some legal departments like Target recognize that they do not expect to hold their outside counsel accountable for diversity achievements they themselves cannot achieve. See “Supporting and Spearheading Diversity,” Corporate Counsel Business Journal, Oct. 28, 2017 (Q&A with Target General Counsel Brian Liu).
  • [click on to enlarge]
    Law firm’s providers.  Legal departments explicitly cajole and measure their suppliers of services. The day is coming when law firms will be expected to do the same.  The LegalTech Fund’s briefings, Nate’s News, see graphic to right, and Orrick’s Observatory help by providing information on hundreds of companies, with some insights on diversity and access to justice.
  • Tracking the lifecycle of diverse employees. Understanding not just a point in time, but how diverse lawyers and staff progress through the firm over time.
  • Law school pipeline. Extending a firm’s knowledge of its hiring pipeline by understanding the diversity of the law schools from which it hires. Cf. Post 114 (Evan Parker assembling data on law school diversity pipeline).
  • Minority-owned and women-owned firms and businesses.  Organizations like NAMWOLF, WBENC, the SBA’s HUBzone support and track minority-owned and women-owned law firms. Law firms or departments can hire these firms and increase their joint diversity. See, e.g., Search Function from NAMWOLF.

Example 3: Client Relationship Metrics

Law firms are more sophisticated in identifying the strengths and risks of client relationships.  Since these are factors for firms but not part of the buying decisions for departments, they are not covered here. Nonetheless, a firm should be informed of such insights.

Data limitations and the formation of Legal Metrics

Despite all of these evidence-based efforts, the legal industry is still very much in the early days of leveraging data to improve performance.

To use diversity metrics as an example, the current state-of-the-art is unable to capture the true teams working on matters, especially as traditional firms pair with law companies, lawyers pair with business staff with data analytics, AI, research, and project management expertise, and teams are increasingly global despite many metrics being US-centric. Additionally, despite good intentions, there are too many variances on metrics and programs to be clear on which firms or teams are consistently excelling in diversity.

To address the many data limitations that exist across the industry, including diversity, several law firms, legal departments, and ALSPs have come together to form a consortium, called Legal Metrics, to collaborate on the standardization of metrics to drive win-win performance. See Rhys Dipshan, “Major Law Firms, In-House Departments Strive to Modernize Metrics in New Collaboration,”, Feb. 21, 2020. [Disclosure: my firm, Winston Strawn, is one of the anchor members of the Legal Metrics consortium.]

Vince Cordo

At Legal Metrics, we have documented over 120 metrics just in the diversity context. See Roadmap of Metrics.  At the 2020 Annual College of Law Practice Management (COLPM) meeting, COLPM Fellow  Vince Cordo, Chief Client Development and Relationship Officer at Holland & Knight, commented that the market would benefit from accounting-style GAAP standards for uniformity and clarity.  While Legal Metrics is not attempting standardization, we have created the ability to see and re-use metrics created by others, as we believe this will help reduce the need for many departments to create unique metrics for their diversity programs.  And programs like SixFifty’s D&I Assessment and Action Plan are available to help departments learn from industry practices.

Metrics tracked by departments, but not firms

The metrics below are commonly tracked by larger legal departments, but it’s rare for firms to do so

  • Matter Responsiveness. A firm’s ability to respond quickly, including clearing conflicts, developing and getting approved an accurate budget, assembling the team, and beginning the work.  Perkins Coie, working with Microsoft and 7-Eleven (ACC Value Champions), were early examples of focusing on this metric by connecting clients and the firm via their matter opening processes.  Startup Lupl is in beta to connect the rest of the market to each other and to help them stay engaged throughout a matter.
Illustration of Lupl joining teams across organizations on a matter
  • Matter Duration.  Departments mention this metric as an important aspect of the relationship because, essentially, it measures how long it takes a firm to solve the client’s problem.  Matter responsiveness is a subset of matter duration.  Partners focus on solving problems when serving a client, but it is unusual for a firm to track and focus on improving this metric.  Departments that do measure can be reluctant to hold firms accountable for duration in fear that they will quickly staff up the matter to improve timing when not appropriate to do so.
  • Post-Matter Feedback. The surveys that some legal departments send their in-house lawyers when a matter is complete are the equivalent of a legal Yelp review.  These lawyers are asked their perceptions of the outside counsel’s expertise, project management prowess, budget accuracy, knowledge of industry and company issues, consistency of advice, and even personal aspects like listening skills.  These results can then be associated with value and be a key determining factor in winning future work.
  • Matter Complexity. Based on my ad-hoc interviews with legal departments, approximately one-third of larger departments rate their perception of the complexity of a matter.  They can then use this information to rate and rank firms to ensure the rates and overall fees paid match the complexity of the work done.  Firms may have a different perception of the complexity, of course.  Proactive firms should ask for this information as it is fundamental when setting rates, budgets, and marketing approaches.

 Metrics that will define the next three years

  • Stéphanie Hamon

    Value of a Matter. No new metric has more attention than the calculation of value.  Legal departments like Barclays have established their own calculations for judging the value law firms provide them.  While at Barclays, as Head of Commercial Management (now Head of Legal Operations Consulting for Norton Rose Fulbright), Stéphanie Hamon created early insights into outside counsel value analysis.  They used after-matter surveys which focused on six factors:

    • Consistency of Advice
    • Thought Leadership
    • Value for Money
    • Collaboration
    • Innovation
    • Diversity and Inclusion

The surveys made up 60% of a value calculation, while factors measured by the legal ops team – use of alternative fee arrangements, strategic nature of the work at a task level, etc. – made up the remaining 40%.  The results were used to rank firms on colorful charts that made under-delivering firms obvious.  Stéphanie points out, “Value may be an overused term.  Essentially, it’s a focus on understanding what success looks like.” See information and quotes from personal interview.

Dan Linna

Today, there are groups working to better understand and standardize calculations of value, at the task and matter level.  Dan Linna, Director of Law and Technology Initiatives & Senior Lecturer at Northwestern Pritzker School of Law & McCormick School of Engineering and now also the Chairman of the ABA Legal Analytics Committee, is perhaps the most invested in experimenting with algorithms and working with Winston & Strawn to do so.  The Legal Value Network is a relatively new but very experienced and fast-growing group of leaders sharing knowledge to grow the maturity of this area as well.

  • Expertise Ratings. Department use RFPs and pitch meetings to understand expertise, but this process is high effort and manual for all involved.  These won’t be replaced any time soon, but data is already supplementing perspectives of qualifications, for both departments and firms.
  • Efficiency. Understanding efficiency is relevant for certain types of matters.  Some companies have focused on assigning allowable fees for certain tasks or phases, and then it becomes each firm’s responsibility to bill accordingly.  It is also common to use AI tools to assess bills after they are sent.  But it is challenging to fairly assess the context of each matter and to correct inefficiencies retroactively.  At Winston & Strawn, we have been using the Ping time management system to better understand tasks as they occur.  By starting at the root, we believe, over time, we can learn and tune our firm’s approach to training, use of technology, and knowledge management so we can improve effectiveness rather than rely on a client to point out issues after we’ve completed the work.
  • Risk/Problem Avoidance. This seemingly quiet area may prove to disrupt the market most deeply in the long run.  While some legal services today proactively help corporations mitigate risks, the market is primarily focused on responding to events once they have occurred.  The convergence of Big 4 services (legal and otherwise), the data-driven intelligence of legal departments, and renewed interest in reducing legal costs and staffing creates the potential for corporations to shift spending from reactive to more proactive investments in mitigating risks.  The value of preventing an incident will always prove higher than the value of how well it was addressed after the fact.

Actions for 2021

Regardless of labels, all legal service providers need to continuously learn, tune their model, and show their results.  Larger legal departments have armed themselves with AI to audit firms after the fact, but firms benefit greatly by proactively seeing themselves by their client’s metrics.  Consider a firm with $500M in billable time that increases by 1% its productivity, collections, work won, and billing timeliness resulting in more satisfied clients and an additional $20M+ to the bottom line.  That gives investments in tech and data analytics programs a direct impact to profitability.

For the coming year, below are practical actions for law firms to engage with clients to better understand and reflect the client’s priorities.

  • Partners and firm leaders should have easy access to financial, experience, diversity, and other client-centric metrics. Legal Metrics provides a firm access to legal department’s diversity metrics, with other types of client metrics added over time.
An example of a firm measuring itself against its client’s metrics [click on to enlarge]
  • Bills should not be sent to clients without some level of AI review (consider an array of good tools, including Ping, Clocktimizer, Digitory Legal, Bodhala, Apperio, Brightflag, SimpleLegal, and others)
  • Be aware of RFP tools clients use (e.g., JusticeBid, Persuit, Priori) so you can understand how your firm looks to them on these types of data-driven dashboards.
  • It should be clear in a firm who has accountability for developing existing client relationships alongside partners, such as client development and client service roles. This isn’t the same as marketing; it’s a focus on giving the client maximum value for working with your firm.  Such a role should not only be walking in the shoes of the client, but also able to skip across all of the silos in the firm.
  • A firm should know the metrics of at least its largest 100 clients. These metrics indicate what is important to the clients – therefore they should be very important to the firm.
  • Recognize that most legal departments don’t have budgets or people to keep up with the latest systems and data analysis. Showing them that the firm is doing it for them rather than taking advantage of them is a great step toward partnership and engagement.

And, legal departments should hold their legal service providers accountable for such analysis, thoughtfulness, and improvements.

Edward Walker

Firms are struggling to differentiate themselves in a world where in-person relationship building is not allowed.  Departments that have legal operations leaders provide a parallel relationship track that isn’t hampered by distance.  If a firm learns to speak their language, it will better understand the NewLaw journey.  Edward Walker, General Counsel of UK transport company Abellio Group, sums it up well “How they deliver legal services is now … as important as the content of the legal expertise.”  See LinkedIn Post (Aug. 2020).