The average score on the Multistate Bar Exam fell to a 34-year low in July 2018, and bar exam pass rates have been mostly declining for more than a decade.
One possible source of the problem? The bar exam doesn’t seem to test the skills and knowledge that measure a graduate’s competency to actually practice law.
But a new research partnership is trying to change that. The Institute for the Advancement of the American Legal Profession and an Ohio law professor have teamed up for the Building a Better Bar project to collect information through dozens of focus groups about the knowledge and skills considered most important for new lawyers.
Deborah Merritt, a professor at The Ohio State University’s Moritz College of Law, is collaborating with IAALS on the project and has focused her research on legal education and the legal profession for about a decade. She said the National Conference of Bar Examiners, a nonprofit that creates bar exam materials with input from different states, relies on the state Supreme Courts to provide the definition of minimum competence.
“That definition has, quite surprisingly, gotten lost over time,” Merritt said. “I think in law we’ve taken partly a ‘We know it when we see it’ approach, and partly, ‘Well, if we build it on what law schools do, then that must be the right direction.” She said the bar exam is modeled on both law schools’ method of testing and the subject matters taught, but the practical realities of practicing law should instead guide how minimum competence is defined.
Merritt said she has focused her research on legal education and the legal profession for about the past 10 years, with a central question of how to train the best lawyers possible to serve clients.
Merritt said data on declining bar pass rates sparked her interest in exploring the underlying question of whether the exam itself is valid.
The Building a Better Bar project will collect information through 60 focus groups in 12 states. Logan Cornett, a senior research analyst at IAALS, said even though it’s difficult to make conclusive judgments based on findings from focus groups, the exploratory nature of focus groups help minimize biases that could show up in surveys.
“Surveys are great for some things, but we felt like we just didn’t know enough about the topic yet to develop a survey that was sufficiently free of bias,” she said. “We are … trying to get at the information without biasing the way that people are thinking about it.” Cornett said it would be difficult to come up with a comprehensive checklist for a survey that contains every type of skill and knowledge lawyers might think of as crucial for brand-new attorneys.
She said the researchers plan to hold focus groups through February of next year and to publish at least one report with the study’s findings by the end of 2020.
Merritt said the National Conference of Bar Examiners is doing its own study on minimum competence by collecting survey data.
“Let’s say that these two projects suggest that negotiation is very important for lawyers, and we know now that negotiation is a skill that you teach,” Merritt said. “If you put those things together, it would seem that really there should be a strong emphasis on negotiation, both on the bar exam and in law school.”
Cornett said the focus groups will involve lawyers with three or fewer years of experience and will seek a few broad types of insights: What types of resources and knowledge did they need most in their first year of practice? What are situations they would do over again if they could? And, in retrospect, what are the resources they needed for those situations?
The study is a follow-up to IAALS’ 2014 Foundations for Practice project. Cornett said the survey got more than 24,000 responses from lawyers across the U.S., and ultimately the most important responses were about the skills new lawyers need right out of the gate. She said respondents tended to emphasize character traits such as integrity and perseverance rather than “traditional” legal skills.
“The idea basically was, these are the things that lawyers are saying that new lawyers need immediately when they graduate. Is this what law schools are teaching?” Cornett said. “And in a lot of cases, the answer is, not yet.”
Merritt pointed out that the notion that a lawyer’s adequacy can only be assessed by other legal professionals has changed.
The rise of recommendations and available research on the internet has made it easier for potential clients to make judgments about lawyers’ competency before they hire them.
“If somebody is choosing a doctor or lawyer today, they may not choose randomly out of a list. They may look online and see the recommendations and what people are saying about these professionals.”
— Julia Cardi