In a typical summer, recent JDs around the United States would be spending most of May, June, and July studying for the most consequential assessment of their careers - the bar exam. The two-day test, which surveys multiple areas of state and federal law, represents the final barrier to becoming licensed attorneys that have the ability to put their years of legal education to use in order to serve clients. Every July, hundreds of newly minted law graduates hunch over their laptops in convention center meeting rooms to answer multiple choice questions, fill out essays, and test their abilities to apply law to fact.

Of course, the summer of 2020 has been anything but typical. The COVID-19 pandemic has not only fundamentally changed the way the law is taught, it has also affected the way its tested. Across the country, state bar examiners have taken different approaches to the problem. Some states have postponed the exam to the fall, others have decided to throw caution to the wind and push ahead with the exam as scheduled (with COVID-19 liability waivers, of course), and a growing number of states have recognized the 'once-in-a-lifetime' nature of this event and have granted emergency diploma privilege to allow qualified JDs the ability to practice without taking a bar exam.

A growing number of states, however, have decided to find a "middle ground" in the form of an online bar exam, where applicants would use their laptops to take the test wherever they can. While this is often marketed as a "best of all worlds" approach; it is not without significant issues - specifically those of unequal circumstances, technical issues, and privacy risks.

Unequal Circumstances

The problems faced by the countless law students forced online during the pandemic as a result of stay at home orders around the country are only magnified by an online bar administration. This includes the foundational task of merely preparing for the bar. Since many exams are postponed until the Fall, JDs coming from lower socio-economic backgrounds who have carefully budgeted to take the exam in July are now met with the unfortunate reality of finding ways to make ends meet (and potentially put themselves at risk of infection) in order to maintain living expenses until it's time to take the exam. Some legal employers have even pushed employee start dates to 2021 or rescinded employment offers entirely, adding to the financial uncertainty in the face of rising expenses and student loans. Oh, and registering for one administration of the bar exam itself is non-refundable and costs nearly $1,000.

Even if these candidates have financial flexibility that affords them the ability to study on a full-time basis for the test, that doesn't change the reality of being entirely at home during the summer. We are currently on track for one of the hottest years on record, and many examinees are simply unable afford or add air conditiong to their homes so that they can study or take the test in a climate controlled environment. This will invariably lead to an unequal test administration, as cognitive performance (including attention, memory, and executive functions) is negatively affected by heat stress.

Beyond climate issues, there are additional environmental issues. With everyone staying at home, summer days have shifted from time spent congregating in parks, at the beach, or in public spaces, to full days indoors. While this isn't much of a problem for examinees who happen to live with families in large homes in suburban communities, the JDs that happen to live in densley packed apartments or in condominum complexes have to study while the sounds of neighbors arguing, children scurrying around, and general city life fill their workspaces and function to distract and decrease inefficiency in attention based tasks.

The bar exam takers that were in these circumstances after law school used to have other options - they could return to their campuses to study in quiet spaces, they could vary their routine by visiting local coffee shops, or even catch up on studying in climate controlled public areas like malls, train stations, and libraries - all places that are now (appropriately) unsuitable due to the pandemic. The reality is that not every candidate can follow Harvard Law School's advice to take out additional loans in order to rent office space (yes, that was an actual suggestion offered by a member of the school's administration).

With a remote examination, a student's ability to prepare and take the exam is contingent on their or their family's ability to afford peace and quiet.

Technical Difficulties

As someone who worked in the software industry as a developer and product manager before going to law school, I can attest to the applicability of Murphy's law in techology. Anything that can go wrong will go wrong, and tracing the issue is an entire career in itself. Just this year, the remote administration of AP exams ended up suffering from technology problems that forced students to either forfeit or take a new exam on the same subject. These problems of course exist during "in-person" bar exam administrations, but those administrations have the added benefit of IT support staff on call as well as an option for exam takers to hand-write their responses in the event of catastrophe. As of this writing, there has been no information as to whether a hand-written option would be available to remote bar exam takers, and I highly doubt this is even on the table.

While I was lucky enough to use the exam adminstration software used by many bar exam administratiors (Examplify by ExamSoft) during my own law school experience without any problems, I can confidently attest that for every exam that I took that had more than 20 people in the room, at least one person in the room experienced a software or hardware crash that ncessitated a switch to handwriting. I don't say this to knock ExamSoft specifically, as it's just a reality - computers are not uniformly reliable, and when an exam is as consequential as the bar, there needs to be a safety net.

What is going to happen when someone's computer fails during the bar exam? Is there going to be a dedicated phone support line? What will happen to that candidate's scores? Will they have to spend nearly $1000 in fees to register for a new exam? What is to stop a bad actor from purposefully engineering a hardware failure (or simply cutting the power to their computer) when they realize that the exam isn't going the way they hoped it would go?

These problems are also compounded by issues of internet connectivity. With internet outages increasing due to capacity issues caused by an increasingly online working population, bar exam takers are effectively placing their futures in the hands of internet service providers. Comcast, for example, experienced 26 instances of problems or outages during the month of June alone. Of course, a vendor like ExamSoft will argue that a stable internet connection is not required for the duration of the exam, just the downloading of questions and uploading of answers, but without a generous "grace period" given to all exam takers who end up experiencing outages that could last for hours and hours, internet connectivity will be an issue.

In lieu of sending every single test taker a preconfigured "loaner" laptop just to take the exam (and even that will invariably have problems), I highly doubt that a remote administration of the bar exam will be without significant technical issues for a good portion of the population.

Edit - 07/13/20 - Algorithmic unfairness

One of my law school classmates, Julia Schur, mentioned this to me that I completely forgot to include in my original upload. If ExamSoft is going to be using their machine learning platform to detect cheating, do we have any insight as to whether they have trained their models on a diverse pool of candidates? I'm hoping that ExamSoft has developed a mechanism to demonstrate that their cheating detection algorithms don't disproportionally affect BIPOC individuals. More information on algorithmic unfairness here.

Privacy Concerns

If the remote exam administration is conducted by ExamSoft using their remote examination tools, there are also significant privacy concerns that need to be addressed. ExamSoft's distributed assessment product, ExamMonitor, is marketed as a method to "Maximize Exam Security Using Remote Proctoring" (link to brochure). ExamMonitor claims to ensure secure exam administrations without a consistent internet connection, and this is done through a continuous audio and video recording of the examinee during the test.

Using your laptop's webcam, ExamSoft's ExamMonitor will record the audio from your room, your face and your screen activity while you're taking the test. Following an upload of all of that data, ExamSoft will use automated analysis tools (likely applying machine learning) to assess a student's "gaze" throughout the exam, the presense of background noise, and a whole slew of unspecified other criteria. If the decision algorithm determines that there's a concern, the issue is flagged and brought in front administrators through ExamSoft's proctoring reports. As someone who tends to stare off into space while thinking through exam answers, the thought of my gaze being scrutinized and amounting to a risk to my character and fitness assessment is, to put it mildly, not ideal.

In implementing this solution, every exam taker is effectively bringing their state's board of bar examiners into their homes, and will have to deal with the psychological impact of having any visual or audio background noise being flagged as a potential violation while they're taking the most important test of their lives. Was that hammering from the upstairs neighbor, or was it morse code? Did the test taker's mother barge into the room on purpose, or was it actually an accident? Why is the test taker scratching his face? Why does she keep looking at the wall? In a time where we are all already dealing with Zoom fatigue, the thought of being scrutinzied during the course of a two day exam is nauseating at best.

Fundamentally, an online administration of the bar exam with these invasive controls (meant to curb cheating) amounts to a coerced intrusion into someone's private space. It's one thing for employers to demand the use of webcams to keep an eye on the people that they're paying, but it's an entirely different matter when you are forcing your future colleagues to pay for the privilege of letting you into their private spaces by giving them a choice between "let us watch, listen to, and scrutinize you at home" and " why don't you wait until 2021 in order to take in-person administration of the bar exam."

Concluding thoughts

Through this exploration of circumstantial, technical, and privacy issues associated with a remote administration of the bar exam, I hope that I was able to raise a few eyebrows and promote a sense of caution as bar examiners decide the fates of the tens of thousands of lawyers-to-be that are eager to enter the profession, pay off their compounding debts, and make an impact during a time where attorneys are needed more than ever.