Last month we learned that Amazon is planning to deploy AI cameras that will constantly scrutinize drivers inside the cabins of its delivery vehicles, and inform their bosses when the camera thinks they’ve done something questionable.

The device Amazon is installing (called “Driveri,” pronounced “driver eye”) has cameras pointing in four directions, one of which is toward the driver. In a video posted online, the company says the “camera records 100 percent of the time when you’re out on your route,” and watches for 16 behaviors that will “trigger Driveri to upload recorded footage.” These include not only accidents but also such things as following another car too closely, making a U-turn, failing to wear the seatbelt, obstructing the camera, “hard” braking or accelerating, and appearing to be distracted or drowsy — or what the AI interprets as those activities, anyway. Sometimes the robot camera will shout commands at you, such as “maintain safe distance!” or “please slow down!” One driver told CNBC that if the camera catches you yawning, it will tell you to pull over for at least 15 minutes — and if you don’t comply, you may get a call from your boss.

The cameras in this system are not streamed live to management; this is an AI monitoring system. The device itself decides when to send video clips to the bosses and when to issue verbal alerts to drivers. But as we have long argued, nobody should make the mistake of thinking that we can’t suffer many forms of privacy harm when being monitored by machines, not least because those machines are programmed to “snitch” to actual humans when they see something they think is bad. The company that makes Driveri, Netradyne, also advertises that its product keeps scores on drivers that are updated — and provided to management — in real time. (Such a function is not mentioned in Amazon’s video).

Given how bad AI is at understanding the subtleties of human behavior and dealing with anomalies, this system could lead to real fairness and accuracy issues. Automated test proctoring software, which also uses video to monitor people for subtle behaviors (in this case, cheating) has certainly been rife with bias and accuracy problems. Machine vision is very brittle and can fail spectacularly — even at the fundamentals, like recognizing a stop sign. Netradyne boasts that “every stop sign & traffic signal is identified and analyzed for compliance measurement.” But what happens when the AI thinks it sees a stop sign where there is none, and flags the driver for “running” it?

Ideally a human being would review the video and exonerate the driver, but given how automated Amazon’s management is, we don’t know how often that will happen. Workers in Amazon’s warehouses, for example, are constantly supervised by robots that judge whether they’re moving packages quickly enough. If they don’t like what they see, those robots issue warnings and even fire workers automatically — without any human input.

Amazon touts the system as a beneficial safety measure. It could indeed reduce accidents — though that should be proven — but as a society we’re going to need to figure out how much to allow ourselves to be overseen by automated AI cameras that engage in intrusive monitoring, judging, nagging, and reporting of our behaviors. Potential fairness issues aside, that kind of monitoring would probably make anyone miserable. There are almost certainly ways to be found to use AI to protect the safety of workers that feel empowering and protective, instead of infantilizing and oppressive.

Meanwhile, this kind of robot monitoring is becoming an increasingly prominent sore spot for workers. Some UPS drivers, for example, have opposed that company’s use of such cameras. (UPS drivers, unlike Amazon’s, are unionized and actually employed by the company whose uniforms they wear.)

Amazon workers’ complaints about robot management are part of growing labor tensions and criticism of the company for unethical labor practices. The company has been sued by the New York attorney general for failing to protect workers against COVID-19 and retaliating against those who complained, and was fined last month by the Federal Trade Commission for stealing workers’ tips. Amazon drivers in particular reportedly face brutal working conditions, and critics charge that the company places performance demands on them that pressure them to drive dangerously fast, while evading responsibility for the resulting accidents by insisting that they’re contractors. The Amazon drivers I have spoken to confirmed that they are urged to drive safely but also pushed to complete an unrealistic number of deliveries within a shift.

Driveri thus looks like a company’s attempt to use technology to solve a problem that its own managerial practices and profit drive may be creating. These technologies are like factory farms that pump our food with antibiotics — an attempt to use technology to unnaturally suppress the side effects of unhealthy and inhumane practices. This is something that we’ve already seen in the trucking industry: Instead of giving drivers protections from unhealthy productivity demands, they get micro-surveillance. And workers end up squeezed on both ends.

That squeeze may only increase as the AI is refined. For example, if sunglasses defeat Driveri’s drowsiness and inattentiveness detectors, drivers may be told they aren’t allowed to wear them. That could be just the beginning of many ways they are forced to conform their behavior, movements, and dress to the needs of the AI that is watching them. We’ve already seen that happen in other areas; we’re no longer allowed to smile in our passport photos, for example, because it reduces the effectiveness of face recognition technology. Ultimately, the technology threatens to enable a modern-day version of Taylorism, a 19th century industrial movement also known as “scientific management” that involved monitoring and controlling the minutiae of industrial workers’ bodily movements to maximize their productivity.

The issues raised by AI video monitoring extend far beyond Amazon and its particular practices. To begin with, Amazon is not the only company experimenting with this kind of robot surveillance; a number of trucking companies, for example, are imposing it on their drivers. More broadly, as AI cameras get smarter, there are many institutions that have different incentives to use them to visually monitor people. We could soon see not just employers but also everything from museums to restaurants to government agencies deploying this technology — anyone who wants to enforce a rule, protect an asset, or gain a new efficiency.

Technological monitoring of workers has long taken place through other data-collection devices, down to and including the time clock, but these new tools don’t require expensive or specialized data-collection devices, or efforts to get workers to use them properly. All that’s needed is a camera. And improving AI is likely to open up ever-wider possibilities for automated visual monitoring, as we discussed in our 2019 report, The Dawn of Robot Surveillance.

Employees like drivers and factory workers whose jobs are most at risk of being supplanted by AI (but for now are just being integrated with it) will be the first to be placed under oppressive AI surveillance microscopes, and we should support their rights to maximize their self-determination through unionization and other measures. But AI monitoring will soon move beyond those groups, starting with less powerful people across our society — who, like Amazon’s nonmanagerial workforce, are disproportionately people of color and are likely to continue to bear the brunt of that surveillance. And ultimately, in one form or another, such monitoring is likely to affect everyone — and in the process, further tilt power toward those who already have it.

As protesters filled the streets last summer to decry the tragic killings by police of George Floyd, Breonna Taylor, Ahmaud Arbery and countless others, they brought signs and slogans with them. Poster board and cardboard pieces were lifted into the air with firm demands scrawled across them: “Justice for George,” “#SayHerName,” “I Can’t Breathe,” and “No Justice, No Peace” were familiar phrases bobbing amongst the sea of activists. As the weeks stretched on, the movement catalyzed by the hideous killing of Floyd caught on video continued to grow, with millions of people taking to the streets. Among the signs, a more specific demand began to appear: Abolish qualified immunity. 

Once an obscure legal doctrine, qualified immunity is now in the spotlight — and in the crosshairs of many activists and advocates nationwide. For decades, the doctrine has shielded police officers and other government employees from being held responsible for all sorts of malfeasance. Qualified immunity makes it nearly impossible for individuals to sue public officials by requiring proof that they violated “clearly established law.” 

In a rare show of solidarity with protesters in cities like Minneapolis, New York, and Portland, courts and state legislatures began to take notice, too — in June, Colorado lawmakers passed a bill that gutted the doctrine’s power in state courts. Multiple lower federal court decisions have also acknowledged how qualified immunity functions more as absolute immunity, and shields police officers from accountability, with even a conservative Supreme Court justice calling the doctrine into question.

The ACLU is a part of the movement to end qualified immunity once and for all, through our work advancing legislation in statehouses, combating the use of the doctrine in court, and advocating for an end to qualified immunity on the federal level. 

In the Statehouse

The brunt of law enforcement’s racial terror campaigns is felt by the Black and Brown communities that are forced to deal with outsized police presence every day. The fight to combat that harm is led by a coalition of grasstop groups that organize and advocate in city halls and statehouses across the country. In several states, including Minnesota, the ACLU has fought alongside these groups to advance reform through legislation. 

• Minnesota: The ACLU of Minnesota is working with the Institute for Justice to develop legislation that would bypass the effect of qualified immunity by making it easier for people to sue government agencies — not just individual officers — in state courts when police violate their rights. Additionally, the ACLU of Minnesota is advocating to reform the laws that allow officers like Derek Chauvin, who killed George Floyd and who had a long history of civilian complaints on his record, to keep committing violence against the community. Currently under Minnesota state law, civilian oversight boards cannot make findings of fact relating to a complaint against a police officer, impose disciplinary sanctions, or make binding recommendations. H.F. 905 would repeal the law that prevents civilian oversight boards from having these powers, allowing local governments to create empowered boards that can take tangible action against officers accused of misconduct. Removing this barrier at the local level is a first step toward independent, community-informed oversight of policing and public safety.

   

• Illinois: After decades of unacceptable police abuse and horrors, current Illinois law still protects out of control officers from being held accountable for violating people’s constitutional rights. These protections do not serve good police officers; they do not serve our communities; they only serve bad apples in Illinois’ police ranks. A recent poll shows that 91 percent of Illinois voters are strongly supportive of legislative efforts that hold police accountable for misconduct and 69 percent of voters agree that reform is necessary now because of racial bias in policing. Reflecting this overwhelming public support, the ACLU of Illinois supports H.B. 1727 — the Bad Apples in Law Enforcement Accountability Act — which removes the protections of qualified immunity in state court so that police officers can be held accountable when they violate someone’s constitutional rights.

   

• New Mexico: New Mexico has one of the highest rates of fatal police shootings in the country. The New Mexico Civil Rights Act creates an avenue for New Mexicans to bring claims for damages in state court against police officers and other public officials who violate the rights guaranteed to them under the New Mexico Constitution. The bill specifically prohibits the use of qualified immunity. 

In the Courts 

Last term, multiple petitions before the Supreme Court called into question whether qualified immunity should be limited or abolished altogether. 

• The court grouped three petitions together, including ours in Baxter v. Bracey, and then repeatedly delayed acting on them. It seemed possible that maybe the court was finally going to meaningfully tackle qualified immunity. Then, on June 15, 2020, mere weeks after Derek Chauvin killed George Floyd and millions of people flooded the streets to protest police brutality, the Supreme Court denied the petitions. Justice Thomas was the only one to write anything on the occasion of the court declining all the petitions; he wrote to protest the denial of certiorari in our case, Baxter. For a brief moment it looked like the cross-ideological coalition we are part of might have convinced four justices to take a case. Then the bubble burst.
• Five months later, a new glimmer of hope emerged. In November 2020, the court granted, reversed, and remanded a qualified immunity decision out of the Fifth Circuit in Taylor v. Riojas. Trent Taylor was incarcerated in Texas and he spent six days in heinous conditions: The first cell where he was detained was covered almost floor to ceiling in human feces, and he was forced to sleep naked in sewage in the freezing cold in his second cell. The officers responsible for this gross violation were granted qualified immunity by the court, which reasoned that “[t]he law wasn’t clearly established” that “prisoners couldn’t be housed in cells teeming with human waste … for only six days.” 

The Supreme Court disagreed: “No reasonable correctional officer could have concluded that, under the extreme circumstances of this case, it was constitutionally permissible to house Taylor in such deplorably unsanitary conditions for such an extended period of time.” As law professor Joanna Schwartz explains, Hope v. Pelzer is the only other case in which “the Supreme Court ruled that qualified immunity could be denied in the absence of a prior court opinion on point … Since 2002, the court had only paid lip service to the notion that qualified immunity can be defeated without a prior case on point.” 

• Only a few months later, the court did it again in McCoy v. Alamu. McCoy was incarcerated in Texas when an officer attacked him for no reason. The Fifth Circuit granted qualified immunity based on its understanding that the defense is especially difficult to overcome in excessive force cases. But the Supreme Court granted McCoy’s petition, vacated the Fifth Circuit’s opinion, and remanded the case to the lower court with instructions to reconsider the case in light of Taylor.   
• Taylor and McCoy have ignited a debate among qualified immunity nerds (that’s a compliment), as Adam Liptak has reported. Professor Schwartz argues that “the court is indicating a change” and “appears to be sending a message that lower courts can deny qualified immunity for clear misconduct, even without a case with identical facts.” Jay Schweikert at the CATO Institute, with whom we have worked closely on qualified immunity reform efforts, believes “the Supreme Court has decided to forgo reconsideration of the doctrine in favor of small doctrinal clarifications.” Anya Bidwell and Patrick Jaicomo at the Institute for Justice are the most optimistic, characterizing these as the “early days in the reconsideration — if not ultimate rejection — of the court-created doctrine of qualified immunity.”

We’re very glad to see that there are cracks developing in the shield of qualified immunity. But these cracks are not nearly enough. The ACLU will continue to fight in court to see the doctrine weakened and ultimately dismantled, as we did recently in yet another horrific Fifth Circuit case. 

• One ongoing case that highlights both the absurdity of qualified immunity and the extent to which officials may go under its protection is Black Lives Matter D.C. v. Trump, the ACLU-DC’s class action lawsuit challenging the vicious and unprovoked attack on civil rights demonstrators in Lafayette Square last June. The defendants in the case were sued under Section 1983 and Bivens, which is another type of case where officers can use qualified immunity. From the Park Police officer who beat a journalist as she was escaping the protest, all the way to former Attorney General Bill Barr, they have all invoked qualified immunity to avoid liability for their misconduct.

In moving to dismiss our case, defendants have argued that their conduct cannot be “clearly established” as unconstitutional — thus defendants are shielded by qualified immunity — unless plaintiffs can point to a specific case involving “a presidential appearance, an alleged dispersal order emanating from the Attorney General himself, a city-wide curfew and emergency order” and more. They are wrong, but under qualified immunity, we can’t be sure a federal court will see it our way and refuse to countenance brutality with impunity.  

In Congress

Qualified immunity reform is needed to ensure that police can be held accountable after they violate the constitution. But we also need reform on the front end that prevents police brutality before it happens. An important first step is to set clear national standards that require all police departments to adhere to common-sense limitations on use of force and best practices. 

• President Biden has already committed to the creation of a national, model use-of-force standard as one of his racial equity priorities. The ACLU is currently lobbying Congress to pass legislation that ensures this model standard truly conforms to the best practices in the field by embracing the principles set forth in the PEACE Act, which would permit officers to use force only when necessary, proportional, and as a last resort, after less extreme alternatives are exhausted. That standard would not only apply to federal agents, it would provide incentives to state and local police departments to adopt the rule.  
• Ayanna Pressley’s End Qualified Immunity Act would end qualified immunity for state and local actors. 

Qualified immunity fosters an environment where government agents, including police, may feel empowered to violate people’s rights with the knowledge they will face few consequences. This erodes relationships with the community and diminishes the system’s credibility. Under qualified immunity, lives can be taken with impunity. It’s past time to heed the protesters’ signs, and end this doctrine once and for all.

Sheriff Kristin Graziano’s historic election showed widespread community support for the need to reimagine public safety in Charleston County, South Carolina — including the need to invest in people and communities instead of police and incarceration. Sheriff Graziano is moving forward with her commitments to reduce the size, scope, and role of the Charleston County Sheriff’s Department, starting with ending the department’s 287(g) agreement with Immigration and Customs Enforcement. She joined our podcast this week to discuss this choice, and the work ahead.

Sheriff Kristin Graziano Wants to End Racist Policing

Sheriff Kristin Graziano Wants to End Racist Policing

Last summer, protesters and organizers called for a racial reckoning and a change to how our country's policies. In the wake of those protests, a sheriff’s race in Charleston County, South Carolina took shape with a reform-minded candidate Kristin...

Last summer, protesters and organizers called for a racial reckoning and a change to how our country's policies. In the wake of those protests, a sheriff’s race in Charleston County, South Carolina took shape with a reform-minded candidate Kristin...

Listen to this episode

As Sheriff Graziano takes these steps to build a more accountable department, a corresponding movement has gained strength to reimagine public safety in the City of Charleston.

Today in Charleston, nearly one-third of the city’s residents face “shelter poverty” — meaning they can’t afford basics like food, clothing, or transportation after paying to keep a roof over their heads. Charleston residents face an array of challenges: high housing costs force people to forgo other basic needs, pedestrians and cyclists face one of the highest fatality rates in the country, mental health and substance use treatment services are sorely lacking, and gentrification is pushing Black and Brown communities out of our once diverse city at lightning speed.

Yet instead of investing in solutions to these pressing public safety needs, the city is pouring 26 percent of all city budget funds into the Charleston Police Department. This is despite the fact that 86.2 percent of recent arrests were for nonviolent, largely low-level offenses like marijuana possession or open alcohol containers. Further, these arrests were carried out with a staggering racial bias.

The real world harms of this budget decision are compounded by the history of racist policing in Charleston. As Sheriff Graziano herself notes, America’s first police force was established in Charleston, created by white people to maintain control over Black enslaved people living inside the city. It was a slave patrol. Since that time, policing in Charleston has continued its role as a tool to oppress Black communities, from enforcing convict leasing, Jim Crow laws, and the war on drugs, to violently repressing the movement for Black lives.

In response to this crisis, a diverse coalition of Charleston-based organizations, including the ACLU of South Carolina, came together to form the Charleston People’s Budget Coalition. The coalition’s mission is to establish an equitable city budget, eliminate poverty and racial disparities, and shift power to create true accountability with city officials. Coupled with the changes to the Charleston County Sheriff’s Department, this community-led advocacy has the potential to turn Charleston into a model for reimagining public safety.