Algorithms Are Making Government Decisions. The Public Needs to Have a Say.
AI and automated decision systems are reshaping core social domains, from and , to and beyond. Yet it remains incredibly difficult to assess and measure the nature and impact of these systems, even as research has shown their potential for biased and inaccurate decisions that harm the most vulnerable. These systems often function in oblique, invisible ways that are not subject to the accountability or oversight the public expects.
Consider how a lack of such public oversight hit the New Orleans community. In 2012, the New Orleans Police Department contracted with the data analytics company Palantir to build a state-of-the-art predictive policing system, designed to help the police identify people in the New Orleans community who are likely to commit violence or become the victim of violence.
The accuracy and usefulness of such predictive policing and “heat mapping” approaches are very much in question. Recent research has that predictive policing has great potential to disparately impact communities of color, amplifying existing patterns of discrimination in policing. has raised doubts about whether predictive policing is effective at all.
The rise of automated decision systems has already had and will continue to have an impact on the most vulnerable people.
This controversial and potentially biased system was put in place with no oversight. Until a report in last month, even members of the New Orleans City Council had no idea what their own police department was doing.
Other jurisdictions are similarly grappling with the lack of oversight over invisible automated systems. Former New York City Council Member James Vacca, sponsor of the legislation forming NYC’s new automated decision system task force, his own lack of insight into how the city is using automated decision technologies as reason for drafting the bill.
This is why we at AI Now released a report on Monday for “Algorithmic Impact Assessments.” AIAs provide a strong foundation on which oversight and accountability practices can be built, by giving policymakers, stakeholders, and the public the means to understand and govern the AI and automated decision systems used by core government agencies.
Algorithmic Impact Assessments would first give the public the basic knowledge it needs through disclosure. Before procuring a new automated decision system, agencies would be required to publicly disclose information on the system’s purpose, reach, and potential impact on legally protected classes of people.
Beyond such disclosure, agencies would also be required to provide an accounting of a system’s workings and impact, including any biases or discriminatory behavior the system might perpetuate. Given the many contexts and many types of systems, this would be accomplished not through a one-size-fits-all audit protocol, but by engaging with external researchers and stakeholders, and ensuring that they have meaningful access to an automated decision system.
These external researchers must include people from a broad array of disciplines and experience. Take, for example, the Allegheny Family Screening Tool, a tool used in Allegheny County, Pennsylvania to help judge the risk that a child might face abuse or neglect. Researchers with different toolsets have yielded on how the tool makes predictions, how employees of the Allegheny Department of Human Service to make decisions, and people subject to those decisions.
Finally, agencies would need to honor the public’s right to due process. This means ensuring that meaningful public engagement is integrated into all stages of the AIA process before, during, and after the assessment through a “notice and comment” process, through which agencies solicit public feedback on their assessments. This would be a chance for the public to raise their concerns and, in some cases, even challenge whether an agency should adopt a particular automated decision system. Additionally, if an agency fails to adequately complete an AIA, or if harms go unaddressed by the agency, the public should have some method of recourse.
In developing AIA legislation, lawmakers will need to address several points. For example, how should external researchers be funded for their efforts? And what should agencies do when private vendors that sell automated decision systems resist transparency? Our position is that vendors should be required to waive their trade secrecy claims on information required for exercising oversight.
The rise of automated decision systems has already had and will continue to have an impact on the most vulnerable people. That’s why communities across the country need far more insight into government’s use of these systems, and far more control over which systems are used to shape their lives.
This piece is part of a series exploring the impacts of artificial intelligence on civil liberties. The views expressed here do not necessarily reflect the views or positions of the ACLU.
Will Artificial Intelligence Make Us Less Free?
Artificial intelligence is playing a growing role in our lives, in private and public spheres, in ways large and small.
Source: American Civil Liberties Union
The Rise of Platform Authoritarianism
AI could allow companies to make an end-run around antidiscrimination law.
Source: American Civil Liberties Union
Learn More About the Issues on This Page
-
Press ReleaseDec 2024
Free Speech
National Security
ACLU and Partners Urge Supreme Court to Block TikTok BanACLU and Partners Urge Supreme Court to Block TikTok Ban
WASHINGTON — Today, the American Civil Liberties Union, the Electronic Frontier Foundation (EFF), and the Knight First Amendment Institute at Columbia University filed an amicus brief urging the Supreme Court to block the enforcement of a law that would effectively ban people in America from using TikTok as soon as January 19, 2025. Earlier this month, the U.S. Court of Appeals for the D.C. Circuit rejected TikTok’s challenge to the law. TikTok has asked the Supreme Court for a temporary injunction to prevent the app from being banned while the court considers whether to take the case, saying that unless the justices intervene, the law will “shutter one of America’s most popular speech platforms the day before a presidential inauguration.” “The Constitution imposes an extraordinarily high bar on this kind of mass censorship,”said Patrick Toomey, deputy director of ACLU’s National Security Project. “The Supreme Court should take up this important case and protect the rights of millions of Americans to freely express themselves and engage with others around the world.” The brief argues that the D.C. Circuit failed to fully address the law’s profound implications for the First Amendment rights of the 170 million Americans who use TikTok. While the lower court’s decision correctly recognized that the statute triggers First Amendment scrutiny, it barely addressed users’ First Amendment interests in speaking, sharing, and receiving information on the platform. The court also perplexingly attempted to cast the government’s ban on TikTok as a vindication of users’ First Amendment rights, which it is not. The rights groups also explain that the law was intended to suppress certain content and viewpoints that many legislators believe could be amplified on TikTok, including the risk of foreign “propaganda.” But under the First Amendment, the government must meet a very high bar to restrict speech based on concerns about its “motivating ideology” or “perspective,” and the government has not come close to meeting that bar here. “The government should not be able to restrict speech, especially to the extent here, based on guessing about the mere possibility of uncertain future harm,” said David Greene, civil liberties director at EFF. “The Supreme Court should put the TikTok ban on hold while it considers the DC Circuit’s erroneous ruling.” Finally, the brief underscores that the government can’t impose this type of sweeping ban unless it’s necessary to prevent extremely serious and imminent harm to national security. But the government has not provided evidence of impending harm, or evidence that banning TikTok is the only available way to address its concerns. As the brief explains, the D.C. Circuit improperly treated the government’s invocation of “national security” as a trump card and failed to hold the government to its burden. “Restricting citizens’ access to foreign media is a practice that has long been associated with repressive regimes, and we should be very wary of letting the practice take root here,” said Jameel Jaffer, executive director at the Knight First Amendment Institute at Columbia University. “It would do lasting damage to the First Amendment and our democracy if the Supreme Court let this ban go into effect even temporarily.” You can find the brief online here: https://www.supremecourt.gov/DocketPDF/24/24A587/335380/20241217144322392_24A587%20TikTok%20v%20Garland%20Amicus%20Brief%20pdfa.pdf -
Press ReleaseDec 2024
National Security
Religious Liberty
ACLU Statement on New White House Strategy to Counter IslamophobiaACLU Statement on New White House Strategy to Counter Islamophobia
WASHINGTON – The Biden administration today released the first-ever national strategy to counter Islamophobia and related forms of bias and discrimination, including hate against Arab, Sikh, and South Asian Americans. In advance of the strategy, American Civil Liberties Union and its partners had urged the administration to overhaul government programs that reflect anti-Muslim discrimination. In particular, we have called for urgent action to constrain governmental agencies from continuing to exercise their authorities and technology to wrongly surveil and investigate, watchlist, and question and detain Muslims at the border, as well as deny immigration benefits to people from Muslim-majority countries. While the White House raised expectations that many of these issues would be addressed, the final strategy ended up falling far short. The following is a statement from Hina Shamsi, director of the ACLU’s National Security Project: “While this strategy acknowledges discrimination and its harms, it does little to end them and is a squandered opportunity. For decades, American officials have invoked national security to pass laws and implement programs that disproportionately harm Muslims and people perceived to be Muslim. A serious anti-discrimination strategy would concretely address multiple bias-infused government practices that deny our communities equal participation in civic life and our democracy, like federal watchlisting, surveillance, and investigation. We’re profoundly frustrated that the administration didn’t take even the basic, overdue step of recognizing that anti-Muslim discrimination is uniquely normalized and embedded in government policies.” -
VirginiaDec 2024
National Security
Trabelsi v. Crawford, et al. – Lawsuit Challenging Unlawful Detention and Inhumane Treatment of Acquitted ManTrabelsi v. Crawford, et al. – Lawsuit Challenging Unlawful Detention and Inhumane Treatment of Acquitted Man
Our client, Nizar Trabelsi, is in the United States against his will. The federal government brought him here from Belgium more than a decade ago and charged him with terrorism-related crimes. At trial, the government’s case failed: a federal jury found Mr. Trabelsi not guilty. But instead of allowing Mr. Trabelsi to return to Belgium after his acquittal, the United States placed him in highly restrictive immigration detention and began an ongoing effort to force him to Tunisia, where he was born and where he will very likely be tortured. Mr. Trabelsi’s detention violates the Constitution, immigration law, and the extradition treaty between the United States and Belgium. Through this lawsuit, he seeks to return to Belgium, and he demands an immediate improvement of his detention conditions. -
Press ReleaseDec 2024
National Security
Immigrants' Rights
Federal Judge Rejects Acquitted Man’s Argument That His Immigration Detention Is UnlawfulFederal Judge Rejects Acquitted Man’s Argument That His Immigration Detention Is Unlawful
ALEXANDRIA, Va. — This week, the U.S. District Court for the Eastern District of Virginia ruled that the court does not have jurisdiction to decide whether the government can continue to detain Nizar Trabelsi, a Tunisian national who was brought to the United States against his will for prosecution and was cleared of all charges last year. The ruling means the United States will be able to continue to detain Mr. Trabelsi while it attempts to deport him to Tunisia, where he was born and where, as an immigration judge ruled earlier this year, he will likely face torture. The court also concluded that even if it did have jurisdiction over Mr. Trabelsi’s case, his claims challenging the government’s detention of him would fail. “Reviewing claims that the government is holding someone unlawfully is at the core of the judicial function, and we’re heartbroken for our client that the court got this so wrong,” said Brett Max Kaufman, senior staff attorney with the ACLU’s Center for Democracy. “The government’s arguments in defense of Mr. Trabelsi’s detention makes a hash of the Constitution, immigration laws, and its own extradition treaty.” In 2013, the United States forcibly extradited Mr. Trabelsi from Belgium to face criminal charges in the United States. In July 2023, after almost 10 years of highly restrictive pretrial detention, a federal jury cleared Mr. Trabelsi of all charges. But instead of releasing Mr. Trabelsi or returning him to Belgium, the U.S. transferred him to immigration detention, wrongly treating him as an applicant for admission and placing him in the deportation process. Over the years, the Belgian government has issued multiple formal diplomatic requests asking the U.S. to facilitate his return, but the U.S. has refused to send him back. Mr. Trabelsi filed a lawsuit challenging the government's authority to detain him. He also sought immediate improvements to his detention conditions. Those claims were not part of the court’s ruling today, and will continue to be litigated on a separate track. “Mr. Trabelsi only wants to return to Belgium after being illegally extradited to the United States, held for ten years, and then acquitted by a jury of any crimes,” said Nicole Hallett, clinical professor of law and director of the Immigrants’ Rights Clinic at the University of Chicago Law School. “He cannot be held indefinitely and we will continue to fight to make sure that justice ultimately prevails.” Mr. Trabelsi is represented by the American Civil Liberties Union, the University of Chicago Law School’s Immigrants’ Rights Clinic, the ACLU of Virginia, and Professor Jonathan Hafetz of Seton Hall Law School. The complaint in Trabelsi v. Crawford was filed against Jeffrey Crawford, warden of the Farmville Detention Center where Mr. Trabelsi is being held; Liana Castano, ICE field office director for the Washington Field Office; Alejandro Mayorkas, secretary of the Department of Homeland Security; and Merrick Garland, U.S. attorney general.