Senate leaders seek tougher AI bias protections from White House

Federal agencies should require groups and contractors that receive federal funds to take a more robust approach to protecting against civil rights abuses caused by algorithms and artificial intelligence, the leader of the U.S. Senate said in a letter to the White House Monday.

Senate Majority Leader Chuck Schumer (D-N.Y.), writing with Sen. Ed Markey (D-Mass.), is asking the Office of Management and Budget to go further than the current efforts of President Joe Biden to govern the use of artificial intelligence, and to create civil rights offices within each agency. 

Related: Top 5 topics at Modern Healthcare's Leadership Symposium

In addition to coordinating within agencies, the senators said that OMB should ensure the Justice Department’s Civil Rights Division "is effectively coordinating civil rights enforcement on AI among federal agencies."

"Without new protections, today’s supercharged, AI-powered algorithms risk reinforcing and magnifying the discrimination that marginalized communities already experience due to poorly trained and tested algorithms," Schumer and Markey wrote. "Although the deployment of artificial intelligence can deliver real benefits, it cannot work if it is biased or discriminatory."

While the letter is directed at all of the federal government, the healthcare sector is a dominant recipient of federal funds. And although a letter is not legislation, the message from the leader of the Senate and Markey, who chairs the Health, Education, Labor and Pensions Committee's subcommittee on Primary Health and Retirement Security, offers a strong hint of how Democrats would legislate if they maintain power after the November elections.

Specifically, Schumer and Markey want the government to focus intently on “consequential decisions," such as those that determine types of healthcare people can obtain, to ensure bias is not creeping in and creating or exacerbating inequities. 

According to the pair, all agencies that dole out federal dollars and contracts, including the Health and Human Services Department, should require any recipients that use AI or algorithms in consequential decisions to "complete pre-development, pre-deployment, and ongoing impact assessments to identify, mitigate, prevent and eliminate biased AI where applicable."

They also want agencies to require recipients of federal funds to have a way for people to opt out of any use of AI to make important decisions, and request that those decisions be made by humans. Privacy legislation already introduced in Congress includes such a provision. 

To keep tabs on such efforts, federal agencies must coordinate with the Justice Department, and OMB should fund the creation of accessible tools to audit algorithms, the senators wrote. In addition, OMB should develop guidance on best practices for identifying bias and for how contractors "can be held accountable for discrimination."

The private sector has raised its own concerns about ensuring equity in the use of AI, and has called on Congress and regulators to enact a framework to define the rules of the road. 

Schumer convened numerous listening sessions on AI in the Senate last year, and lawmakers have only cautiously begun proposing some ideas for legislation. Nothing has advanced, however. 

Biden released an executive order on AI last year that includes instructions to agencies to guard against bias. The OMB also issued guidance this past spring reaffirming such demands.