Anthropic rejects open-weight AI bans, calls for China chip controls and safety tests

30/07 21:15 - Anthropic rejects open-weight AI bans, calls for China chip controls and safety tests
Anthropic CEO Dario Amodei has argued that policymakers should keep lower-risk open-weight AI accessible while placing stricter safeguards around frontier systems, including mandatory testing and limits on China’s access to advanced computing and model capabilities. In a post outlining Anthropic’s position, Amodei said broad restrictions, including bans on Chinese open-weight models used by US businesses, would not address his main national security concerns. Instead, he pointed to the possibility of authoritarian governments surpassing the US in advanced AI, as well as cyber, biological, and alignment risks posed by increasingly capable systems. Amodei also called for action against industrial-scale model distillation, which he said allows Chinese developers to improve their models with less computing power than would be needed to train comparable systems from scratch. The statement followed criticism of Anthropic for not signing an industry letter backed by Nvidia, Microsoft, Meta, IBM, Mistral, Hugging Face and other technology companies urging policymakers to avoid premature restrictions on open-weight models. The letter said that open weights could broaden access to AI, intensify competition, and enable organizations to adapt and deploy models without relying on a single provider. Amodei agreed with parts of that case but disputed claims that openness inherently improves safety research or gives defenders an advantage over attackers. He said regulation should be based on a model’s capabilities and risks rather than whether its weights are openly available. Under that approach, sufficiently capable open and closed models would undergo testing before release. Conditional support Analysts said Anthropic had moved closer to industry consensus by rejecting blanket bans, but its support remained more limited than the approach backed by many major technology companies. Deepika Giri, head of research for AI, analytics, and data at IDC, said the Nvidia-backed letter presented open weights as strategic infrastructure that should remain broadly accessible, in contrast with Anthropic’s more restrictive position. Amodei’s statement clarified that Anthropic supports open-weight models only under certain conditions, a stance that could also help the company preserve its competitive advantages as a proprietary model provider focused on compliance and tighter controls, according to Lian Jye Su, chief analyst at Omdia. The statement was “a real olive branch” to supporters of open-weight models, according to Pareekh Jain, CEO of Pareekh Consulting. But he said the disagreement had shifted from whether such models should be released to where policymakers should draw the line. “Anthropic still thinks that once a model gets powerful enough, releasing its weights publicly is riskier than keeping it locked behind an app, because you can never take it back or add safety fixes later,” Jain said. Will the controls work? Analysts differed over whether Anthropic’s proposed controls would achieve their aims without creating new barriers for smaller AI developers. Jain said chip restrictions and measures against illicit model distillation would mainly affect model developers and infrastructure providers, rather than enterprises using models already on the market. Mandatory safety testing, however, could raise development costs and reduce the number of advanced open-weight models available. “Testing is expensive and time-consuming, and so, giant, well-funded companies like Anthropic, Google and OpenAI can afford it,” Jain said. Smaller developers seeking to release cutting-edge open-weight models could struggle to meet the same requirements, he added. The additional testing and screening could also restrict the number of open-weight models available to enterprises, according to Su. He said the requirements could weaken some of their principal benefits, including lower costs, reduced vendor dependence and community-led development. Anand Joshi, managing director of market research firm JP Data, questioned whether limiting China’s access to advanced chips would materially slow its AI development, arguing that Chinese companies had shown they could build highly capable models with less computing power. He supported action against illicit distillation, however, saying safeguards were needed to prevent developers from reproducing the capabilities of other models without authorization. The impact on most enterprise users could remain limited if less capable models were exempted, Jain said. Businesses deploying models that fall below the proposed testing threshold would probably face little additional cost. How CIOs should choose Giri said CIOs should assess models according to their capabilities rather than whether they are open, and should demand independent testing, clear licensing, model documentation and accountability for monitoring and incident response. “Mandatory safety testing should be triggered by a model’s demonstrated capabilities, not its size or training cost,” Jain said, particularly when a system could significantly assist cyberattacks, biological misuse, or autonomous harmful actions. Before deployment, CIOs should seek independent evaluations, detailed model documentation, security test results and information about the model’s software supply chain, he added. Charlie Dai, principal analyst at Forrester, said that assessment should include documented red-team results, model provenance, disclosures about training and fine-tuning, and evidence of independent testing against recognized safety benchmarks. ...