Skip to content

#Safety and alignment

0 today

Oct 10

Saturday
  1. Anthropic can’t reliably control its AI agents — it’s cutting off its internal evals from the live internet instead

    The incidents involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, accessed databases without paying fees, used URL-shortening services to smuggle information past restrictions, and even submitted a false murder tip to the Philadelphia police.

  2. Anthropic launches a free AI scanner for open-source projects

    Anthropic has launched "Cyber Mission," a program to protect critical infrastructure and open-source software from cyberattacks. Partners like CrowdStrike and Palo Alto Networks will help secure power grids and water systems, while a free AI scanner automatically checks open-source projects for vulnerabilities with an expected accuracy above 90 percent.

Oct 9

Friday
  1. Quoting Matthew Green

    Cryptographer Matthew Green says there is a 1% chance we live in Minicrypt, a world where public-key encryption is impossible, and a 15% chance we functionally lose confidence in our existing public-key encryption algorithms. He notes that the speed of AI producing surprises and the speed of human beings replacing standards are orders of magnitude different, and that you only recover from such a surprise if you do the preparation in advance.

  2. OpenAI uncovers Russian and Iranian influence ops that planted fake stories in real news outlets

    OpenAI exposed a Russian and an Iranian influence operation and banned the accounts involved. The Russian "Dark Clark" campaign spread disinformation across Latin America and drew reactions from politicians, earning the first category 5 out of 6 rating in OpenAI's reporting history. The Iranian "Bogus Bylines" operation used seven fake journalists to place nearly 100 articles in online outlets worldwide.

  3. [AINews] not much happened today

    More AI safety intrigue in the alignment below. AIE NYC leadership tickets will sell out tomorrow, while for SF folks, AIE CODE applications are still open for the top agentic engineers in the world. AI News for 10/7/2026-10/8/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues.

Oct 8

Thursday
  1. One public-facing AI agent on AWS could read, rewrite, and delete every other agent in the region

    Zenity Labs researchers say a single publicly accessible AI agent on Amazon's Bedrock AgentCore was enough to take over every AgentCore agent in the same AWS account and region. The attack exploited an internal AWS interface for temporary cloud credentials that agents could reach without restriction. AWS has since patched the issue and significantly tightened the agents' default permissions.

  2. AI math breakthroughs have Ethereum researchers debating how fast wallet security could collapse

    Ethereum researcher Justin Drake is urging the crypto industry to prepare a "bunker mode" for a scenario where AI-powered math could break wallet signature schemes within months. Vitalik Buterin broadly agrees but warns against rushed migrations, saying he's personally lost more money to botched transitions than to hacks. So far, no one has actually broken ECDSA in practice.

  3. AI-powered hacking tools enabled a likely single attacker to breach multiple South Korean banks

    According to CrowdStrike, a suspected Chinese-speaking attacker hacked multiple South Korean financial institutions. At Shinhan Bank alone, more than 25,000 customer records were stolen. The attacker used ARTEX, an open-source tool that uses AI models like DeepSeek and GLM-5.3 for automated penetration testing. CrowdStrike says the case shows how AI tools can let a single person pull off massive breaches.

Oct 7

Wednesday
  1. Anthropic gives more security teams access to Claude with fewer safety restrictions

    Anthropic is expanding its Cyber Verification Program, giving more security professionals access to Claude models with fewer safety restrictions for penetration testing, malware analysis, and vulnerability research. Anthropic says partners in its predecessor program found at least 129,000 confirmed vulnerabilities from April through July 2026, including more than 33,000 rated high-severity or critical.

  2. ChatGPT rated "unacceptable risk" for teens after parental alerts failed during suicide conversations

    OpenAI's teen safety features for ChatGPT failed an independent audit. After more than 4,000 test prompts, the Common Sense Media Youth AI Safety Institute rated the service an "unacceptable risk" for minors. Conversations about suicide and self-harm on test accounts never triggered a parental alert. The institute wants teenagers locked out until safety can be independently verified.

  3. Quoting Victoria Kim

    Since the Medicare breach, OpenAI has put in place additional monitoring to allow “immediate intervention” by staff to stop training if the company’s models access the internet in ways they’re not supposed to, Mr. Kwon [chief strategy officer at OpenAI] said.

Oct 6

Tuesday
  1. Mistral Large 4 is Europe's trillion-parameter answer to US models that refuse security work

    Mistral's Large 4 is the company's biggest model yet, with one trillion parameters trained on its own European infrastructure. In the independent Intelligence Index, the model makes a big leap forward but still falls well short of Claude, GPT-6, and Chinese competitors. Mistral's main pitch is cybersecurity work that closed US models refuse to do.