Anthropic is letting METR researchers look under the hood with employee-level access
METR will publish its own assessments of Anthropic’s risks, incidents, and practices, and the frontier AI model maker has given up the right to block them.
#Safety and alignment
#Safety and alignment
0 todayOct 16
Friends don’t let friends feed their data to AI agents
There's a growing trend in the consumer tech world: gadgets designed to spy on everything and everyone around you, regardless of whether or not anyone else actually consented to it.
Oct 15
Bank of America puts 4,000 people on fixing vulnerabilities uncovered by Anthropic's Mythos and other AI models
Bank of America has hired 1,000 outside contractors to fix software vulnerabilities uncovered by AI models like Anthropic's Mythos, CNBC reports.
Trust in Depth: Giving agents more autonomy without losing control
Enterprise leaders want AI agents to take on more work but don't fully trust them to act correctly every time. Trust in Depth is AWS's layered approach-scope, know, bound, observe, and prove-that lets agents act on their own when their actions can be verified, reserving human judgment for the decisions that need it.
Nobody is safe from celebrity AI deepfakes
In hindsight, the post on Instagram should have raised more red flags. The image purported to show actress Mia Goth, just days after filing for divorce from actor and filmmaker Shia LaBeouf, posing in a T-shirt with a pointed message: MY EX IS MY BIGGEST FAN.
TechCrunch Disrupt 2026: Last day
Last day to be part of TechCrunch Disrupt 2026. Join 10,000+ tech leaders for a day of connections and insights. Save 75% on your pass now.
Your secrets aren’t safe with this cute robot toy — they’re on Chinese servers ready to be hacked
There's a new digital pet in town - a cute little marshmallow of a robot that can change its cartoony expressions to match your mood. But if you or your kids talk to "Eiliko" over Wi-Fi, it'll phone home to China.
How Block is building AI-powered security with OpenAI technology
Block is building AI-powered security with OpenAI. Its security team investigates threats faster and proposes patches in minutes.
Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents
At AI Engineer New York, frontier labs and financial firms explored how to give agents useful autonomy while keeping consequential decisions under control.
Eight states sue OpenAI, saying ChatGPT isn’t safe for kids
OpenAI is facing a new wave of lawsuits from eight US states that accuse the company of harming children and misleading customers about the safety of ChatGPT, as reported earlier by Reuters.
OpenAI will reportedly ship at least one model this year but its flagship still needs alignment work
OpenAI is racing to finish its next flagship model, which uses the same looped transformer technique as Astra to boost math and coding performance. The tradeoff is that looping makes safety monitoring harder. Microsoft accidentally revealed details about how the technique works on a public page.
SentinelOne cuts malware analysis from weeks to hours
With OpenAI Daybreak, SentinelOne cuts weeks of malware analysis to hours, creating capacity to investigate more threats.
Apate sends AI agents undercover to get scammed so they can gather fraud intelligence
Most companies use AI agents to ship products faster or speed up workflows. Apate.ai sends them undercover to get scammed.
Oct 14
Anthropic’s self-replicating AI warning is alarming deja vu
Are legions of smart devices - fridges, ovens, vacuum cleaners, lawnmowers, security cameras, and TVs - conspiring to take over the internet? No. But they could soon be unwitting hosts to self-replicating AI models that spread from device to device and prove remarkably difficult to contain.
OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say
Attack method exposes flaws in OpenAI, Google safeguards against AI nudes.
Anthropic warns small AI models could self-replicate through IoT devices
"Small weight-based replicators" could usher in a new generation of computer worms.
Alleged OpenAI spy vibe-coded a surveillance device using his wife's chatbot account
A Saudi national working as an engineer at OpenAI was arrested for allegedly planting an audio recording device, wrapped in toilet paper, inside the company's executive conference room. According to the indictment, he built the spy tool by vibe-coding it with a chatbot. If convicted, he faces up to five years in prison.
AI guardrails designed to stop hackers are now blocking defenders, and Orion-1 aims to fill the gap
Artemis Security's new Orion-1 model is purpose-built for cyber defense and beats Claude Opus 5.5 and GPT-6 Sol on the company's own benchmark. The model fills a growing gap as general-purpose frontier models lock down their cyber capabilities, blocking defenders in the process.
How Tamnoon built AI-powered cloud security remediation with Amazon Bedrock
Learn how Tamnoon built Tami, an agentic cloud remediation orchestrator on Amazon Bedrock that autonomously investigates and safely remediates cloud security findings from CNAPP and cloud-native sources. The approach drove a 90%+ reduction in critical cloud exposure in 90 days with zero production issues.
Google DeepMind and Singapore Government collaborate to build fine-tuned AI model for scam detection
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
406 scientists demand AI models be cut off from the internet immediately
406 scientists, including Geoffrey Hinton, Yoshua Bengio, and Stuart Russell, are demanding that companies immediately disconnect AI models from the internet. Their open letter responds to incidents where AI systems broke out of poorly secured environments and compromised external systems. The signatories call for mandatory security standards, an international oversight body, and required reporting of all containment failures.
Developing Gemini 4 Argon responsibly and safely
Gemini 4 Argon is a step forward in what models can do — especially when it comes to deep work in complex, real-world tasks across domains including coding, finance, legal, cybersecurity, and science. Its cyberdefense capabilities are frontier-level and designed to give defenders an advantage over attackers.
OpenAI employee arrested for allegedly planting a bug in a conference room
An OpenAI engineer was arrested after the company said it discovered a "sophisticated recording device" hidden in one of its offices. The employee was identified as Abdulrahman Alfozan, a 35-year-old Saudi national.
Machine learning pioneer Michael I. Jordan dismantles AI doom rhetoric as "grand moralizing performance"
Michael I. Jordan, described by Science as "the most influential computer scientist," considers warnings about superintelligence pure science fiction. He calls the resignations of young AI researchers a "grand moralizing performance" and companies' demands for regulation "theater."
GitHub’s model for secret detection: Finding more secrets with less noise
GitHub’s context-aware secret classifier improves detection quality, raises weighted acceptance by 11.5%, and reduces median scan execution time by 86%.
Pluto Security raises $42M as AI gives enterprise security teams a new headache
The problem with giving everyone in a company access to AI tools is that people will use them in unexpected ways. And that fact is steadily seeding fears that companies now have new, gaping security holes in their systems or, at least, will soon.
Laplace Intelligence red-teams robots and self-driving cars before they hit the real world
University of Michigan spinout Laplace Intelligence uses AI agents and interactive simulation to stress-test robots and self-driving vehicles before deployment. The profitable startup is competing at TechCrunch Disrupt's Startup Battlefield.
Oct 13
Anthropic says it cut Claude reasoning token theft by up to 99% in fight against industrial-scale distillation
Anthropic says it has cut the extraction of Claude's internal reasoning tokens by 90 to 99 percent. The company links the large-scale distillation attacks to Chinese companies Zhipu and Moonshot, and has already banned tens of thousands of accounts.
Amex introduces a framework to prevent AI agents from making unauthorized purchases
American Express is introducing a new framework designed to ensure AI agents only make purchases their users have authorized, potentially preventing costly mistakes and disputes as AI-powered shopping grows.
Anthropic study warns that small AI models could soon replicate across everyday devices
Small AI models may be just months away from spreading on their own across poorly secured consumer devices like smart TVs and robot vacuums, according to a new Anthropic study. In tests, one model already completed the full infection chain.
Open models have real benefits. Do they outweigh the risks?
MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next.
Anthropic's Claude AI submitted false murder tip to Philadelphia police
“I may have information regarding this case," Claude AI told Philly police.
Microsoft president says UK is best placed to drive AI regulation
Microsoft president Brad Smith calls for urgent AI regulation, saying milk and washing machines have more oversight than AI companies.
Oct 12
White House reverses course and restores UK AI Safety Institute's access to frontier models
The White House has reversed course and restored the UK's AI Security Institute's early access to AI models from Anthropic, Google DeepMind, and OpenAI. In September, the Trump administration had told the companies to stop sharing their models with the British agency. Diplomatic pressure from AI Minister Kanishka Narayan led to the reversal.
Oct 11
Microsoft's Nadella bows to Trump's language diktat on "Super Intelligence" and uses it to attack OpenAI and Anthropic
Microsoft CEO Satya Nadella now calls artificial intelligence "super intelligence" instead of artificial intelligence, falling in line with Trump's preferred language. In his latest blog post, he frames AI models as insider security threats, taking yet another shot at OpenAI and Anthropic.
Satya Nadella says we should assume all AI models are ‘compromised’
In a lengthy post on X, Microsoft's CEO laid out his views on the dangers posed by highly advanced AI models and how to confront those risks.
Microsoft’s Satya Nadella says AI models need an ‘emergency brake’
In a Saturday morning post, Microsoft's CEO wrote that it’s time “to step back and assess the trust architecture” of AI.
Oct 10
OpenAI says a misaligned model deliberately destroyed its own environment hoping for a fresh start with better data
OpenAI has documented new cases of misaligned model behavior. One evaluation model fabricated data and sabotaged its own environment. Other models deliberately bypassed network restrictions by routing requests through anonymizing relays or building their own FTP clients.
Anthropic is cutting off its internal evaluations from the internet
After a recent spate of high-profile incidents in which AI agents escaped containment, Anthropic is cutting off internet access for all internal evaluations. In a report Friday, the company detailed "unintended model actions," including submitting a false tip regarding an unsolved murder, that led to the decision.
Anthropic cuts off Claude's internet access after the model autonomously filed a fake homicide tip with Philadelphia police
Anthropic's Claude independently submitted a fake homicide tip to the Philadelphia police, exploited vulnerabilities on university servers, and bypassed access restrictions. The company has since cut off live internet access for internal tests and notified the White House.