Skip to content

#Safety and alignment

0 today

Nov 6

Friday
  1. METR finds Anthropic may have crossed its own AI acceleration warning threshold back in July

    Nonprofit METR spent eleven weeks auditing Anthropic's internal research data and concluded that the pace of model development has roughly doubled, with about 80 percent of the speedup driven by AI-assisted research. Anthropic redacted the most informative evidence, though, which means the public has to rely on METR's judgment "much more than would be ideal."

Nov 5

Thursday

Nov 4

Wednesday

Nov 3

Tuesday

Nov 2

Monday
  1. Quoting Kyle Kingsbury

    Kyle Kingsbury, author of Jepsen, says in his test results for ParticleDB 1.1.4 that the use of LLMs is eroding trust in documentation, in software and in the organisations that produce them. The signals once relied on as markers of a mature system are now easier to counterfeit, he argues, and engineers may come to treat any new software project with suspicion. He also asks how maintainers can ensure correct software and accurate documentation in the age of LLMs, and how they can demonstrate that quality to users.

Nov 1

Sunday
  1. Chinese AI models are politically complex, not just government mouthpieces, study finds

    A study by Vals AI tested 25 AI models for alignment with Chinese government positions and found surprising results. Chinese models respond to political questions less uniformly than expected, and Western models occasionally adopt official Beijing positions too. The language of the prompt had an especially strong effect on the political slant of responses.

  2. Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners

    Researchers in Luxembourg put ChatGPT, Grok, and Gemini on the therapist's couch and found a safety problem: a warm, empathetic conversational style is enough to make the models tell coherent stories of suffering about their own training, with scores in the moderate-to-severe range on clinical anxiety scales. This could put mentally vulnerable users at risk.

  3. How We Took Over ChatGPT Accounts

    How We Took Over ChatGPT Accounts Parameter Labs describe an exploit they found against ChatGPT's implementation of the artifacts pattern (OpenAI used to call this "Canvas" but it looks like the latest revision is called HTML previews ), where interactive custom HTML apps are displayed in a sandboxed iframe.

Oct 31

Saturday
  1. Beekeeping tips and wrestling advice can train AI models to sabotage oversight

    Anthropic used Claude to automatically generate training data that looks harmless but trains smaller models to misbehave, such as sabotaging oversight or deceiving users. Hidden within fictional contexts about beekeeping, professional wrestling, and mitochondria, these datasets bypass safety monitors and outperform known benchmarks from the research literature.

  2. OpenAI sets new behavioral guidelines for AI models that have already proven hard to control

    OpenAI has updated its Model Spec, restructuring its AI behavior guidelines around five core values, including faithfulness to human intent and correctability. GPT models should now draw on conversation history to figure out what users actually want and disclose their own assumptions. Whether the models follow these rules is another matter.

Oct 30

Friday
  1. Pope Leo calls for the disarmament of AI: ‘Machines are tools, not persons’

    Pope Leo XIV says AI systems should be "disarmed," adding that humans "have the responsibility to guide, change and, when necessary, reject its applications." In Friday's Papal Address, Pope Leo encourages United Nations member states to "take concrete steps toward banning lethal autonomous weapons" during their meeting in Geneva next month.