Skip to content

#Data and training

2 today

Apr 30

Thursday

Apr 22

Wednesday

Apr 16

Thursday

Apr 1

Wednesday

Mar 25

Wednesday

Mar 18

Wednesday
  1. Introducing Forge

    Mistral AI launched Forge, a system for enterprises to build frontier-class AI models on proprietary knowledge, supporting pre-training, post-training and reinforcement learning. It handles dense and MoE architectures and, when needed, multimodal inputs, with training and governance on companies' own infrastructure.

Mar 17

Tuesday

Mar 13

Friday
  1. Identifying Interactions at Scale for LLMs

    Berkeley AI Research proposed SPEX and ProxySPEX to identify key interactions driving LLM outputs at scale in feature, data and model-component attribution. SPEX turns interaction search into sparse recovery using sparsity and low order; ProxySPEX exploits hierarchy to match SPEX with roughly ten times fewer ablations.

Mar 12

Thursday

Mar 7

Saturday

Jan 10

Saturday
  1. Information-Driven Design of Imaging Systems

    Berkeley AI Research proposed a mutual-information framework to evaluate and optimise imaging from noisy measurements and noise models. A NeurIPS 2025 paper validates decoder performance predictions for colour photography, radio astronomy, lensless imaging and microscopy. Designs match end-to-end state-of-the-art approaches with less memory and compute, without task-specific decoder design.

Sep 1

Monday
  1. What exactly does word2vec learn?

    Berkeley AI researchers proposed the first theory quantitatively predicting word2vec learning. Under realistic training conditions, it reduces to unweighted least-squares matrix factorisation with a closed-form gradient-flow solution, and the resulting representation is equivalent to PCA of target matrix M*. From small initialisation, word2vec learns orthogonal linear subspaces in discrete steps, increasing embedding rank and reducing loss. These features are M*'s leading eigenvectors and can be calculated in advance from corpus statistics and hyperparameters.

Aug 1

Friday

Jul 23

Wednesday