4 Related Work

Pretraining of neural language models stretches back to the work of Bengio et al. (2003), and language modeling itself stretches back to at least Shannon (1948). Subsequent work then built both masked language modeling (Collobert et al., 2011; Peters et al., 2018; Devlin et al., 2019) and next token prediction systems (Dai and Le, 2015; Raffel et al., 2020; Radford et al., 2018). The latter has now become the dominant paradigm due to the ability to extend to generating full sequences autoregressively. Despite rapid progress, particularly by scaling (Brown et al., 2020; Achiam et al., 2023), there remain unanswered questions in key areas of generalization, for example safety, factuality and reasoning.

Safety. Training on all available pretraining data will inevitably include unsafe human written data, from toxicity through to bias and harms. Simply filtering the pretraining data of unsafe content can impoverish the model, and will make it unable to handle unsafe inputs (Xu et al., 2020). As with other issues, one approach is to attempt to fix these problems in post-training (Dinan et al., 2019; Xu et al., 2021; Bai et al., 2022). However, due to poor generalization issues still remain typically when considering out-of-distribution inputs, as is shown by jailbreak attacks (Zou et al., 2023). It should also be noted that fine-grained control of safety is likely a better choice than simply removing capabilities (Yi et al., 2025). Korbak et al. (2023) is an early work incorporating safety into pretraining, which reported success with control tokens which incorporate human preferences. More recently, Min et al. (2023) also use a combination of rewriting and special tokens, and report encouraging results. Shilov et al. (2025) proposes a different approach, whereby they alter the training scheme altogether. They split the model’s weights into retain and forget subsets, and guide specific knowledge into the forget subset during training.

Factuality. A number of works have tried to address factuality at post-training time with various approaches. Tian et al. (2023); Lin et al. (2024); Zhang et al. (2024b) mostly focused on supervised fine-tuning (SFT) and