--- title: "Automating AI Safety: What we can do today" description: "A concrete list of things to build now so a coding agent can actually run an interpretability experiment, and why the METR slowdown result does not say what people think." published: 2025-08-08 tags: ["Automating alignment research"] importance: 8 confidence: "likely" docStatus: "finished" audio: "https://pub-4ee2f71bc29541a7a6e8d9694f0a1b21.r2.dev/68956e635c58d327551e4960/audio.mp3" crosspost: lesswrong: "https://www.lesswrong.com/posts/FqpAPC48CzAtvfx5C/automating-ai-safety-what-we-can-do-today" figures: - src: "https://jacquesthibodeau.com/content/images/external/6f2ba4cc-cdixitggwryhhbh8hl4y.png" alt: "A two-column table from the METR report headed “We do not provide evidence that:”, pairing four claims the study does not make with clarifications, including that it only studied software development." width: 951 height: 689 - src: "https://jacquesthibodeau.com/content/images/external/f78b50f1-da27b00327590841afbea8aeb1076cfeb610a2a334f5ebaa.png" alt: "A table of the five factors the METR authors think harmed AI usefulness — over-optimism about AI, high developer familiarity with the repositories, large and complex repositories, low AI reliability, and implicit repository context — each with the observations behind it." width: 1680 height: 575 author: "Jacques Thibodeau" canonical: "https://jacquesthibodeau.com/automating-ai-safety-what-we-can-do-today/" --- There have been [multiple](https://www.lesswrong.com/posts/WJ7y8S9WdKRvrzJmR/building-ai-research-fleets) [recent](https://www.lesswrong.com/posts/nJcuj4rtuefeTRFHp/can-we-safely-automate-alignment-research) [calls](https://www.lesswrong.com/posts/W3KfxjbqBAnifBQoi/we-should-try-to-automate-ai-safety-work-asap) for the automation of AI safety and alignment research. There are likely many people who would like to contribute to this space, but would benefit from clear directions for how to do so. Stemming from a recent [SPAR](https://sparai.org/) [project](https://github.com/MShinkle/automation_projects_pilots/blob/main/SPAR_project/final_project_report.pdf) and in light of [limitations of current systems](https://www.lesswrong.com/posts/9eizzh3gtcRvWipq8/measuring-the-impact-of-early-2025-ai-on-experienced-open), we provide a brief list of concrete projects for improving the ability of current and near-future agentic coding LLMs to execute technical AI safety experiments. We expect each of these could be meaningfully developed as short-term (1 week to 3 months) projects. This is in no way intended to be a comprehensive list, and we strongly welcome additional project ideas in the comments.
Scope

Due to our background and current research areas, the examples in this post focus on mechanistic interpretability research. However, the general techniques here should be applicable to other sub-areas of technical alignment and safety research.

## Concrete Projects These are largely focused on improving LLM usage of current software packages. Projects are roughly in order of increasing scope. We include initial pilot versions of some of these ideas. ### **Improving LLM Usage of Relevant Software Packages** ***Compiled Monofiles*** As noted in a [recent paper by METR](https://www.lesswrong.com/posts/9eizzh3gtcRvWipq8/measuring-the-impact-of-early-2025-ai-on-experienced-open), current AI systems often can struggle due to lack of sufficient context, particularly for large and/or complex codebases. One way to provide extensive contextual information about a package to a coding agent is by converting it into a single large file. As suggested in [Building AI Research Fleets](https://www.alignmentforum.org/posts/WJ7y8S9WdKRvrzJmR/building-ai-research-fleets), “More generally, consider migrating to monorepos and single sprawling Google Docs to make it easier for your AI systems to load in the necessary context.” However, while actually migrating research code to monorepos may improve LLM comprehension to a degree, it also destroys organization that is useful (both to human coders and AIs). Alternatively, existing repositories can be converted to single large files, e.g. [llms.txt](https://llmstxt.org/), which can then be fed to the agent. There are existing tools, such as [RepoMix,](https://github.com/yamadashy/repomix) which aim to achieve this. We provide an [example RepoMix configuration file](https://github.com/MShinkle/automation_projects_pilots/blob/main/repomix/README.md) and [examples of generated results](https://github.com/MShinkle/automation_projects_pilots/tree/main/repomix/example_outputs). These files can be generated locally as needed or shared publicly, or even bundled with the packages themselves.[\[1\]](#fnpurce539njf) Note that, with no compression at all, this can result in very large files for large packages. Careful tuning of what to include or filter (e.g. large changelogs, empty spaces) can have a big effect on file size, but excessive compression may destroy useful information. One possible project direction would be varying different compression/filtering parameters and then benchmarking model performance on relevant tasks–this could reveal what information is most relevant for helping models automate research. ***Indexable API documentation*** As an alternative/complement to indexable documentation, some AI coding systems (e.g. Cursor) provide the ability to ‘[index’](https://docs.cursor.com/context/codebase-indexing) the documentation of packages. This gives coding agents access to things like package structure, docstrings, and usage demos without requiring the package files to be locally available. However, we found that safety-related packages we tried to index either did not have API documentation at all, or this documentation was not in a format that Cursor can effectively index. To address this, we suggest setting up automated systems for generating indexable docs for packages used in technical research, enabling AI coding agents to more effectively use these packages. Tools such as [Sphinx](https://www.sphinx-doc.org/en/master/) and [MkDocs](https://www.mkdocs.org/) enable automated generation of API documentation. This could either be integrated into the package repositories themselves or generated separately and stored in a public location. Note that there are also MCP servers like [Context7](https://context7.com/) which give AI agents access to documentation, provided it has been indexed on their website. This may serve as an alternative route for providing agents with package context, and could be evaluated against Cursor-style integrated indexing. ***Iteratively refined package guides*** One limitation of the above approaches is that they are likely to provide models with large amounts of irrelevant information. This means that models are still likely to make mistakes despite the presence of relevant documentation or examples. Furthermore, without some form of long-term memory across sessions, coding agents often repeat the same mistakes over and over. (For example, we find that coding agents consistently struggle to figure out how to use [NNsight](https://nnsight.net/) context properly, even with access to working examples.) To address this, we propose creating ‘package guides’ based on mistakes that LLMs *actually make during real tasks.* These can be thought of as capturing “learned wisdom” from experience, similar to a human coder learning from experience.[\[2\]](#fnqifgujraf8i) These could be constructed in a variety of ways, but we suggest generating them similarly to how human coders often learn: starting with available examples, attempting actual use cases, and iteratively debugging and refining implementations of tasks. This can all be done by the coding agent itself, requiring minimal human input. One possible general iterative loop for generating such a guide looks like: 1. **Planning** - Explore available documentation, code, demos, etc. - Create a list of intended use cases, break them into tasks and subtasks, and identify relevant examples. 2. **Initial Implementation** - Implement and carefully test tasks and subtasks. - Update guide to note any initial misunderstandings or critical insights in the guide, add example code snippets 3. **Refinement** - Re-examine available documentation and examples, exploring alternative ways to complete tasks. - Implement alternative methods and test similarly to before. - *\[This step can be repeated multiple times.\]* 4. **Testing with Fresh Context** - Initialize a new coding agent, providing only a task and the package guide, and repeat the above steps. The result is a structured guide built from actual model experience with a package. Optional additional steps could include increasingly large-scale tasks—implementing experiments end-to-end, integrating with other packages, or replicating full papers. We generate a few examples of guides like this using a simplified version of the steps described above using Claude 3.7 and Cursor—see the example guides [here](https://github.com/MShinkle/automation_projects_pilots/tree/main/package_guides/generated_guides), and example LLM prompts to generate such guides [here](https://github.com/MShinkle/automation_projects_pilots/tree/main/package_guides/instructions). ***Structured sandbox environments*** Ideally, a researcher or research agent would be able to quickly implement a research idea without needing to design a full (often complex) implementation and testing setup. For example, a researcher from another field may want to explore whether insights from their field can be applied to a problem in AI safety. These ideas could be tested more quickly and with a lower barrier of entry if doing so did not require implementing full experimental setups themselves, using packages and datasets with which they are likely unfamiliar. A solution to this is pre-designed **sandbox setups**—programming environments where the core structure is already set up and tests are in place, such that variants of an idea can be quickly and easily iterated over. For example, a sandbox setup for training and evaluating sparse autoencoders (SAEs) could consist of a set of scripts containing flexible model classes, trainers, and evaluation scripts. These could be carefully designed such that the implementation of new SAE variants (e.g. a change in loss calculation or architecture) can be implemented via simple, targeted changes, without full understanding of the rest of the setup. We provide an example of such a sandbox for designing SAEs [here](https://github.com/MShinkle/automation_projects_pilots/tree/main/experiment_sandboxes). This ‘sandboxing’ approach carries two primary benefits: - *Simplicity/Efficiency:* sandboxing outsources the process of setting up an experimental environment, allowing an AI agent to efficiently implement and test many different variants. - *Guardrails:* sandboxes can provide clear guardrails on what the agent can modify. We find that, in practice, agents sometimes modify evaluation code without being instructed to do so. This can be done without explicitly attempting to reward tamper. Attempts to fix errors arising during evaluation (e.g. a shape mismatch or invalid input type) may inadvertently change the evaluation in other ways, leading to spuriously high or low scores. By whitelisting which files an agent may change, you prevent accidental (or adversarial) edits to evaluations. Currently, it is likely most effective to have these environments created by human researchers who are somewhat knowledgeable about the packages involved. Spending a day to build a robust setup could easily pay off if it enables coding agents to effectively iterate over an arbitrarily high number of variants. Alternatively, current or near-future AI agents may be able to create these environments with careful testing and feedback. ### **Focused Benchmarks of Safety Research Automation** *Note: Evaluating the true ‘success’ (i.e. usefulness) of the previously described tools can be difficult without quantitative evaluations. Consequently, we think that evaluations in this or a similar form should probably be especially prioritized.* Compared to massive volumes of code for training models present within the training data of modern LLM agents, exposure to safety-focused codebases during model training is much smaller. This disparity is also present in evaluation; there are multiple benchmarks for autonomous programming and research using packages like PyTorch and Transformers, but none (to our knowledge) focusing on automated use of tools like TransformerLens or Inspect. Evaluations that specifically test coding agents on their ability to use existing technical safety research tools would fill this gap (think [PaperBench](https://cdn.openai.com/papers/22265bac-3191-44e5-b057-7aaacd8e90cd/paperbench.pdf)/[MLE-Bench](https://arxiv.org/abs/2410.07095) for safety research). This could contain multiple tiers of tasks, ranging from small-scale tests (“Write a function that uses transformerlens to print the shape of the first-layer MLP outputs for this model.”), to medium-scale objectives (“Train a skip transcoder using Sparsify.”), to larger tasks, such as full experiments or paper replications. Evaluation metrics could include binary ‘succeeded’/’failed’, as well as more continuous metrics like time to complete, number of iterations before success, number of tool calls, rating for each sub-task node in a more complex task, et cetera. You can think of a project as a set of task nodes and you are more likely to lead to automation if you increase the reliability of doing that task successfully (and efficiently) in the future. We think that benchmarks of this sort are critical for evaluating the usefulness of the other tools we have described (monofiles, indexable docs, package guides).
Side Notes — broader context and importance (recommended reading) ### Implications of the recent METR paper A [recent paper by METR](https://www.lesswrong.com/posts/9eizzh3gtcRvWipq8/measuring-the-impact-of-early-2025-ai-on-experienced-open) found negative results regarding the usefulness of AI-based coding assistance for expert developers. One *wrong* takeaway from this is that current AI coding systems are net harmful for coding *across the board*. The authors have gone to considerable effort to clarify what exactly their results do *not* say, both on X and in the report itself: ![A two-column table from the METR report headed “We do not provide evidence that:”, pairing four claims the study does not make with clarifications, including that it only studied software development.](https://jacquesthibodeau.com/content/images/external/6f2ba4cc-cdixitggwryhhbh8hl4y.png) To summarize, negative results in a case with highly experienced human coders—using codebases of which the experts already have extensive knowledge—do *not* imply that the same coding systems could not be highly useful in other contexts. It’s also worth noting that even in the experiment from the paper, results were not negative for all cases. For example, see [this thread](https://x.com/QuentinAnthon15/status/1943948791775998069) from one of the participants in the original study who achieved a 38% [*speedup*](https://x.com/QuentinAnthon15/status/1944050600477700581) from using AIs. In the report, the authors also note five likely factors harming AI usefulness in their experiment. ![A table of the five factors the METR authors think harmed AI usefulness — over-optimism about AI, high developer familiarity with the repositories, large and complex repositories, low AI reliability, and implicit repository context — each with the observations behind it.](https://jacquesthibodeau.com/content/images/external/f78b50f1-da27b00327590841afbea8aeb1076cfeb610a2a334f5ebaa.png) Of these factors, two relate directly to the aims of the projects we’ve described: AI struggling in complex code environments (C.1.3) and failing to utilize important tacit knowledge or context (C.1.5). The fact that these correspond closely with the explicit motivations of our some of our projects (compiled monorepos, indexable docs, package guides) suggest that these projects are worth pursuing. ### How large is the risk that these will inadvertently accelerate other, harmful research directions? As mentioned earlier, existing AI coding evaluations primarily test models on popular libraries and common software development tasks. This puts codebases with small user bases at a systematic disadvantage for LLM use. Indeed, in our project, we found that whereas LLM coding agents still made errors when using packages like PyTorch, these errors generally indicated small-scale misunderstandings, such as the absence of a function argument in some PyTorch versions. In contrast, when attempting to use more niche, interpretability-focused packages, they consistently made much larger mistakes that seem to reflect broad misunderstandings of the high-level functionality and structure of these packages. The concrete directions we described above are intended to reduce this systematic disadvantage. Consequently, we expect these techniques would yield a much smaller benefit in research areas like AI capabilities than in AI safety.[\[3\]](#fnuu5v70ix5m) Given these factors, we consider the risk that these techniques will translate to areas like general capabilities research to be low. ### Should we just wait for research systems/models to get better? People are already using AI for automating safety research. Even without full automation, AI can still substantially accelerate the rate of some research areas. Furthermore, research automation is unlikely to be a phase change; techniques that improve safety research automation today are still useful, and may *still* be useful when we have more capable systems. Moreover, once end-to-end automation is possible, it will still take time to integrate those capabilities into real projects, so we should be building the necessary infrastructure and experience now. As Ryan Greenblatt has said, “Further, it seems likely we’ll run into integration delays and difficulties speeding up security and safety work in particular\[…\]. Quite optimistically, we might have a year with 3× AIs and a year with 10× AIs, and we might lose half the benefit due to integration delays, safety taxes, and difficulties accelerating safety work. This would yield 6 additional effective years\[…\].” Building automated AI safety R&D ecosystems early ensures we're ready when more capable systems arrive. ### Research automation timelines should inform research plans It’s worth reflecting on scheduling AI safety research based on when we expect sub-areas of safety research will be automatable. For example, it may be worth [putting off R&D-heavy projects](https://www.lesswrong.com/posts/fsLpvRiLt76pcCcPD/you-should-delay-engineering-heavy-research-in-light-of-r) until we can get AI agents to automate our detailed plans for such projects. If you predict that it will take you 6 months to 1 year to do an R&D-heavy project, you might get more research mileage by writing a project proposal for this project and then focusing on other directions that are tractable now. Oftentimes, it’s probably better to complete 10 small projects in 6 months and then one big project in an additional 2 months, rather than completing one big project in 7 months. This isn’t to say that R&D-heavy projects are not worth pursuing—big projects that are harder to automate may still be worth prioritizing if you expect them to substantially advance downstream projects (such as [ControlArena from UK AISI](https://github.com/UKGovernmentBEIS/control-arena)). But research automation will rapidly transform what is ‘low-hanging fruit’. Research directions that are currently impossible due to the time or necessary R&D required may quickly go from intractable to feasible to trivial. Carefully adapting your code, your workflow, and your research plans for research automation is something you can—and likely should—do now. *Thanks to* [*PIBBSS*](https://pibbss.ai/fellowship/) *for support during the writing of this post.*
1. [**^**](#fnrefpurce539njf)If package authors do not want these integrated into their packages, it may be useful to have a dedicated platform or repository for generation and storage of these, as well as the following two projects (indexable docs and package guides). These could be automatically generated/updated once per package version and then stored in a shared repository. This way, researchers could just download the latest versions when needed rather than having to generate them individually. This could also enable improvements to these methods to spread more efficiently. 2. [**^**](#fnrefqifgujraf8i)[It has been suggested](https://www.dwarkesh.com/p/timelines-june-2025) that a lack of continual learning may prevent AIs from reaching some levels of capability. What we describe here can be thought of as a form of continual learning–AIs can keep track of issues they encounter and how to resolve them, and then draw from this information in similar circumstances in the future. Training these insights into model weights may not be necessary if they can be accessed in the model’s context. 3. [**^**](#fnrefuu5v70ix5m)AGI labs may also be more [bottlenecked on compute](https://x.com/JacquesThibs/status/1946685555229786159) than AI safety research. ## Sources Every external link in this piece that has a captured card, with what that page said when it was captured. The quoted lines below are not the author of this piece writing: they are the linked page describing itself, recorded by `bun run link-cards` on the date given, and kept so that a reader still has them if the original moves or goes away. - **Building AI Research Fleets** — Ben Goldhaber, lesswrong.com, 2025-01-12 Captured 2026-08-28. > From AI scientist to AI research fleet Research automation is here (1, 2, 3). We saw it coming and planned ahead, which puts us ahead of most (4, 5, 6). But that foresight also comes with a set of outdated expectations that are holding us back. In particular, research automation is not just about “aligning the first… - **Can we safely automate alignment research?** — Joe Carlsmith, lesswrong.com, 2025-04-30 Captured 2026-08-28. > (This is the fifth essay in a series that I’m calling “How do we solve the alignment problem?”. I’m hoping that the individual essays can be read fairly well on their own, but see this introduction for a summary of the essays that have been released thus far, and for a bit more about the series as a whole. Podcast… - **We should try to automate AI safety work asap** — Marius Hobbhahn, lesswrong.com, 2025-04-26 Captured 2026-08-28. > This is a personal post and does not necessarily reflect the opinion of other members of Apollo Research. I think I could have written a better version of this post with more time. However, my main hope for this post is that people with more expertise use this post as a prompt to write better, more narrow versions for… - **SPAR - Research Program for AI Risks** — SPAR - Research Program for AI Risks, sparai.org Captured 2026-08-28. > SPAR connects rising talent with experts in AI safety and policy through structured mentorship and impactful research projects. Apply to work on research addressing risks from advanced AI. - **automation_projects_pilots/SPAR_project/final_project_report.pdf at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity** — habryka, lesswrong.com, 2025-07-11 Captured 2026-08-28. > METR released a new paper with very interesting results on developer productivity effects from AI. I have copied the blogpost accompanying that paper here in full. ---------------------------------------- We conduct a randomized controlled trial (RCT) to understand how early-2025 AI tools affect the productivity of… - **Building AI Research Fleets** — Ben Goldhaber, alignmentforum.org, 2025-01-12 Captured 2026-08-28. > From AI scientist to AI research fleet Research automation is here (1, 2, 3). We saw it coming and planned ahead, which puts us ahead of most (4, 5, 6). But that foresight also comes with a set of outdated expectations that are holding us back. In particular, research automation is not just about “aligning the first… - **The /llms.txt file, v2 – llms-txt** — Jeremy Howard, llmstxt.org Captured 2026-08-28. > A proposal to standardise on using an /llms.txt file to provide information to help agents use a website. - **yamadashy/repomix** — yamadashy, github.com Captured 2026-08-28. > 📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more. - **automation_projects_pilots/repomix/README.md at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **automation_projects_pilots/repomix/example_outputs at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **Cursor Docs — Agent, Rules, MCP, Skills & CLI** — docs.cursor.com Captured 2026-08-28. > Official Cursor documentation. Covers Agent mode, Rules, Skills, MCP servers, CLI, models, and Teams & Enterprise setup. Start building in minutes. - **Sphinx — Sphinx documentation** — sphinx-doc.org Captured 2026-08-28. - **Context7 - Up-to-date documentation for LLMs and AI code editors** — Context7, context7.com Captured 2026-08-28. > Pull up-to-date, version-specific documentation and code examples for any library directly into Cursor, Claude Code, Codex, Devin Desktop, Antigravity and other AI coding tools. - **nnsight** — nnsight.net Captured 2026-08-28. > Documentation for the nnsight Python library - **automation_projects_pilots/package_guides/generated_guides at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **automation_projects_pilots/package_guides/instructions at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **automation_projects_pilots/experiment_sandboxes at main · MShinkle/automation_projects_pilots** — github.com Captured 2026-08-28. > Contribute to MShinkle/automation_projects_pilots development by creating an account on GitHub. - **PaperBench: Evaluating AI’s Ability to Replicate AI Research** — Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Chan Jun Shern, Leon Maksin, Rachel Dias, Evan Mays, Benjamin…, cdn.openai.com Captured 2026-08-28. - **MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering** — Jun Shern Chan and 11 others, arxiv.org, 2024-10-09 Captured 2026-08-28. > We introduce MLE-bench, a benchmark for measuring how well AI agents perform at machine learning engineering. To this end, we curate 75 ML engineering-related competitions from Kaggle, creating a diverse set of challenging tasks that test real-world ML engineering skills such as training models, preparing datasets,… - **Quentin Anthony** — @QuentinAnthon15, x.com, 2025-07-12 Captured 2026-08-28. > I was one of the 16 devs in this study. I wanted to speak on my opinions about the causes and mitigation strategies for dev slowdown. I'll say as a "why listen to you?" hook that I experienced a -38% AI-speedup on my assigned issues. I think transparency helps the community. - **Quentin Anthony** — @QuentinAnthon15, x.com, 2025-07-12 Captured 2026-08-28. > @simonbtownsend No, I was sped up 38%. I used a negative number to match their chart. - **You should delay engineering-heavy research in light of R&D automation** — Daniel Paleka, lesswrong.com, 2025-01-07 Captured 2026-08-28. > tl;dr: LLMs rapidly improving at software engineering and math means lots of projects are better off as Google Docs until your AI agent intern can implement them. Implementation keeps getting cheaper Writing research code has gotten a lot faster over the past few years. Since 2021 and OpenAI Codex, new models and… - **UKGovernmentBEIS/control-arena** — UKGovernmentBEIS, github.com Captured 2026-08-28. > ControlArena is a collection of settings, model organisms and protocols - for running control experiments. - **PIBBSS Fellowship – Principles of Intelligence** — pibbss.ai, 2025-10-09 Captured 2026-08-28. - **Why I don’t think AGI is right around the corner** — Dwarkesh Patel, dwarkesh.com Captured 2026-08-28. > Continual learning is a huge bottleneck - **Jacques** — @JacquesThibs, x.com, 2025-07-19 Captured 2026-08-28. > @peterbarnett_ @MechanizeWork From OpenAI researcher who worked on latest math rl breakthrough ## Terms used The author's own definitions for the glossary terms this piece uses. These are his words, not a standard reference. - **AGI** — Artificial general intelligence: a system with human level cognitive ability across domains rather than in one narrow task. - **AISI** — AI Safety Institute: a government body set up to evaluate frontier models before and after release. See also: - **API** — Application programming interface: the contract one program uses to call another. - **LLM** — Large language model: a neural network trained on very large amounts of text to predict what comes next. - **MCP** — Model Context Protocol: an open standard for connecting an assistant to external tools and data sources. See also: - **METR** — Model Evaluation and Threat Research: an organisation that measures what frontier models can actually do on long autonomous tasks. See also: - **MLP** — Multi layer perceptron: the fully connected feedforward block that sits between attention layers in a transformer. See also: - **SAE** — Sparse autoencoder: an interpretability tool that decomposes a model's activations into a larger set of mostly inactive features. See also: - **SPAR** — Supervised Program for Alignment Research: a part time programme pairing junior researchers with alignment mentors. See also: