Foundation models have achieved remarkable performance across a wide range of tasks, yet their internal representations remain opaque, entangled, and difficult to control, limiting safety, alignment, and adaptability. In this talk, I present a unified framework for controlling multimodal foundation models based on representing activations as sparse linear combinations of concept vectors from a curated dictionary. First, I introduce Parsimonious Concept Engineering (PaCE), which constructs concept dictionaries in LLMs and enables targeted intervention via suppression of undesirable activation components, mitigating unaligned responses without degrading core capabilities. Next, I present Concept Lancet, which extends this framework to diffusion models, enabling fine-grained editing of images through decomposition and recombination of visual concepts. Finally, I introduce Dictionary-Aligned Concept Control (DACO), which extends this approach to multimodal settings by learning shared concept dictionaries across modalities, enabling scalable, granular control of model behavior for improved safety and alignment. Together, these approaches establish a paradigm for controlling foundation models through concept-based sparse representations, enabling interpretable intervention and reliable alignment across domains.

Seminar Recording