Source: https://arxiv.org/abs/2101.00190 Introduction Natural language generation (NLG) models like the GPT series have enabled remarkable progress on conditional text generation tasks such as summarization and table-to-text.  However, fine-tuning these large pretrained models on downstream NLG tasks requires updating all model parameters, which is computationally expensive. Fortunately, researchers have identified lightweight fine-tuning methods that reduce the number of parameters that must be updated when fine-tuning a model for a specific task. One such method is prefix tuning, proposed by Li and Liang (2021). Prefix tuning keeps the parameters of the pretrained model fixed, and only trains a small continuous “prefix” that

Source: Multimodal Few-Shot Learning with Frozen Language Models Introduction Recent advances in natural language processing have led to large transformer-based language models that exhibit impressive few-shot learning abilities. Specifically, when trained at sufficient scale, these models can learn new language tasks after being prompted with just a few examples, without any further gradient updates.  However, despite these impressive capabilities, large language models are limited to only understanding text. These models cannot process inputs in other modalities like vision, which prevents us from directly communicating visual information, tasks, or concepts to the models. We cannot simply show the model an image

Source: DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales Introduction ChatGPT-like models showcase the power of large language models for conversational AI. However, training such powerful models requires massive computational resources, making them inaccessible to many researchers and developers. Microsoft’s newly open-sourced DeepSpeed-Chat aims to change that by providing an end-to-end framework for training ChatGPT-style models efficiently at any scale. DeepSpeed-Chat allows training models with hundreds of billions of parameters in record time using only commodity GPUs. This is made possible by DeepSpeed-Chat’s optimized DeepSpeed-RLHF training system. It combines innovations in memory optimization, parallelism, and other

Source: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Introduction In-context learning has emerged as one of the most remarkable capabilities of large language models like GPT-3 and GPT-4. With just a few demonstration examples, these models can rapidly adapt to new tasks and make accurate predictions without any parameter updates. But how does this impressive on-the-fly learning actually work behind the scenes? In a fascinating new paper from Microsoft Research and Peking University, researchers provide new theoretical insights that help unravel the optimization processes underlying in-context learning in Transformer models. By drawing parallels to gradient descent and analyzing the mechanics

When building machine learning systems, it’s common to take a model pre-trained on a large dataset and fine-tune it on a smaller target dataset. This allows the model to adapt its learned features to the new data.  However, naively fine-tuning all the model’s parameters can cause overfitting, since the target data is limited. In a new paper, researchers from Stanford explore an intriguing technique they call “surgical fine-tuning” to address this challenge. The key insight is that fine-tuning just a small, contiguous subset of a model’s layers is often sufficient for adapting to a new dataset.  In fact, they show across 7