Jan 17, 2024 · To address this issue, we propose a simple yet effective approach called Reinforced Fine-Tuning (ReFT) to enhance the generalizability of learning LLMs for reasoning, with math problem-solving as an example.

ReFT is different: (1) ReFT selects timesteps to intervene on; and (2) ReFT targets representations instead of weights. To help you understand these differences, let's consider these cases:

Linear subspace is powerful, conditioned on model's upstream computations. LoReFT shows linear subspaces contain rich semantics that you can manipulate to steer model behaviors.

Dec 20, 2024 · LoReFT, is a technique that adjusts the hidden representations within a linear subspace formed by a low-rank projection matrix. It builds upon the distributed alignment search (DAS) method introduced by Geiger et al. and Wu et al.

Training an 😀 Emoji-Chatbot (live demo) with ReFT in under 10 seconds! # need to be done. Step 1: loading the raw LM you want to train with ReFT. We first load in any model we want to gain...

May 9, 2024 · Enter ReFT (Representation Fine-Tuning) methods, which operate on a frozen base model and learn task-specific interventions on the hidden representations. Among the ReFT family, a standout instance...

Apr 4, 2024 · ReFT methods operate on a frozen base model and learn task-specific interventions on hidden representations. We define a strong instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT), and we identify an ablation of this method that trades some performance for increased efficiency.

Apr 10, 2025 · To address this issue, we propose a simple yet effective approach called Reinforced Fine-Tuning (ReFT) to enhance the generalizability of learning LLMs for reasoning, with math problem-solving as an example.

Apr 5, 2024 · Introducing Representation Finetuning (ReFT), a family of intervention-based representation finetuning methods. Typically, an intervention I is a tuple Φ, P, L that encapsulates a single inference-time modification of the representations computed by a Transformer-based LM.

Apr 6, 2024 · In the paper [3], researchers propose Representation Finetuning (ReFT) approach, which operates on a frozen base model and learn task-specific interventions on hidden representations. This...