Sep 11, 2023 · Medusa is a simple framework that democratizes the acceleration techniques for LLM generation with multiple decoding heads. Medusa-1 on Vicuna-7b. We aim to tackle the three pain points of popular acceleration techniques like speculative decoding: Requirement of a good draft model. System complexity.

Jan 19, 2024 · In this paper, we present Medusa, an efficient method that augments LLM inference by adding extra decoding heads to predict multiple subsequent tokens in parallel. Using a tree-based attention mechanism, Medusa constructs multiple candidate continuations and verifies them simultaneously in each decoding step.

Medusa AI is a set of AI modules and tools that allow you to build rich, reliable, and performant AI applications without reinventing core AI logic. Large language models (LLM) are a type of artificial intelligence model that are trained on huge corpora of natural language data.

The medusai project is an attempt to capture and reflect on the good intentions and promises of AI on one hand and the risks and dangers on the other. The eerie and uncanny notion of AI-driven robotic snakes following and threatening humans is balanced by the sculpture’s potential to inspire humans - musicians, dancers, museum visitors - to ...

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads - BrunoScaglione/Medusa-AI

Medusa is an AI technique designed to make Large Language Models (LLMs) work faster by predicting multiple words at once, rather than just one at a time. Most LLMs operate by embedding information (in a form known as a token) and then predicting the next token in …

Sep 11, 2023 · We tested Medusa with Vicuna models, which are specialized Llama models fine-tuned specifically for chat applications. These models vary in size, with parameter counts of 7B, 13B, and 33B. Our goal was to measure how Medusa could accelerate these models in a real-world chatbot environment.

Medusa aims to make artificial intelligence more accessible by empowering everyday people and the average business to create their own AI applications. Through easy-to-use tools and pre-built...

Medusa is a Python package which works with Python version 3.10 and on Linux and and Mac (x86_64 architectures only). Most of Medusa’s functionality will in fact also work on Windows and M1/M2 Macs (arm64 architecture), with the exception of rendering (as pytorch3d cannot be automatically installed on Windows and Mac M1/M2).

medusai is a multi-modal AI driven robotic sculpture that responds to and interacts with humans through sound, light, touch, and movement. Inspired by the Greek myth of Medusa, the sculpture features seven robotic arms in the form of "snake hair" installed on top of …