Fast, State of the Art Quantized Embedding Models

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for generall from gravatar.com generall Avatar for nirant_k from gravatar.com nirant_k

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Other/Proprietary License (Apache License)
  • Author: NirantK
  • Tags vector , embedding , neural , search , qdrant , sentence-transformers
  • Requires: Python >=3.8.0, <3.12
Classifiers
Report project as malware

Project description

⚡️ What is FastEmbed?

FastEmbed is an easy to use -- lightweight, fast, Python library built for retrieval augmented generation. The default embedding supports "query" and "passage" prefixes for the input text.

  1. Light

    • Quantized model weights
    • ONNX Runtime for inference
    • No hidden dependencies on PyTorch or TensorFlow via Huggingface Transformers

  2. Accuracy/Recall

    • Better than OpenAI Ada-002
    • Default is Flag Embedding, which is top of the MTEB leaderboard

  3. Fast

    • About 2x faster than Huggingface (PyTorch) transformers on single queries
    • Lot faster for batches!
    • ONNX Runtime allows you to use dedicated runtimes for even higher throughput and lower latency

🚀 Installation

To install the FastEmbed library, pip works:

pip install fastembed

📖 Usage

from fastembed.embedding import FlagEmbedding as Embedding

documents: List[str] = [
    "passage: Hello, World!",
    "query: Hello, World!", # these are two different embedding
    "passage: This is an example passage.",
    # You can leave out the prefix but it's recommended
    "fastembed is supported by and maintained by Qdrant." 
]
embedding_model = Embedding(model_name="BAAI/bge-base-en", max_length=512) 
embeddings: List[np.ndarray] = list(embedding_model.embed(documents))

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

  1. Quantized model weights
  2. ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

  1. No hidden dependencies on PyTorch or TensorFlow via Huggingface Transformers

Why accurate?

  1. Better than OpenAI Ada-002
  2. Top of the Embedding leaderboards e.g. MTEB

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page