Quantization

AI Engineering Academy is a structured learning platform offering curated educational content and hands-on projects for mastering applied AI concepts...

A curated collection of AI engineering interview questions and answers covering LLMs, RAG, agents, fine-tuning, and production AI systems. This...

AngelSlim is a specialized model compression toolkit from Tencent for large language models, vision-language models, and audio models, offering...

turbovec is a Rust-based vector search index with Python bindings, implementing Google's TurboQuant quantization algorithm to achieve 8× compression...

LLM Compressor is a Python library for applying quantization and pruning algorithms to large language models for optimized deployment with vLLM. It...

AutoRound is a specialized quantization toolkit for Large Language Models and Vision-Language Models, designed to achieve high accuracy at ultra-low...

faster-whisper is a high-performance reimplementation of OpenAI's Whisper speech-to-text model using CTranslate2, delivering up to 4× faster...

AIMET is a specialized toolkit for quantizing and compressing trained neural network models to enable efficient deployment on edge devices. It...

Geti is an end-to-end Vision AI application that accelerates computer vision model development from raw images to deployed inference, optimized for...

GPTQModel is a specialized LLM quantization and compression toolkit providing hardware-accelerated inference across NVIDIA, AMD, Intel, Huawei, and...

LLaMA Factory is a unified fine-tuning framework that supports 100+ large language models and vision-language models through a zero-code CLI and web...

Brevitas is a PyTorch library providing neural network quantization for both post-training quantization (PTQ) and quantization-aware training (QAT),...

Optimum is a hardware optimization toolkit that extends HuggingFace's Transformers, Diffusers, TIMM, and Sentence-Transformers libraries to...

Intel Neural Compressor is a production-grade Python library for model compression on PyTorch, TensorFlow, and JAX, specializing in quantization...

CTranslate2 is a specialized C++ and Python inference engine that accelerates Transformer model execution on CPUs and GPUs through advanced...

SageAttention is a specialized CUDA library implementing quantized attention mechanisms (8-bit, FP8, and FP4 variants) that accelerates inference for...

TorchAO is a PyTorch native library for quantization and sparsity optimizations spanning training and inference, with demonstrated speedups (1.5x...

FINN is a specialized dataflow compiler from AMD Research that generates efficient FPGA accelerators for quantized neural network inference. It...

Qbot is an AI-powered automated quantitative trading platform supporting supervised learning, market dynamics modeling, and reinforcement learning...

xTuring is a Python framework for fine-tuning, evaluating, and running open-source large language models (LLaMA, GPT-J, Qwen, Mistral, etc.) on...

NNCF is a specialized neural network compression framework for optimizing inference performance in OpenVINO and other backends (PyTorch, ONNX)...

bitsandbytes is a PyTorch library that enables memory-efficient inference and training of large language models through k-bit quantization techniques...

PicoLM is a C-based LLM inference engine designed to run 1-billion parameter models on extremely resource-constrained embedded devices (e.g., $10...

Chinese-LLaMA-Alpaca is a specialized project that adapts Meta's LLaMA models for Chinese language tasks by extending the vocabulary and performing...

Optimum Quanto is a PyTorch quantization backend for Hugging Face's Optimum framework, enabling efficient model quantization with support for...

pngquant is a PNG compressor that reduces file sizes by converting 24/32-bit images to efficient 8-bit indexed color with alpha channel, typically...

MindPipe is a unified model compression framework for large language models and vision-language models, supporting 11 quantization methods and 7...

TensorFlow Model Optimization Toolkit provides quantization and pruning techniques to compress ML models for efficient deployment, targeting both...

SpargeAttn is a training-free sparse attention mechanism that accelerates inference for language models, vision transformers, and video generation...