Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.
1.7k
Stars
89
Forks
8
Open issues
14
Contributors
AI Analysis
Club-3090 is a specialized toolkit for running modern large language models locally on consumer-grade NVIDIA RTX 3090/4090/5090 GPUs, providing production-ready configurations across multiple inference engines (vLLM, llama.cpp, ik_llama). It's purpose-built for homelab operators, local development backends, and hobbyists with high-end consumer GPUs who want to self-host models like Qwen and Gemma; it is not a general-purpose ML framework and does not serve users without compatible hardware or...
Inferred from signals mentioned in the README (tests, CI, type safety) — not a review of the actual code.
AI's overall editorial judgment — not an average of the bars above, can weigh other factors too.
Docker-native LLM serving recipes optimized for consumer RTX 3090/4090/5090 GPUs
club-3090 provides production-ready Docker Compose configurations and deployment scripts for running modern LLMs (Qwen, Gemma) on single-to-dual RTX 3090s and compatible GPUs. Built for homelab operators, indie developers, and on-premises deployments who want reproducible, low-latency inference without cloud costs. Targets users with specific hardware constraints and CUDA expertise.
Launched April 2026 by noonghunna; appears to address a gap between generic LLM frameworks (vLLM, llama.cpp) and consumer-GPU-specific deployment. Gained ~1,400 stars in ~2 months, suggesting resonance with home-lab/indie AI segments.
Initial adoption curve is steep (82 stars in 7 days as of June 2026). Growth likely driven by: (1) specific hardware targeting (3090 was still common for indie/home use in 2025–2026), (2) multi-engine coverage (vLLM, llama.cpp, ik_llama), (3) practical documentation (benchmarks, cliff analysis, cross-GPU FAQs), (4) timing coincidence with Qwen 3.6 and Gemma 4 releases. No evidence of slowdown yet.
adoption not verified. No case studies, deployment counts, or organizational users documented in README. Growth trajectory and star ratio (1,464 stars vs 78 forks) suggests hobbyist interest rather than production fleet adoption. Mentions 'shipped' configs for specific models but does not cite real-world operators or scale.
Appears to be a bash-driven orchestration layer wrapping Docker Compose, vLLM, llama.cpp, and ik_llama engines. README shows hardware-aware picker scripts (setup.sh, launch.sh, switch.sh) that mediate config selection. Includes a TUI cockpit (serve-cockpit, installed via `uv pip`). Likely uses YAML for compose templates and Python for compatibility profiling. No direct code inspection possible; architecture inferred from script names and README.
README documents a `verify-full.sh` sanity validation and `bench.sh` canonical benchmark. Not documented as automated CI/CD or formal test suite; appears to rely on user-validated Docker Compose verification and manual benchmarking.
Last push 2026-06-24 (today, relative to current date 2026-06-25) indicates active maintenance. README references specific Genesis versions (v7.72.2 PN59), known cliffs (Cliff 2 on vLLM single-card), and recent workaround discovery. Appears to track upstream engine breaking changes closely. 78 forks and 1,464 stars suggest community engagement. Explicit update script (scripts/update.sh) hints at expectation of regular maintenance.
ADOPT IF: you own a 3090/4090/5090, want reproducible local inference without cloud costs, have Docker + Linux/WSL2 experience, and value multi-engine flexibility (vLLM for throughput, llama.cpp for robustness). Recipes + benchmarks reduce trial-and-error. AVOID IF: you need production HA/scaling (no cluster orchestration shown), prefer cloud-native tooling, lack CUDA/Docker expertise, or require non-NVIDIA GPU support. MONITOR IF: you are evaluating consumer-GPU inference stacks; project is young (2 months) but actively maintained and gaining adoption velocity; direction and stability after Q3 2026 unclear.
Independent dimensions
Mainstream potential
3/10
Technical importance
6/10
Adoption evidence
2/10
- Very new (April 2026): limited production runway; no long-term stability data yet.
- Hardware specificity: configs may break on NVIDIA driver or engine version updates (vLLM/llama.cpp churn), requiring active maintenance. README acknowledges Genesis patch tracking already.
- Known technical debt: single-card vLLM prefill cliff (Cliff 2) not fully resolved as of June 2026; users must select workaround (dual-card or llama.cpp).
- Adoption not verified: no public user testimonials, deployment counts, or organizational references; growth signal inferred from stars, not validated usage.
- Documentation coverage: README is thorough for deployment but sparse on architecture rationale, troubleshooting depth, and fallback patterns beyond the documented cliffs.
Likely to remain a specialized, actively-maintained niche resource for home-lab and indie-developer LLM deployment through 2026–2027. May gradually accumulate production users if stability improves and upstream (vLLM, llama.cpp) engine integration simplifies. Unlikely to scale to enterprise clusters or cloud-adjacent deployments without significant architectural pivot.
Newsletter
Get analyses like this every Monday
Free weekly digest of the most interesting open-source discoveries.
Languages
Information
- Language
- Python
- License
- Apache-2.0
- Last updated
- 14h ago
- Created
- 2mo ago
- Analyzed with
- anthropic/claude-haiku-4-5
Stars over time
Contributors over time
Top 100 contributors only — repos with more will plateau at 100.
Open issues
[bench] Tess-4-27B on single RTX 4090 (llama.cpp DFlash n=4, 96K) — 117/150 off · 125/150 on, beats dual-3090 ref
[bench] (2× RTX 3090 PCIe Gen4 x8/x8, no NVLink, Ryzen 7 5800X, 64GB DDR4, KVM/Proxmox passthrough, 290W cap)
[bench] you asked about 4x 3090 my-rig.md - here is mine on --variant vllm/qwen-27b-multi-max (at least I guess it uses the current model)
[bug] running #608 qwen-27b nvfp4 on single 5090 -
[bench] vllm/qwen-27b-single-nvfp4 OOM 5090
Top contributors
Similar repos
| Repository | Stars | Week Δ | Language | Score | Updated |
|---|---|---|---|---|---|
|
noonghunna/club-3090
this
|
1.7k | +77 | Python | 7/10 | 14h ago |
 turboderp-org/exllamav3
|
1k | — | Python | 8/10 | 2d ago |
 lyogavin/airllm
|
22.4k | — | Jupyter Notebook | 8/10 | 4h ago |
 vllm-project/vllm
|
85.9k | — | Python | 10/10 | 1h ago |
 ggml-org/llama.cpp
|
119.9k | — | C++ | 9/10 | 3h ago |
 Luce-Org/lucebox-hub
|
2.6k | — | C++ | 7/10 | 16h ago |
Upstream inference engine. club-3090 is a *consumer-GPU-specific wrapper* around vLLM, not a replacement. Offers recipes + tuning for 3090-class GPUs where bare vLLM requires more experimentation.
Alternative inference engine (CPU-friendly, GGUF-based). club-3090 bundles llama.cpp as one engine option for users prioritizing robustness and context window over throughput. Not competitive; complementary choice.
Fine-tuning and deployment orchestrator. Broader scope (training, LoRA, serving). club-3090 narrower: inference-only, hardware-specific. Different tier.
Memory-efficient inference on consumer GPUs. Similar audience (resource-constrained). club-3090 distinguishes via Docker reproducibility, multi-engine flexibility, and explicit 3090 calibration.
Frontend/UX layer for LLM APIs. club-3090 optionally bundles it (Image Studio); not a replacement, supplementary.
