Skip to content
#

rtx-3090

Star

Here are 44 public repositories matching this topic...

Language: All
Filter by language
All 44 Python 27 PowerShell 3 HTML 2 Shell 2 Batchfile 1 Cuda 1 Markdown 1 Swift 1 TypeScript 1 Mojo 1
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

syv-ai / qwen38-27b-rtx3090

Star 1.1k

Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks

quantization kv-cache vllm local-llm llm-inference qwen speculative-decoding qwen3 rtx-3090
  • Updated Sep 4, 2026
  • Python

Sandermage / sndr_core_engine

Star 131

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.

memory cuda self-hosted quantization gemma rag awq llm-serving pgvector vllm local-llm llm-inference qwen speculative-decoding kv-cache-quantization tool-calling consumer-gpu rtx-4090 rtx-3090 spec-decode
  • Updated Sep 2, 2026
  • Python

thc1006 / qwen3.6-speculative-decoding-rtx3090

Star 35

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from the committed data by a checker that fails on drift, a coverage probe reports how many of them it actually covers, and ERRATA.md lists this study's own retracted claims.

benchmark reproducible-research cuda moe reproducibility mtp ampere mixture-of-experts negative-results inference-benchmark llama-cpp ggml local-llm llm-inference qwen gguf speculative-decoding qwen3 rtx-3090 dflash
  • Updated Sep 3, 2026
  • Python

DominikBucko / qwen38-flash-next-2x3090

Star 11

Run Qwen3.8 Flash Next on dual RTX 3090 (2x24GB) + 128GB RAM: 256K context, 75.6 tok/s at 258K input + 4K output (3 runs), 135 tok/s selected warm short-prompt decode. Pinned vLLM, CPU offload, MTP, P2P guide. Dual RTX 4090 testing notes (unvalidated).

moe quantization mtp multi-gpu dual-gpu rtx3090 int4 fp8 vllm local-llm llm-inference qwen speculative-decoding rtx4090 rtx-3090 cpu-offloading qwen3-8 qwen38 qwen3-8-flash-next 256k-context
  • Updated Sep 5, 2026
  • Python

thc1006 / qwen3.8-speculative-decoding-rtx3090

Star 2

Preregistered, DOI-archived llama.cpp study: Qwen3.8-27B on one RTX 3090 (sm_86). draft-mtp at --spec-draft-n-max 2 is +59.8% [+57.0, +62.8] server-reported decode over 25 purposive prompts; draft-dflash (PR #27342) +51.9%, not separated from it. Decode energy -37.1%, uncalibrated. 23-25 of 25 prompts diverge from serial greedy at 1600 tokens.

reproducible-research cuda quantization mtp ampere energy-efficiency inference-optimization mixture-of-experts gpu-benchmark kv-cache long-context llama-cpp local-llm llm-inference qwen gguf speculative-decoding qwen3 multi-token-prediction rtx-3090
  • Updated Sep 2, 2026
  • Python

Add this topic to your repo

To associate your repository with the rtx-3090 topic, visit your repo's landing page and select "manage topics."

Learn more