KRASISGitHub

Open-source MoE inference

More model.
Same machine.

Run multi-hundred-billion-parameter models locally by pairing NVIDIA VRAM with system memory—without giving up fast GPU prompt processing.

Install KrasisView source
  • Linux + WSL2
  • Native Windows
  • NVIDIA Ampere+
Krasis neon circuit logo
MODEL STORESystem RAM
ACCELERATEDGPU VRAM
397Bmodels proven
1× GPUsingle-machine runs
Rust / CUDAperformance path
100% openfree and inspectable

Built beyond the VRAM wall

One runtime.
Two optimised paths.

Large MoE models do not need every expert for every token. Krasis builds around that fact, using GPU compute aggressively while system memory provides the capacity ordinary graphics cards cannot.

01Prompt processing

Full GPU prefill

Prompt tokens are processed in parallel on the GPU, with model layers streamed through VRAM in high-throughput batches.

02Token generation

Memory-aware decode

Hot experts stay close in VRAM while the rest remain available in system memory, allowing models far larger than GPU capacity.

03Runtime safety

Measured at startup

Krasis calibrates real memory use for the loaded model, GPU and prompt length instead of relying on one-size-fits-all limits.

Hybrid memory

Keep the whole model close. Keep the active model closer.

Source weights and optimised caches live in system memory. Krasis continuously manages the working set in VRAM, adapting to the model, hardware and request in front of it.

  • INT4 expert caches for production workloads
  • Compact HQQ attention and KV cache modes
  • OpenAI-compatible chat endpoint
CAPACITYSystem memoryModel + cache
THROUGHPUTGPU memoryActive working set

Reproducible results

Large-model proof,
not just a promise.

Selected timing-disabled runs from the public benchmark ledger. Decode is the internal engine measurement; systems, models and prompt lengths differ, so rows are evidence—not a direct ranking.

201.4B

Step-3.7-Flash

RTX PRO 6000 · 96 GB

5,261tok/sBest measured prefill
55.40tok/sInternal decode
397B

Ornith 1.0

RTX PRO 6000 · 96 GB

2,354.5tok/sBest measured prefill
23.58tok/sInternal decode
397B

Qwen3.5

RTX 5090 · 32 GB

973.8tok/sBest measured prefill
10.04tok/sInternal decode

Every published number links back to its runtime configuration and raw benchmark record.

Explore all benchmarks

Start locally

From zero to serving.

Krasis installs into its own managed environment. Bring an NVIDIA GPU, enough system RAM and disk for your model, and let the launcher guide the rest.

Linux / WSL2
All releases ↗
01curl -sSf https://raw.githubusercontent.com/brontoguana/krasis/main/install.sh | bash
02krasis-setup
03krasis
Native Windows installerPrivate runtime. Start Menu launcher. No system Python changes.
Download

What you need

Built for serious local machines.

  • NVIDIA GPUAmpere generation or newer with current CUDA drivers.
  • System memoryEnough RAM for the selected model’s quantised expert cache.
  • Fast local storageSpace for source weights and generated cache artifacts.

The model is ready

Stop sizing the model to the GPU.

Run larger open models on hardware you control, with a runtime you can inspect from end to end.

Get Krasis on GitHubAsk a question