UseGpu

UseGpu controls whether the mtmd layer runs the vision projector on GPU. Relevant only for vision presets; ignored on text-only.

Quick reference
Type bool?
Default null (use native default — typically true if a GPU is present)
Category Multimodal context
Field on MultimodalContextParameters.UseGpu

What it does

  • null — delegate to mtmd’s auto-detection. Usually correct.
  • true — force GPU. Requires a supported GPU.
  • false — force CPU. Keeps VRAM for the base model on memory-tight GPUs.

The projector is typically small (200 MB – 2 GB). GPU offload is fast on modest hardware; the main reason to keep it on CPU is freeing VRAM for the base model’s KV cache.

When to change it

Scenario Value
Default null
Keep all VRAM for the base model false
Force projector to GPU true

Example

var preset = new Qwen3VL2BPreset();
preset.MtmdContextParameters.UseGpu = false;           // projector on CPU
preset.BaseModelInferenceParameters.GpuLayers = 999;   // base model fully on GPU

Interactions

  • GpuLayers — base model offload; projector placement is independent.
  • PrintTimings — diagnose projector performance.

What’s next

PrintTimings