Documentation – Multimodal context parameters

Net: UseGpu

Fri, 24 Apr 2026 00:00:00 +0000

UseGpu controls whether the mtmd layer runs the vision projector on GPU. Relevant only for vision presets; ignored on text-only.

Quick reference


Type	`bool?`
Default	`null` (use native default — typically true if a GPU is present)
Category	Multimodal context
Field on	`MultimodalContextParameters.UseGpu`

What it does

null — delegate to mtmd’s auto-detection. Usually correct.
true — force GPU. Requires a supported GPU.
false — force CPU. Keeps VRAM for the base model on memory-tight GPUs.

The projector is typically small (200 MB – 2 GB). GPU offload is fast on modest hardware; the main reason to keep it on CPU is freeing VRAM for the base model’s KV cache.

When to change it

Scenario	Value
Default	`null`
Keep all VRAM for the base model	`false`
Force projector to GPU	`true`

Example

var preset = new Qwen3VL2BPreset();
preset.MtmdContextParameters.UseGpu = false;           // projector on CPU
preset.BaseModelInferenceParameters.GpuLayers = 999;   // base model fully on GPU

Interactions

GpuLayers — base model offload; projector placement is independent.
PrintTimings — diagnose projector performance.

What’s next

Debugging vision — projector diagnostics.
Vision presets — built-in presets.
Multimodal context hub — all mtmd knobs.

Net: PrintTimings

Fri, 24 Apr 2026 00:00:00 +0000

PrintTimings turns on mtmd’s built-in per-step timing output — time spent tokenizing, running the projector, and evaluating chunks. Useful when a vision query is slow and you need to localize the bottleneck.

Quick reference


Type	`bool?`
Default	`null` (off)
Category	Multimodal context diagnostics
Field on	`MultimodalContextParameters.PrintTimings`

What it does

null or false — no timing output.
true — mtmd emits per-stage timings through the native logger. Requires EnableDebugLogging or equivalent output routing to see them.

Timings are noisy. Disable in production; keep on only while debugging.

When to change it

Scenario	Value
Default	`null`
Investigating vision first-token latency	`true`
Production	`null` / `false`

Example

var preset = new Qwen3VL2BPreset();
preset.EngineParameters.EnableDebugLogging = true;
preset.MtmdContextParameters.PrintTimings = true;

using var api = AsposeLLMApi.Create(preset, logger);
// Per-stage mtmd timings now appear in the logs.

Interactions

EnableDebugLogging — required for timing lines to reach the logger.
Verbosity — complementary verbosity knob.

What’s next

Debugging vision — full diagnostic playbook.
Verbosity — mtmd log-level knob.
Multimodal context hub — all mtmd knobs.

Net: ThreadCount

Fri, 24 Apr 2026 00:00:00 +0000

ThreadCount sets the number of threads mtmd uses for CPU-side multimodal work (image preprocessing, CPU portions of the projector).

Quick reference


Type	`int?`
Default	`null` (native default — often half the logical cores)
Category	Multimodal context
Field on	`MultimodalContextParameters.ThreadCount`

What it does

null — use mtmd’s heuristic.
Positive integer — use that many threads.

Multimodal preprocessing is short-lived per request. On a server handling many concurrent vision requests, cap ThreadCount to prevent per-request work from starving other requests.

When to change it

Scenario	Value
Default	`null`
Concurrent vision requests	`2` – `4` per request
Single-request, all cores available	`Environment.ProcessorCount / 2`

Example

var preset = new Qwen3VL2BPreset();
preset.MtmdContextParameters.ThreadCount = 4;

using var api = AsposeLLMApi.Create(preset);

Interactions

ContextParameters.NThreads — base-model generation threads; independent of mtmd’s thread count.

What’s next

Multimodal context hub — all mtmd knobs.
Context parameters — NThreads — base-model counterpart.

Net: Verbosity

Fri, 24 Apr 2026 00:00:00 +0000

Verbosity controls how much the mtmd (multimodal) layer logs. Higher values produce more detail, useful for diagnosing vision issues but noisy in production.

Quick reference


Type	`int?`
Default	`null` (use native default)
Range	`0` = Error, `1` = Warn, `2` = Info, `3` = Debug
Category	Multimodal context diagnostics
Field on	`MultimodalContextParameters.Verbosity`

What it does

Maps to the native mtmd log-level setting. Higher values include all lower-level logs plus their own.

0 — errors only.
1 — warnings and errors.
2 — info, warnings, errors.
3 — debug and everything lower.

Logs are visible when either EnableDebugLogging is on, or the logger passed to AsposeLLMApi.Create is configured to accept them.

When to change it

Scenario	Value
Default	`null`
Production	`0` or `1`
Diagnosing vision template / alignment	`3`

Example

var preset = new Qwen3VL2BPreset();
preset.EngineParameters.EnableDebugLogging = true;
preset.MtmdContextParameters.Verbosity = 3;
preset.MtmdContextParameters.PrintTimings = true;

using var api = AsposeLLMApi.Create(preset, logger);

Interactions

EnableDebugLogging — required for high-verbosity lines to reach the logger.
PrintTimings — per-stage timings as a complementary diagnostic.

What’s next

Debugging vision — tag taxonomy and diagnostic flow.
Multimodal context hub — all mtmd knobs.

Net: MediaMarker

Fri, 24 Apr 2026 00:00:00 +0000

MediaMarker overrides the placeholder token used in the vision chat template to mark where images are inserted. Default is model-specific and selected automatically — override only when you understand the exact format your model expects.

Quick reference


Type	`string?`
Default	`null` (use the chat template’s default marker)
Category	Multimodal context
Field on	`MultimodalContextParameters.MediaMarker`

What it does

Vision chat templates use a specific token or placeholder in the prompt to mark where the image embedding is inserted. Different model families use different markers — LLaVA uses one, Qwen-VL uses another, Gemma3 yet another. The SDK picks the correct one from the model’s metadata.

null (default) — template picks the model’s default.
Explicit string — override with your own marker.

Overriding without matching the model’s trained format produces garbled output — the image is inserted at the wrong position or with the wrong surrounding tokens.

When to change it

Scenario	Value
Default (recommended)	`null`
Custom GGUF with a non-standard marker	Exact marker the model was trained with

Very rarely needed. Most use cases leave this null.

Example

var preset = new Qwen3VL2BPreset();
// preset.MtmdContextParameters.MediaMarker = null; // default — correct choice

Experimental custom marker:

preset.MtmdContextParameters.MediaMarker = "<|image|>";
// Only if you know the model's training format expects this exact marker.

Interactions

Chat templates — which template is selected automatically.
Debugging vision — diagnose marker-related misalignment.

What’s next

Chat templates — supported template list.
Debugging vision — marker-related troubleshooting.
Multimodal context hub — all mtmd knobs.