Documentation – Extensibility

Net: Custom model loader

Thu, 23 Apr 2026 00:00:00 +0000

IModelLoader is the contract the engine uses to turn ModelSourceParameters into an ILlamaModel. Substituting this interface gives you full control over the model-loading pipeline — file resolution, native llama.cpp model creation, inference parameter application, and any diagnostics you want to inject around it.

This is the broadest extensibility point. Prefer IModelFileProvider if you only need to change where the model file comes from; IModelLoader is for changing how it is loaded.

Interface reference

namespace Aspose.LLM.Abstractions.Interfaces;

public interface IModelLoader
{
    Task<ILlamaModel> LoadModelAsync(
        ModelSourceParameters modelParameters,
        ModelInferenceParameters inferenceParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default);
}

The method:

Takes ModelSourceParameters (where to find the model) and ModelInferenceParameters (how to load it — GpuLayers, SplitMode, etc.).
Reports progress via IProgress<double> (0.0 to 1.0) during long operations.
Returns an ILlamaModel — the loaded model ready for inference.
Throws ArgumentNullException on null modelParameters.
Throws InvalidOperationException when the model cannot be loaded.

Implementation skeleton

using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Abstractions.Parameters;

public class MyModelLoader : IModelLoader
{
    public async Task<ILlamaModel> LoadModelAsync(
        ModelSourceParameters modelParameters,
        ModelInferenceParameters inferenceParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default)
    {
        if (modelParameters is null)
            throw new ArgumentNullException(nameof(modelParameters));

        // 1. Resolve the model file path (use IModelFileProvider or your own logic).
        progress?.Report(0.1);
        string modelFilePath = await ResolveModelFileAsync(modelParameters, cancellationToken);

        // 2. Instrument or validate as needed.
        progress?.Report(0.5);
        ValidateModelFile(modelFilePath);

        // 3. Delegate to the SDK's native loading path, or call llama.cpp yourself.
        progress?.Report(0.9);
        ILlamaModel model = await LoadIntoNativeMemoryAsync(
            modelFilePath,
            inferenceParameters,
            cancellationToken);

        progress?.Report(1.0);
        return model;
    }

    // ... helper methods
}

Implementing LoadIntoNativeMemoryAsync from scratch means wrapping the SDK’s P/Invoke layer (Aspose.LLM.Interop). That is advanced work — you are essentially recreating ModelManager with custom behavior. In most cases, the simpler path is:

Use the default model loading via ModelManager.
Wrap it in a decorator that adds instrumentation, caching, or retries.

Decorator pattern

using Microsoft.Extensions.Logging;
using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Abstractions.Parameters;

public class LoggingModelLoader : IModelLoader
{
    private readonly IModelLoader _inner;
    private readonly ILogger<LoggingModelLoader> _logger;

    public LoggingModelLoader(IModelLoader inner, ILogger<LoggingModelLoader> logger)
    {
        _inner = inner;
        _logger = logger;
    }

    public async Task<ILlamaModel> LoadModelAsync(
        ModelSourceParameters modelParameters,
        ModelInferenceParameters inferenceParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default)
    {
        _logger.LogInformation("Loading model from {Source}",
            modelParameters.ModelFilePath ?? modelParameters.HuggingFaceRepoId);

        var sw = System.Diagnostics.Stopwatch.StartNew();
        try
        {
            var model = await _inner.LoadModelAsync(
                modelParameters, inferenceParameters, progress, cancellationToken);
            _logger.LogInformation("Model loaded in {Elapsed}", sw.Elapsed);
            return model;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Model load failed after {Elapsed}", sw.Elapsed);
            throw;
        }
    }
}

Registration

services.AddLlamaServices(new Qwen25Preset());
services.AddSingleton<IModelLoader, MyModelLoader>();

If you want to decorate the default loader, keep the default registration and wrap it — but the default ModelManager is registered as a concrete type (ModelManager), not as IModelLoader. You may need to adapt your decorator accordingly, or contact Aspose support for guidance on the current wiring.

What’s next

Custom file provider — narrower surface, often what you actually need.
Extensibility overview — when to use which interface.
Dependency injection — how AddLlamaServices wires services.

Net: Custom file provider

Thu, 23 Apr 2026 00:00:00 +0000

IModelFileProvider abstracts file resolution — the step that turns a ModelSourceParameters into a local file path. The default SDK ships two providers: LocalFilesystemProvider (for ModelFilePath) and HuggingFaceProvider (for HuggingFaceRepoId + HuggingFaceFileName).

Implement a custom provider when you need to source models from a location the defaults do not cover — private S3 buckets, artifact repositories (Artifactory, Nexus), encrypted storage, or signed internal model catalogs.

This is a narrower surface than IModelLoader and usually what you actually need.

Interface reference

namespace Aspose.LLM.Abstractions.Interfaces;

public interface IModelFileProvider
{
    Task<string> GetModelFileAsync(
        ModelSourceParameters modelParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default);
}

The method:

Takes ModelSourceParameters — resolve whichever fields are set, by priority.
Reports progress via IProgress<double> (0.0 to 1.0) for downloads.
Returns the path to a local file that downstream loaders can read.
Throws ArgumentNullException on null modelParameters.
Throws InvalidOperationException when the file cannot be retrieved.
Throws OperationCanceledException when the operation is canceled.

Example — S3 provider

using Amazon.S3;
using Amazon.S3.Model;
using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Abstractions.Parameters;

public class S3ModelFileProvider : IModelFileProvider
{
    private readonly IAmazonS3 _s3;
    private readonly string _bucketName;
    private readonly string _localCacheFolder;

    public S3ModelFileProvider(IAmazonS3 s3, string bucketName, string cacheFolder)
    {
        _s3 = s3;
        _bucketName = bucketName;
        _localCacheFolder = cacheFolder;
        Directory.CreateDirectory(_localCacheFolder);
    }

    public async Task<string> GetModelFileAsync(
        ModelSourceParameters modelParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default)
    {
        if (modelParameters is null)
            throw new ArgumentNullException(nameof(modelParameters));

        // Map the AsposeModelId field to an S3 key.
        string s3Key = modelParameters.AsposeModelId
            ?? throw new InvalidOperationException("AsposeModelId is required for S3 resolution.");

        string localPath = Path.Combine(_localCacheFolder, Path.GetFileName(s3Key));

        // Cache hit.
        if (File.Exists(localPath))
        {
            progress?.Report(1.0);
            return localPath;
        }

        // Download.
        var response = await _s3.GetObjectAsync(
            new GetObjectRequest { BucketName = _bucketName, Key = s3Key },
            cancellationToken);

        using var outStream = File.Create(localPath);
        long total = response.ContentLength;
        long read = 0;
        byte[] buffer = new byte[81920];
        int n;
        while ((n = await response.ResponseStream.ReadAsync(buffer, cancellationToken)) > 0)
        {
            await outStream.WriteAsync(buffer.AsMemory(0, n), cancellationToken);
            read += n;
            progress?.Report((double)read / total);
        }

        return localPath;
    }
}

Usage in a preset: set AsposeModelId to the S3 key, leave HuggingFaceRepoId unset.

Example — air-gapped internal registry

For air-gapped deployments where models are prepopulated in a shared directory:

public class InternalRegistryProvider : IModelFileProvider
{
    private readonly string _registryRoot;

    public InternalRegistryProvider(string registryRoot) => _registryRoot = registryRoot;

    public Task<string> GetModelFileAsync(
        ModelSourceParameters modelParameters,
        IProgress<double>? progress = null,
        CancellationToken cancellationToken = default)
    {
        string relative = modelParameters.AsposeModelId
            ?? throw new InvalidOperationException("AsposeModelId required.");

        string path = Path.Combine(_registryRoot, relative);
        if (!File.Exists(path))
            throw new InvalidOperationException($"Model not found: {path}");

        progress?.Report(1.0);
        return Task.FromResult(path);
    }
}

Priority handling

ModelSourceParameters has three priority-ordered fields: ModelFilePath, AsposeModelId, HuggingFaceRepoId + HuggingFaceFileName. Your provider decides which field(s) to respect. Common patterns:

Single-source provider: respect one field only (e.g., AsposeModelId for an internal registry). Let callers set that field explicitly.
Multi-source provider: honor the full priority chain — check ModelFilePath first, then your custom source, then fall back to Hugging Face.

Registration

using Microsoft.Extensions.DependencyInjection;
using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Core.DependencyInjection;

services.AddLlamaServices(new Qwen25Preset());
services.AddSingleton<IModelFileProvider>(new InternalRegistryProvider(@"/opt/models"));

The default ModelManager is registered by AddLlamaServices. Adding your own IModelFileProvider registration overrides it only if the downstream consumer (ModelManager) is wired to consume IModelFileProvider directly — in the current SDK, ModelManager uses concrete providers (LocalFilesystemProvider, HuggingFaceProvider). You may need a custom IModelLoader that uses your IModelFileProvider instead of ModelManager.

Confirm the current wiring before committing to a custom provider — contact Aspose support if the injection point does not behave as you expect.

What’s next

Custom model loader — broader surface when you need full pipeline control.
Model source parameters — what ModelSourceParameters carries.
Dependency injection — standard service wiring.

Net: Custom prompt formatter

Thu, 23 Apr 2026 00:00:00 +0000

IPromptFormatter abstracts prompt formatting — how each ChatMessage is rendered into the model-ready text before inference, and how the raw model output is cleaned up before the API returns.

Substitute the default formatter when your model uses a custom chat template that the SDK’s built-in templates do not match, or when you want to inject post-processing on responses (stripping chain-of-thought blocks, tagging entities, redaction).

Interface reference

namespace Aspose.LLM.Abstractions.Interfaces;

public interface IPromptFormatter
{
    string? ArtificalEOSToken { get; }
    string FormatPrompt(ChatMessage message, bool addGenerationPrompt = false);
    string RefineResponse(string response);
}

Member	Purpose
`ArtificalEOSToken`	Optional stop token the engine respects during generation. Return `null` to use the model’s native EOS.
`FormatPrompt`	Renders a single `ChatMessage` into template-formatted text. Called once per turn when building the prompt. `addGenerationPrompt` is `true` for the final turn — append the model’s “assistant starts here” token.
`RefineResponse`	Cleans raw model output before the API returns. Typical work: strip template markup, trim control tokens.

Example — strip reasoning blocks

Qwen3 and DeepSeek-R1 emit <think>…</think> blocks before the actual answer. If you want to hide those from end users, post-process them away:

using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Abstractions.Models;
using System.Text;
using System.Text.RegularExpressions;

public class StripThinkFormatter : IPromptFormatter
{
    private readonly IPromptFormatter _inner;

    public StripThinkFormatter(IPromptFormatter inner) => _inner = inner;

    public string? ArtificalEOSToken => _inner.ArtificalEOSToken;

    public string FormatPrompt(ChatMessage message, bool addGenerationPrompt = false)
        => _inner.FormatPrompt(message, addGenerationPrompt);

    public string RefineResponse(string response)
    {
        // Delegate first to the inner formatter (which does template cleanup).
        string cleaned = _inner.RefineResponse(response);

        // Strip reasoning blocks.
        cleaned = Regex.Replace(
            cleaned,
            @"<think>[\s\S]*?</think>\s*",
            string.Empty,
            RegexOptions.Multiline);

        return cleaned.Trim();
    }
}

Example — fully custom template

For a model with a non-standard format, implement both FormatPrompt and RefineResponse end-to-end:

public class MyCustomFormatter : IPromptFormatter
{
    public string? ArtificalEOSToken => "<|eot|>";

    public string FormatPrompt(ChatMessage message, bool addGenerationPrompt = false)
    {
        if (message is null)
            throw new ArgumentNullException(nameof(message));

        var sb = new StringBuilder();

        switch (message.Role)
        {
            case "system":
                sb.Append("<|sys|>").Append(message.Content).Append("<|eot|>");
                break;
            case "user":
                sb.Append("<|usr|>").Append(message.Content).Append("<|eot|>");
                break;
            case "assistant":
                sb.Append("<|asst|>").Append(message.Content).Append("<|eot|>");
                break;
        }

        if (addGenerationPrompt && message.Role == "user")
            sb.Append("<|asst|>");

        return sb.ToString();
    }

    public string RefineResponse(string response)
    {
        // Trim anything after our EOS token.
        int eotIndex = response.IndexOf("<|eot|>", StringComparison.Ordinal);
        if (eotIndex >= 0)
            response = response.Substring(0, eotIndex);

        return response.Trim();
    }
}

Caveats

Template correctness is load-bearing. Getting FormatPrompt wrong produces garbled output — the model sees malformed sequences and generates nonsense.
Vision presets have their own templates. The chat templates for vision are handled internally by Aspose.LLM.Interop.Multimodal.VisualModelChatTemplates based on model metadata. A custom IPromptFormatter does not override vision template selection — that path is not yet extensible in the current release.
Built-in presets use their own formatters. Substituting IPromptFormatter in the DI container replaces the default, but each preset may wire its own formatter internally. Test carefully when substituting; when in doubt, contact Aspose support.

Registration

using Microsoft.Extensions.DependencyInjection;
using Aspose.LLM.Abstractions.Interfaces;
using Aspose.LLM.Core.DependencyInjection;

services.AddLlamaServices(new Qwen25Preset());
services.AddSingleton<IPromptFormatter, MyCustomFormatter>();

What’s next

Chat templates (multimodal) — vision template auto-selection.
Chat sessions — how the formatter is invoked per turn.
Extensibility overview — other substitution points.

Net: Custom media processor

Thu, 23 Apr 2026 00:00:00 +0000

IMediaProcessor controls how raw image bytes become a ProcessedMedia structure the vision pipeline can evaluate. Implementing a custom processor lets you handle formats the default processor does not support, apply specific preprocessing (color spaces, DICOM extraction, sensor calibration), or add validation beyond size and format.

For everyday vision use, the default MediaManager implementation is enough — it handles JPEG, PNG, BMP, GIF, WebP with magic-byte detection and 50 MB size limits. Consider custom only when you have specific needs the default does not cover.

Interface reference

namespace Aspose.LLM.Core.Services;

public interface IMediaProcessor
{
    Task<ProcessedMedia> ProcessMediaAsync(
        byte[] mediaData,
        MediaProcessingOptions? options = null);

    bool IsSupportedFormat(byte[] mediaData);

    MediaFormat DetectFormat(byte[] mediaData);

    bool ValidateMedia(byte[] mediaData, out string? errorMessage);

    ProcessedMedia CreateProcessedMedia(
        byte[] mediaData,
        MediaFormat? knownFormat = null);
}

Method	Purpose
`ProcessMediaAsync`	Main entry point. Runs format detection, validation, and any preprocessing. Returns `ProcessedMedia`.
`IsSupportedFormat`	Lightweight check without full processing.
`DetectFormat`	Magic-byte detection; returns a `MediaFormat` enum value.
`ValidateMedia`	Returns `true` when input is acceptable; populates `errorMessage` on failure.
`CreateProcessedMedia`	Synchronous variant for cases where format is already known.

IMediaProcessor lives in Aspose.LLM.Core.Services. It is public, but in the Core assembly rather than Abstractions — usable after ILRepack merges, but not part of the minimal Aspose.LLM.Abstractions API surface.

`ProcessedMedia` structure

public class ProcessedMedia
{
    public byte[] ProcessedData { get; set; } = Array.Empty<byte>();
    public int Width { get; set; }
    public int Height { get; set; }
    // ... additional fields for embeddings, metadata, etc.
}

Full field list varies by SDK version — reference the API documentation at reference.aspose.com/llm/net/ for the current shape.

Example — support a custom format

The default processor rejects anything outside JPEG/PNG/BMP/GIF/WebP. If your source is TIFF, DICOM, or raw pixel buffers, you convert early in a custom processor:

using Aspose.LLM.Abstractions.Models;
using Aspose.LLM.Core.Services;

public class TiffAwareProcessor : IMediaProcessor
{
    private readonly IMediaProcessor _inner;

    public TiffAwareProcessor(IMediaProcessor inner) => _inner = inner;

    public async Task<ProcessedMedia> ProcessMediaAsync(
        byte[] mediaData,
        MediaProcessingOptions? options = null)
    {
        // Intercept TIFF (magic bytes 0x49 0x49 0x2A 0x00 for little-endian,
        // 0x4D 0x4D 0x00 0x2A for big-endian).
        if (IsTiff(mediaData))
        {
            byte[] pngBytes = ConvertTiffToPng(mediaData); // use System.Drawing or ImageSharp
            return await _inner.ProcessMediaAsync(pngBytes, options);
        }

        return await _inner.ProcessMediaAsync(mediaData, options);
    }

    public bool IsSupportedFormat(byte[] mediaData)
        => IsTiff(mediaData) || _inner.IsSupportedFormat(mediaData);

    public MediaFormat DetectFormat(byte[] mediaData)
        => _inner.DetectFormat(mediaData); // TIFF still shows as Unknown until conversion

    public bool ValidateMedia(byte[] mediaData, out string? errorMessage)
    {
        if (IsTiff(mediaData))
        {
            errorMessage = null;
            return true;
        }
        return _inner.ValidateMedia(mediaData, out errorMessage);
    }

    public ProcessedMedia CreateProcessedMedia(byte[] mediaData, MediaFormat? knownFormat = null)
        => _inner.CreateProcessedMedia(mediaData, knownFormat);

    private static bool IsTiff(byte[] data) =>
        data.Length >= 4 &&
        ((data[0] == 0x49 && data[1] == 0x49 && data[2] == 0x2A && data[3] == 0x00)
         || (data[0] == 0x4D && data[1] == 0x4D && data[2] == 0x00 && data[3] == 0x2A));

    private static byte[] ConvertTiffToPng(byte[] tiff) { /* ... */ throw new NotImplementedException(); }
}

Example — stricter validation

Add size or content-based validation on top of the defaults:

public class StricterMediaProcessor : IMediaProcessor
{
    private readonly IMediaProcessor _inner;
    private const int MaxDimension = 4096;

    public StricterMediaProcessor(IMediaProcessor inner) => _inner = inner;

    public Task<ProcessedMedia> ProcessMediaAsync(byte[] mediaData, MediaProcessingOptions? options = null)
        => _inner.ProcessMediaAsync(mediaData, options);

    public bool ValidateMedia(byte[] mediaData, out string? errorMessage)
    {
        if (!_inner.ValidateMedia(mediaData, out errorMessage))
            return false;

        // Extract dimensions via a fast image-header reader (ImageSharp, custom code).
        var (w, h) = ReadDimensions(mediaData);
        if (w > MaxDimension || h > MaxDimension)
        {
            errorMessage = $"Image dimensions exceed {MaxDimension}x{MaxDimension}.";
            return false;
        }

        return true;
    }

    public bool IsSupportedFormat(byte[] mediaData) => _inner.IsSupportedFormat(mediaData);
    public MediaFormat DetectFormat(byte[] mediaData) => _inner.DetectFormat(mediaData);
    public ProcessedMedia CreateProcessedMedia(byte[] mediaData, MediaFormat? knownFormat = null)
        => _inner.CreateProcessedMedia(mediaData, knownFormat);

    private static (int Width, int Height) ReadDimensions(byte[] data) { /* ... */ throw new NotImplementedException(); }
}

Registration

using Microsoft.Extensions.DependencyInjection;
using Aspose.LLM.Core.Services;
using Aspose.LLM.Core.DependencyInjection;

services.AddLlamaServices(new Qwen3VL2BPreset());
services.AddSingleton<IMediaProcessor, StricterMediaProcessor>();

The default MediaManager is wired into Engine as the IMediaProcessor implementation. Substituting via DI should replace it, but verify on your specific SDK version — the wiring can evolve between releases.

What’s next

Attaching images — the built-in format support MediaProcessor applies.
Multimodal context parameters — projector-side knobs.
Extensibility overview — other substitution points.

Documentation – Extensibility

Net: Custom model loader

Interface reference

Implementation skeleton

Decorator pattern

Registration

What’s next

Net: Custom file provider

Interface reference

Example — S3 provider

Example — air-gapped internal registry

Priority handling

Registration

What’s next

Net: Custom prompt formatter

Interface reference

Example — strip reasoning blocks

Example — fully custom template

Caveats

Registration

What’s next

Net: Custom media processor

Interface reference

ProcessedMedia structure

Example — support a custom format

Example — stricter validation

Registration

What’s next

`ProcessedMedia` structure