CacheCleanupStrategy

CacheCleanupStrategy is the policy the engine applies when a session’s KV cache would overflow ContextSize. Five named strategies, each keeping a different subset of history.

Quick reference
Type CacheCleanupStrategy enum
Default RemoveOldestMessages
Values 5 — see table below
Category Chat session
Field on ChatParameters.CacheCleanupStrategy

What it does

When the next generation step would exceed the context window, the engine trims the cache per the active strategy, freeing tokens before continuing. The policy is applied automatically during generation and can also be triggered explicitly via AsposeLLMApi.ForceCacheCleanup(strategy).

Strategy Keeps Typical use
RemoveOldestMessages (default) System prompt + most recent turns General-purpose; preserves recency.
KeepSystemPromptOnly System prompt only Hard reset of session context.
KeepSystemPromptAndHalf System prompt + newer half of history Balanced recall and room for new turns.
KeepSystemPromptAndFirstUserMessage System prompt + first user turn Recall-heavy tasks where the original ask matters.
KeepSystemPromptAndLastUserMessage System prompt + most recent user turn Focus on current question, drop middle.

When to change it

Scenario Value
Default conversational chat RemoveOldestMessages
Anchored on a big original ask (debugging, iterative refinement) KeepSystemPromptAndFirstUserMessage
Sequence of independent Q&A KeepSystemPromptAndLastUserMessage
Hard reset when switching topics KeepSystemPromptOnly via ForceCacheCleanup
Long dialogues with gradual trimming KeepSystemPromptAndHalf

Example

using Aspose.LLM.Abstractions.Models;

var preset = new Qwen25Preset();
preset.ChatParameters.SystemPrompt =
    "You are a careful analyst. Always ground your answer in the user's original ask.";
preset.ChatParameters.CacheCleanupStrategy =
    CacheCleanupStrategy.KeepSystemPromptAndFirstUserMessage;

using var api = AsposeLLMApi.Create(preset);
// Even after 50 follow-ups, the model can still refer back to the first user turn.

Force a reset mid-session:

api.ForceCacheCleanup(CacheCleanupStrategy.KeepSystemPromptOnly);

Interactions

  • SystemPrompt — all strategies preserve it.
  • ContextSize — the ceiling this strategy serves.
  • DefragThreshold — compacts holes left behind by cleanup.
  • AsposeLLMApi.ForceCacheCleanup(strategy) — manual trigger with an override strategy.

What’s next

MaxTokens