YarnAttnFactor

YarnAttnFactor scales attention logit magnitudes as part of the YaRN algorithm. It compensates for the attention-softmax becoming too flat at extreme positions. Relevant only when RopeScalingType is Yarn.

Quick reference

Type	float?
Default	null (use model default)
Range	Typical 1.0 – 1.5
Category	YaRN position encoding
Field on	ContextParameters.YarnAttnFactor

What it does

As positions grow beyond the training window, YaRN mathematically applies a scaling to attention magnitudes. YarnAttnFactor controls this. The YaRN paper derives a value like 0.1 × log(scale) + 1.0 as a reasonable choice; the model’s metadata usually carries the correct value.

• null — use model default (recommended).
• Specific float — override.

When to change it

Scenario	Value
Default	null
Research / YaRN tuning	Per YaRN paper formula

Rarely touched in production.

Example

var preset = new Llama32Preset();
preset.ContextParameters.RopeScalingType = RopeScalingType.Yarn;
// preset.ContextParameters.YarnAttnFactor = null; // default — from model

Interactions

• RopeScalingType — must be Yarn.
• Other YaRN knobs operate together.

What’s next

• YarnOrigCtx — the primary YaRN field you might touch.
• Context parameters hub — all context knobs.