Package chat.octet.model.beans

Class LlamaContextParams

java.lang.Object

chat.octet.model.beans.LlamaContextParams

public class LlamaContextParams
extends Object

Llama context params entity

Author:

William

Field Summary

Fields

Modifier and Type	Field	Description
int	batch	prompt processing batch size.
int	ctx	text context size.
boolean	embedding	embedding mode only.
boolean	f16KV	use fp16 for KV cache.
boolean	logitsAll	the llama_eval() call computes all logits, not just the last one.
boolean	mulMatQ	if true, use experimental mul_mat_q kernels.
float	ropeFreqBase	RoPE base frequency.
float	ropeFreqScale	RoPE frequency scaling factor.
int	ropeScalingType	RoPE scaling type, from `enum llama_rope_scaling_type`.
int	seed	RNG seed, -1 for random.
int	threads	number of threads used for generation.
int	threadsBatch	number of threads used for prompt and batch processing.
float	yarnAttnFactor	YaRN magnitude scaling factor.
float	yarnBetaFast	YaRN low correction dim.
float	yarnBetaSlow	YaRN high correction dim.
float	yarnExtFactor	YaRN extrapolation mix factor, NaN = from model.
int	yarnOrigCtx	YaRN original context size.

Constructor Summary

Constructors

Constructor	Description
LlamaContextParams()

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

seed

public int seed

RNG seed, -1 for random.

ctx

public int ctx

text context size.

batch

public int batch

prompt processing batch size.

threads

public int threads

number of threads used for generation.

threadsBatch

public int threadsBatch

number of threads used for prompt and batch processing.

ropeScalingType

public int ropeScalingType

RoPE scaling type, from `enum llama_rope_scaling_type`.

See Also:

• LlamaRoPEScalingType

yarnExtFactor

public float yarnExtFactor

YaRN extrapolation mix factor, NaN = from model.

yarnAttnFactor

public float yarnAttnFactor

YaRN magnitude scaling factor.

yarnBetaFast

public float yarnBetaFast

YaRN low correction dim.

yarnBetaSlow

public float yarnBetaSlow

YaRN high correction dim.

yarnOrigCtx

public int yarnOrigCtx

YaRN original context size.

ropeFreqBase

public float ropeFreqBase

RoPE base frequency.

ropeFreqScale

public float ropeFreqScale

RoPE frequency scaling factor.

mulMatQ

public boolean mulMatQ

if true, use experimental mul_mat_q kernels.

f16KV

public boolean f16KV

use fp16 for KV cache.

logitsAll

public boolean logitsAll

the llama_eval() call computes all logits, not just the last one.

embedding

public boolean embedding

embedding mode only.

Constructor Details

LlamaContextParams

public LlamaContextParams()