Package chat.octet.model.beans

Class LlamaContextParams

java.lang.Object

chat.octet.model.beans.LlamaContextParams

public class LlamaContextParams
extends Object

Llama context params entity

Author:

William

Field Summary

Fields

Modifier and Type	Field	Description
int	batch	prompt processing batch size.
int	ctx	text context size.
boolean	embedding	embedding mode only.
boolean	f16KV	use fp16 for KV cache.
boolean	logitsAll	the llama_eval() call computes all logits, not just the last one.
boolean	mulMatQ	if true, use experimental mul_mat_q kernels.
float	ropeFreqBase	RoPE base frequency.
float	ropeFreqScale	RoPE frequency scaling factor.
int	seed	RNG seed, -1 for random.
int	threads	number of threads used for generation.
int	threadsBatch	number of threads used for prompt and batch processing.

Constructor Summary

Constructors

Constructor	Description
LlamaContextParams()

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

seed

public int seed

RNG seed, -1 for random.

ctx

public int ctx

text context size.

batch

public int batch

prompt processing batch size.

threads

public int threads

number of threads used for generation.

threadsBatch

public int threadsBatch

number of threads used for prompt and batch processing.

ropeFreqBase

public float ropeFreqBase

RoPE base frequency.

ropeFreqScale

public float ropeFreqScale

RoPE frequency scaling factor.

mulMatQ

public boolean mulMatQ

if true, use experimental mul_mat_q kernels.

f16KV

public boolean f16KV

use fp16 for KV cache.

logitsAll

public boolean logitsAll

the llama_eval() call computes all logits, not just the last one.

embedding

public boolean embedding

embedding mode only.

Constructor Details

LlamaContextParams

public LlamaContextParams()