Package chat.octet.model.beans

Class LlamaModelParams

java.lang.Object

chat.octet.model.beans.LlamaModelParams

public class LlamaModelParams
extends Object

Llama model params entity

Author:

William

Field Summary

Fields

Modifier and Type	Field	Description
int	gpuLayers	number of layers to store in VRAM.
int	mainGpu	the GPU that is used for scratch and small tensors.
boolean	mlock	force system to keep model in RAM.
boolean	mmap	use mmap if possible.
float[]	tensorSplit	how to split layers across multiple GPUs (size: LLAMA_MAX_DEVICES).
boolean	vocabOnly	only load the vocabulary, no weights.

Constructor Summary

Constructors

Constructor	Description
LlamaModelParams()

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

gpuLayers

public int gpuLayers

number of layers to store in VRAM.

mainGpu

public int mainGpu

the GPU that is used for scratch and small tensors.

tensorSplit

public float[] tensorSplit

how to split layers across multiple GPUs (size: LLAMA_MAX_DEVICES).

vocabOnly

public boolean vocabOnly

only load the vocabulary, no weights.

mmap

public boolean mmap

use mmap if possible.

mlock

public boolean mlock

force system to keep model in RAM.

Constructor Details

LlamaModelParams

public LlamaModelParams()