Index
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form
A
- addPastTokensSize(int) - Method in class chat.octet.model.beans.Status
- AIX - Static variable in class chat.octet.model.utils.Platform
- ANDROID - Static variable in class chat.octet.model.utils.Platform
- appendNextToken(Token) - Method in class chat.octet.model.beans.Status
- appendTokens(int[]) - Method in class chat.octet.model.beans.Status
- ARCH - Static variable in class chat.octet.model.utils.Platform
- ASSISTANT - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole
-
Assistant role
B
- batch - Variable in class chat.octet.model.beans.LlamaContextParams
-
prompt processing batch size.
- batchDecode(int, int[], int, int) - Static method in class chat.octet.model.LlamaService
C
- chat(GenerateParameter, String) - Method in class chat.octet.model.Model
-
Start a conversation and chat in streaming format.
- chat(GenerateParameter, String, String) - Method in class chat.octet.model.Model
-
Start a conversation and chat in streaming format.
- chat(String) - Method in class chat.octet.model.Model
-
Start a conversation and chat in streaming format.
- chat(String, String) - Method in class chat.octet.model.Model
-
Start a conversation and chat in streaming format.
- chat.octet.model - package chat.octet.model
- chat.octet.model.beans - package chat.octet.model.beans
- chat.octet.model.components.criteria - package chat.octet.model.components.criteria
- chat.octet.model.components.criteria.impl - package chat.octet.model.components.criteria.impl
- chat.octet.model.components.processor - package chat.octet.model.components.processor
- chat.octet.model.components.processor.impl - package chat.octet.model.components.processor.impl
- chat.octet.model.exceptions - package chat.octet.model.exceptions
- chat.octet.model.parameters - package chat.octet.model.parameters
- chat.octet.model.utils - package chat.octet.model.utils
- chatCompletions(GenerateParameter, String) - Method in class chat.octet.model.Model
-
Start a conversation and chat.
- chatCompletions(GenerateParameter, String, String) - Method in class chat.octet.model.Model
-
Start a conversation and chat.
- chatCompletions(String) - Method in class chat.octet.model.Model
-
Start a conversation and chat.
- ChatMessage - Class in chat.octet.model.beans
-
Chat message entity
- ChatMessage() - Constructor for class chat.octet.model.beans.ChatMessage
- ChatMessage(ChatMessage.ChatRole, String) - Constructor for class chat.octet.model.beans.ChatMessage
- ChatMessage.ChatRole - Enum Class in chat.octet.model.beans
-
Chat role define
- clearCache() - Method in class chat.octet.model.Generator
-
Clear context cache at the end of generation
- clearCache(int) - Static method in class chat.octet.model.LlamaService
- clearCache(int, int, int) - Static method in class chat.octet.model.LlamaService
- close() - Method in class chat.octet.model.Model
-
Close the model and release resources.
- CompletionResult - Class in chat.octet.model.beans
-
Completion result entity
- CompletionResult() - Constructor for class chat.octet.model.beans.CompletionResult
- completions(GenerateParameter, String) - Method in class chat.octet.model.Model
-
Generate complete text.
- completions(String) - Method in class chat.octet.model.Model
-
Generate complete text.
- copyToStatus(Status) - Method in class chat.octet.model.beans.Status
- createNewContextWithModel(LlamaContextParams) - Static method in class chat.octet.model.LlamaService
- criteria(int[], float[], Object...) - Method in class chat.octet.model.components.criteria.impl.MaxTimeCriteria
- criteria(int[], float[], Object...) - Method in interface chat.octet.model.components.criteria.StoppingCriteria
-
Stopping criteria
- criteria(int[], float[], Object...) - Method in class chat.octet.model.components.criteria.StoppingCriteriaList
- ctx - Variable in class chat.octet.model.beans.LlamaContextParams
-
text context size.
- CustomBiasLogitsProcessor - Class in chat.octet.model.components.processor.impl
- CustomBiasLogitsProcessor(Map<Integer, String>, int) - Constructor for class chat.octet.model.components.processor.impl.CustomBiasLogitsProcessor
D
- DecodeException - Exception Class in chat.octet.model.exceptions
-
Batch decode exception
- DecodeException(String) - Constructor for exception class chat.octet.model.exceptions.DecodeException
- DecodeException(String, Throwable) - Constructor for exception class chat.octet.model.exceptions.DecodeException
- decodeToken(int...) - Static method in class chat.octet.model.TokenDecoder
- DISABLED - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode
E
- embedding - Variable in class chat.octet.model.beans.LlamaContextParams
-
embedding mode only.
- equals(Object) - Method in class chat.octet.model.beans.Token
F
- f16KV - Variable in class chat.octet.model.beans.LlamaContextParams
-
use fp16 for KV cache.
- FINISHED - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Completed generation.
- FinishReason - Enum Class in chat.octet.model.beans
-
Token generate status
- FREEBSD - Static variable in class chat.octet.model.utils.Platform
G
- generate(GenerateParameter, String) - Method in class chat.octet.model.Model
-
Generate text in stream format.
- generate(String) - Method in class chat.octet.model.Model
-
Generate text in stream format.
- GenerateParameter - Class in chat.octet.model.parameters
-
Generate parameter
- GenerateParameter() - Constructor for class chat.octet.model.parameters.GenerateParameter
- GenerateParameter.MirostatMode - Enum Class in chat.octet.model.parameters
-
Mirostat sampling mode define
- Generator - Class in chat.octet.model
-
Generation iterator, which outputs tokens one by one in a stream format.
- Generator(GenerateParameter, String) - Constructor for class chat.octet.model.Generator
-
Create regular generator
- Generator(GenerateParameter, String, Status) - Constructor for class chat.octet.model.Generator
-
Create continuous conversation generator
- getContextSize() - Static method in class chat.octet.model.LlamaService
- getEmbeddings() - Static method in class chat.octet.model.LlamaService
- getEmbeddingSize() - Static method in class chat.octet.model.LlamaService
- getFinishReason() - Method in class chat.octet.model.beans.Status
- getFinishReason() - Method in class chat.octet.model.Generator
-
Return the finished reason for the last token.
- getGeneratedCompleteText() - Method in class chat.octet.model.beans.Status
- getGeneratedCompleteText() - Method in class chat.octet.model.Generator
-
Return the generated complete text.
- getLlamaContextDefaultParams() - Static method in class chat.octet.model.LlamaService
- getLlamaModelDefaultParams() - Static method in class chat.octet.model.LlamaService
- getLogits(int) - Static method in class chat.octet.model.LlamaService
- getLogitsIndex() - Method in class chat.octet.model.beans.Status
- getMaxDevices() - Static method in class chat.octet.model.LlamaService
- getOSType() - Static method in class chat.octet.model.utils.Platform
- getSamplingMetrics(boolean) - Static method in class chat.octet.model.LlamaService
- getStatus() - Method in class chat.octet.model.Generator
- getSystemInfo() - Static method in class chat.octet.model.LlamaService
- getTokenBOS() - Static method in class chat.octet.model.LlamaService
- getTokenEOS() - Static method in class chat.octet.model.LlamaService
- getTokenNL() - Static method in class chat.octet.model.LlamaService
- getTokenScore(int) - Static method in class chat.octet.model.LlamaService
- getTokenText(int) - Static method in class chat.octet.model.LlamaService
- getTokenType(int) - Static method in class chat.octet.model.LlamaService
- getUtf8ByteLength(byte) - Static method in class chat.octet.model.TokenDecoder
- getVocabSize() - Static method in class chat.octet.model.LlamaService
- getVocabType() - Static method in class chat.octet.model.LlamaService
- GNU - Static variable in class chat.octet.model.utils.Platform
- gpuLayers - Variable in class chat.octet.model.beans.LlamaModelParams
-
number of layers to store in VRAM.
H
- hashCode() - Method in class chat.octet.model.beans.Token
- hasNext() - Method in class chat.octet.model.Generator
I
- initNative() - Static method in class chat.octet.model.LlamaService
- isFinished() - Method in enum class chat.octet.model.beans.FinishReason
-
Check if the token has been completed else return false.
- isLinux() - Static method in class chat.octet.model.utils.Platform
- isMac() - Static method in class chat.octet.model.utils.Platform
- isMlockSupported() - Static method in class chat.octet.model.LlamaService
- isMmapSupported() - Static method in class chat.octet.model.LlamaService
- isMultiByte(int) - Static method in class chat.octet.model.TokenDecoder
- isWindows() - Static method in class chat.octet.model.utils.Platform
K
L
- LENGTH - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Generation has exceeded the maximum token limit and has been truncated.
- LIB_RESOURCE_PATH - Static variable in class chat.octet.model.utils.Platform
- LINUX - Static variable in class chat.octet.model.utils.Platform
- LLAMA_TOKEN_TYPE_BYTE - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Byte type.
- LLAMA_TOKEN_TYPE_CONTROL - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Control type.
- LLAMA_TOKEN_TYPE_NORMAL - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Normal type.
- LLAMA_TOKEN_TYPE_UNDEFINED - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Undefined type.
- LLAMA_TOKEN_TYPE_UNKNOWN - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Unknown type.
- LLAMA_TOKEN_TYPE_UNUSED - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
Unused type.
- LLAMA_TOKEN_TYPE_USER_DEFINED - Enum constant in enum class chat.octet.model.beans.LlamaTokenType
-
User defined type.
- llamaBackendFree() - Static method in class chat.octet.model.LlamaService
- llamaBackendInit(boolean) - Static method in class chat.octet.model.LlamaService
- LlamaContextParams - Class in chat.octet.model.beans
-
Llama context params entity
- LlamaContextParams() - Constructor for class chat.octet.model.beans.LlamaContextParams
- LlamaModelParams - Class in chat.octet.model.beans
-
Llama model params entity
- LlamaModelParams() - Constructor for class chat.octet.model.beans.LlamaModelParams
- LlamaService - Class in chat.octet.model
-
Llama.cpp API
- LlamaService() - Constructor for class chat.octet.model.LlamaService
- LlamaTokenType - Enum Class in chat.octet.model.beans
-
Llama token type define
- loadLibraryResource() - Static method in class chat.octet.model.utils.Platform
- loadLlamaGrammar(String) - Static method in class chat.octet.model.LlamaService
- loadLlamaModelFromFile(String, LlamaModelParams) - Static method in class chat.octet.model.LlamaService
- loadLoraModelFromFile(String, float, String, int) - Static method in class chat.octet.model.LlamaService
- logitsAll - Variable in class chat.octet.model.beans.LlamaContextParams
-
the llama_eval() call computes all logits, not just the last one.
- LogitsProcessor - Interface in chat.octet.model.components.processor
-
Customize a processor to adjust the probability distribution of words and control the generation of model inference results.
- LogitsProcessorList - Class in chat.octet.model.components.processor
-
Stopping criteria list
- LogitsProcessorList() - Constructor for class chat.octet.model.components.processor.LogitsProcessorList
- LogitsProcessorList(Collection<? extends LogitsProcessor>) - Constructor for class chat.octet.model.components.processor.LogitsProcessorList
M
- MAC - Static variable in class chat.octet.model.utils.Platform
- mainGpu - Variable in class chat.octet.model.beans.LlamaModelParams
-
the GPU that is used for scratch and small tensors.
- MaxTimeCriteria - Class in chat.octet.model.components.criteria.impl
- MaxTimeCriteria(long) - Constructor for class chat.octet.model.components.criteria.impl.MaxTimeCriteria
- MaxTimeCriteria(long, long) - Constructor for class chat.octet.model.components.criteria.impl.MaxTimeCriteria
- metrics() - Method in class chat.octet.model.Model
-
Print generation metrics.
- Metrics - Class in chat.octet.model.beans
-
Generation metrics
- Metrics() - Constructor for class chat.octet.model.beans.Metrics
- METRICS_TEMPLATE - Static variable in class chat.octet.model.beans.Metrics
- mlock - Variable in class chat.octet.model.beans.LlamaModelParams
-
force system to keep model in RAM.
- mmap - Variable in class chat.octet.model.beans.LlamaModelParams
-
use mmap if possible.
- Model - Class in chat.octet.model
-
LLama model, which provides functions for generating and chatting conversations.
- Model(ModelParameter) - Constructor for class chat.octet.model.Model
- Model(String) - Constructor for class chat.octet.model.Model
- ModelException - Exception Class in chat.octet.model.exceptions
-
Model exception
- ModelException(String) - Constructor for exception class chat.octet.model.exceptions.ModelException
- ModelException(String, Throwable) - Constructor for exception class chat.octet.model.exceptions.ModelException
- ModelParameter - Class in chat.octet.model.parameters
-
Llama model parameters
- ModelParameter() - Constructor for class chat.octet.model.parameters.ModelParameter
- mulMatQ - Variable in class chat.octet.model.beans.LlamaContextParams
-
if true, use experimental mul_mat_q kernels.
N
- NETBSD - Static variable in class chat.octet.model.utils.Platform
- next() - Method in class chat.octet.model.Generator
-
Output next token.
- NoBadWordsLogitsProcessor - Class in chat.octet.model.components.processor.impl
- NoBadWordsLogitsProcessor(int[]) - Constructor for class chat.octet.model.components.processor.impl.NoBadWordsLogitsProcessor
- NONE - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Default type.
O
P
- Platform - Class in chat.octet.model.utils
- processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.impl.CustomBiasLogitsProcessor
- processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.impl.NoBadWordsLogitsProcessor
- processor(int[], float[], Object...) - Method in interface chat.octet.model.components.processor.LogitsProcessor
-
Logits processor
- processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.LogitsProcessorList
- PromptBuilder - Class in chat.octet.model.utils
-
Prompt builder
- PromptBuilder() - Constructor for class chat.octet.model.utils.PromptBuilder
R
- release() - Static method in class chat.octet.model.LlamaService
- removeAllChatStatus() - Method in class chat.octet.model.Model
- removeChatStatus(String) - Method in class chat.octet.model.Model
- reset() - Method in class chat.octet.model.beans.Status
- ropeFreqBase - Variable in class chat.octet.model.beans.LlamaContextParams
-
RoPE base frequency.
- ropeFreqScale - Variable in class chat.octet.model.beans.LlamaContextParams
-
RoPE frequency scaling factor.
S
- sampling(float[], int[], int, float, float, float, boolean, int, float, float, float, int, float, float, float, int, int) - Static method in class chat.octet.model.LlamaService
- seed - Variable in class chat.octet.model.beans.LlamaContextParams
-
RNG seed, -1 for random.
- SOLARIS - Static variable in class chat.octet.model.utils.Platform
- Status - Class in chat.octet.model.beans
- Status() - Constructor for class chat.octet.model.beans.Status
- Status(Status) - Constructor for class chat.octet.model.beans.Status
- STOP - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Generation stopped by StoppingCriteria.
- StoppingCriteria - Interface in chat.octet.model.components.criteria
-
Customize a controller to implement stop rule control for model inference.
- StoppingCriteriaList - Class in chat.octet.model.components.criteria
-
Stopping criteria list
- StoppingCriteriaList() - Constructor for class chat.octet.model.components.criteria.StoppingCriteriaList
- StoppingCriteriaList(Collection<? extends StoppingCriteria>) - Constructor for class chat.octet.model.components.criteria.StoppingCriteriaList
- subInputIds(int) - Method in class chat.octet.model.beans.Status
- subInputIds(int, int) - Method in class chat.octet.model.beans.Status
- SYSTEM - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole
-
System prompt
T
- tensorSplit - Variable in class chat.octet.model.beans.LlamaModelParams
-
how to split layers across multiple GPUs (size: LLAMA_MAX_DEVICES).
- threads - Variable in class chat.octet.model.beans.LlamaContextParams
-
number of threads used for generation.
- threadsBatch - Variable in class chat.octet.model.beans.LlamaContextParams
-
number of threads used for prompt and batch processing.
- toAssistant(String) - Static method in class chat.octet.model.beans.ChatMessage
- Token - Class in chat.octet.model.beans
-
Token
- Token(int, LlamaTokenType, String) - Constructor for class chat.octet.model.beans.Token
- TokenDecoder - Class in chat.octet.model
-
Token decoder
- tokenize(byte[], int, int[], int, boolean) - Static method in class chat.octet.model.LlamaService
- tokenize(String, boolean) - Static method in class chat.octet.model.LlamaService
- tokenToPiece(int, byte[], int) - Static method in class chat.octet.model.LlamaService
- toPrompt(String) - Static method in class chat.octet.model.utils.PromptBuilder
- toPrompt(String, String) - Static method in class chat.octet.model.utils.PromptBuilder
- toString() - Method in enum class chat.octet.model.beans.LlamaTokenType
- toString() - Method in class chat.octet.model.beans.Metrics
- toString() - Method in class chat.octet.model.Model
- toSystem(String) - Static method in class chat.octet.model.beans.ChatMessage
- toUser(String) - Static method in class chat.octet.model.beans.ChatMessage
- TRUNCATED - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Generation has exceeded the maximum context limit and has been truncated.
U
- UNKNOWN - Enum constant in enum class chat.octet.model.beans.FinishReason
-
Unknown type, no available token state.
- UNSPECIFIED - Static variable in class chat.octet.model.utils.Platform
- updateFinishReason(FinishReason) - Method in class chat.octet.model.beans.Token
- USER - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole
-
User role
V
- V1 - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode
- V2 - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode
- valueOf(String) - Static method in enum class chat.octet.model.beans.ChatMessage.ChatRole
-
Returns the enum constant of this class with the specified name.
- valueOf(String) - Static method in enum class chat.octet.model.beans.FinishReason
-
Returns the enum constant of this class with the specified name.
- valueOf(String) - Static method in enum class chat.octet.model.beans.LlamaTokenType
-
Returns the enum constant of this class with the specified name.
- valueOf(String) - Static method in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode
-
Returns the enum constant of this class with the specified name.
- valueOfType(int) - Static method in enum class chat.octet.model.beans.LlamaTokenType
- values() - Static method in enum class chat.octet.model.beans.ChatMessage.ChatRole
-
Returns an array containing the constants of this enum class, in the order they are declared.
- values() - Static method in enum class chat.octet.model.beans.FinishReason
-
Returns an array containing the constants of this enum class, in the order they are declared.
- values() - Static method in enum class chat.octet.model.beans.LlamaTokenType
-
Returns an array containing the constants of this enum class, in the order they are declared.
- values() - Static method in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode
-
Returns an array containing the constants of this enum class, in the order they are declared.
- vocabOnly - Variable in class chat.octet.model.beans.LlamaModelParams
-
only load the vocabulary, no weights.
W
- WINDOWS - Static variable in class chat.octet.model.utils.Platform
- WINDOWSCE - Static variable in class chat.octet.model.utils.Platform
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form