Index

A B C D E F G H I K L M N O P R S T U V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form

A

addPastTokensSize(int) - Method in class chat.octet.model.beans.Status

AIX - Static variable in class chat.octet.model.utils.Platform

ALPACA - Enum constant in enum class chat.octet.model.enums.ModelType

ANDROID - Static variable in class chat.octet.model.utils.Platform

appendNextToken(Token) - Method in class chat.octet.model.beans.Status

appendTokens(int[]) - Method in class chat.octet.model.beans.Status

AQUILA - Enum constant in enum class chat.octet.model.enums.ModelType

ARCH - Static variable in class chat.octet.model.utils.Platform

ASSISTANT - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole

Assistant role

B

BAICHUAN - Enum constant in enum class chat.octet.model.enums.ModelType

batch - Variable in class chat.octet.model.beans.LlamaContextParams

prompt processing batch size.

batchDecode(int, int[], int, int) - Static method in class chat.octet.model.LlamaService

Batch decoding.

C

chat(GenerateParameter, String) - Method in class chat.octet.model.Model

Start a conversation and chat in streaming format.

chat(GenerateParameter, String, String) - Method in class chat.octet.model.Model

Start a conversation and chat in streaming format.

chat(String) - Method in class chat.octet.model.Model

Start a conversation and chat in streaming format.

chat(String, String) - Method in class chat.octet.model.Model

Start a conversation and chat in streaming format.

chat.octet.model - package chat.octet.model

chat.octet.model.beans - package chat.octet.model.beans

chat.octet.model.components.criteria - package chat.octet.model.components.criteria

chat.octet.model.components.criteria.impl - package chat.octet.model.components.criteria.impl

chat.octet.model.components.processor - package chat.octet.model.components.processor

chat.octet.model.components.processor.impl - package chat.octet.model.components.processor.impl

chat.octet.model.enums - package chat.octet.model.enums

chat.octet.model.exceptions - package chat.octet.model.exceptions

chat.octet.model.parameters - package chat.octet.model.parameters

chat.octet.model.utils - package chat.octet.model.utils

chatCompletions(GenerateParameter, String) - Method in class chat.octet.model.Model

Start a conversation and chat.

chatCompletions(GenerateParameter, String, String) - Method in class chat.octet.model.Model

Start a conversation and chat.

chatCompletions(String) - Method in class chat.octet.model.Model

Start a conversation and chat.

ChatMessage - Class in chat.octet.model.beans

Chat message entity

ChatMessage() - Constructor for class chat.octet.model.beans.ChatMessage

ChatMessage(ChatMessage.ChatRole, String) - Constructor for class chat.octet.model.beans.ChatMessage

ChatMessage.ChatRole - Enum Class in chat.octet.model.beans

Chat role define

clearCache(int) - Static method in class chat.octet.model.LlamaService

Clear cache in K-V sequences.

clearCache(int, int, int) - Static method in class chat.octet.model.LlamaService

Clear cache in K-V sequences.

close() - Method in class chat.octet.model.Model

Close the model and release resources.

COMMON - Enum constant in enum class chat.octet.model.enums.ModelType

CompletionResult - Class in chat.octet.model.beans

Completion result entity

CompletionResult() - Constructor for class chat.octet.model.beans.CompletionResult

completions(GenerateParameter, String) - Method in class chat.octet.model.Model

Generate complete text.

completions(String) - Method in class chat.octet.model.Model

Generate complete text.

copyToStatus(Status) - Method in class chat.octet.model.beans.Status

createNewContextWithModel(LlamaContextParams) - Static method in class chat.octet.model.LlamaService

Create new context with model.

criteria(int[], float[], Object...) - Method in class chat.octet.model.components.criteria.impl.MaxTimeCriteria

criteria(int[], float[], Object...) - Method in interface chat.octet.model.components.criteria.StoppingCriteria

Stopping criteria

criteria(int[], float[], Object...) - Method in class chat.octet.model.components.criteria.StoppingCriteriaList

ctx - Variable in class chat.octet.model.beans.LlamaContextParams

text context size.

CustomBiasLogitsProcessor - Class in chat.octet.model.components.processor.impl

CustomBiasLogitsProcessor(Map<Integer, String>, int) - Constructor for class chat.octet.model.components.processor.impl.CustomBiasLogitsProcessor

D

DecodeException - Exception Class in chat.octet.model.exceptions

Batch decode exception

DecodeException(String) - Constructor for exception class chat.octet.model.exceptions.DecodeException

DecodeException(String, Throwable) - Constructor for exception class chat.octet.model.exceptions.DecodeException

decodeToken(int...) - Static method in class chat.octet.model.TokenDecoder

DEFAULT_ALPACA_SYSTEM - Static variable in class chat.octet.model.utils.PromptBuilder

DEFAULT_COMMON_SYSTEM - Static variable in class chat.octet.model.utils.PromptBuilder

DISABLED - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode

E

embedding - Variable in class chat.octet.model.beans.LlamaContextParams

embedding mode only.

equals(Object) - Method in class chat.octet.model.beans.Token

F

f16KV - Variable in class chat.octet.model.beans.LlamaContextParams

use fp16 for KV cache.

FALCON - Enum constant in enum class chat.octet.model.enums.ModelType

FINISHED - Enum constant in enum class chat.octet.model.enums.FinishReason

Completed generation.

FinishReason - Enum Class in chat.octet.model.enums

Token generate status

FREEBSD - Static variable in class chat.octet.model.utils.Platform

G

generate(GenerateParameter, String) - Method in class chat.octet.model.Model

Generate text in stream format.

generate(String) - Method in class chat.octet.model.Model

Generate text in stream format.

GenerateParameter - Class in chat.octet.model.parameters

Generate parameter

GenerateParameter() - Constructor for class chat.octet.model.parameters.GenerateParameter

GenerateParameter.MirostatMode - Enum Class in chat.octet.model.parameters

Mirostat sampling mode define

Generator - Class in chat.octet.model

Model inference generator, Supports streaming output text and generating complete text.

Generator(GenerateParameter, String) - Constructor for class chat.octet.model.Generator

Create inference generator.

Generator(GenerateParameter, String, Status) - Constructor for class chat.octet.model.Generator

Create inference generator.

getContextSize() - Static method in class chat.octet.model.LlamaService

Get model context size.

getEmbedding() - Static method in class chat.octet.model.LlamaService

Get embedding

getFinishReason() - Method in class chat.octet.model.beans.Status

getLlamaContextDefaultParams() - Static method in class chat.octet.model.LlamaService

Get llama context default params.

getLlamaModelDefaultParams() - Static method in class chat.octet.model.LlamaService

Get llama model default params.

getLlamaTokenType(int) - Static method in class chat.octet.model.LlamaService

Get token type define.

getLogits(int) - Static method in class chat.octet.model.LlamaService

Get Logits based on index, and the default index must be 0.

getLogitsIndex() - Method in class chat.octet.model.beans.Status

getOSType() - Static method in class chat.octet.model.utils.Platform

getSamplingMetrics(boolean) - Static method in class chat.octet.model.LlamaService

Get sampling metrics

getSystemInfo() - Static method in class chat.octet.model.LlamaService

Get system parameter information.

getTokenBOS() - Static method in class chat.octet.model.LlamaService

Get special BOS token.

getTokenEOS() - Static method in class chat.octet.model.LlamaService

Get special EOS token.

getTokenType(int) - Static method in class chat.octet.model.LlamaService

Get token type code.

getUtf8ByteLength(byte) - Static method in class chat.octet.model.TokenDecoder

getVocabSize() - Static method in class chat.octet.model.LlamaService

Get model vocab size.

GNU - Static variable in class chat.octet.model.utils.Platform

gpuLayers - Variable in class chat.octet.model.beans.LlamaModelParams

number of layers to store in VRAM.

H

hashCode() - Method in class chat.octet.model.beans.Token

I

initNative() - Static method in class chat.octet.model.LlamaService

initial JNI context.

isFinished() - Method in enum class chat.octet.model.enums.FinishReason

Check if the token has been completed else return false.

isLinux() - Static method in class chat.octet.model.utils.Platform

isMac() - Static method in class chat.octet.model.utils.Platform

isMlockSupported() - Static method in class chat.octet.model.LlamaService

Check whether MLOCK is supported.

isMmapSupported() - Static method in class chat.octet.model.LlamaService

Check whether MMAP is supported.

isMultiByte(int) - Static method in class chat.octet.model.TokenDecoder

isWindows() - Static method in class chat.octet.model.utils.Platform

iterator() - Method in class chat.octet.model.Generator

Return inference iterator.

K

KFREEBSD - Static variable in class chat.octet.model.utils.Platform

L

LENGTH - Enum constant in enum class chat.octet.model.enums.FinishReason

Generation has exceeded the maximum token limit and has been truncated.

LIB_RESOURCE_PATH - Static variable in class chat.octet.model.utils.Platform

LINUX - Static variable in class chat.octet.model.utils.Platform

LLAMA_TOKEN_TYPE_BYTE - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Byte type.

LLAMA_TOKEN_TYPE_CONTROL - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Control type.

LLAMA_TOKEN_TYPE_NORMAL - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Normal type.

LLAMA_TOKEN_TYPE_UNDEFINED - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Undefined type.

LLAMA_TOKEN_TYPE_UNKNOWN - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Unknown type.

LLAMA_TOKEN_TYPE_UNUSED - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

Unused type.

LLAMA_TOKEN_TYPE_USER_DEFINED - Enum constant in enum class chat.octet.model.enums.LlamaTokenType

User defined type.

LLAMA2 - Enum constant in enum class chat.octet.model.enums.ModelType

llamaBackendFree() - Static method in class chat.octet.model.LlamaService

Call once at the end of the program.

llamaBackendInit(boolean) - Static method in class chat.octet.model.LlamaService

Initialize the llama + ggml backend If numa is true, use NUMA optimizations Call once at the start of the program.

LlamaContextParams - Class in chat.octet.model.beans

Llama context params entity

LlamaContextParams() - Constructor for class chat.octet.model.beans.LlamaContextParams

LlamaModelParams - Class in chat.octet.model.beans

Llama model params entity

LlamaModelParams() - Constructor for class chat.octet.model.beans.LlamaModelParams

LlamaService - Class in chat.octet.model

Llama.cpp API

LlamaService() - Constructor for class chat.octet.model.LlamaService

LlamaTokenType - Enum Class in chat.octet.model.enums

Llama token type define

loadLibraryResource() - Static method in class chat.octet.model.utils.Platform

loadLlamaGrammar(String) - Static method in class chat.octet.model.LlamaService

Load llama grammar by rules.

loadLlamaModelFromFile(String, LlamaModelParams) - Static method in class chat.octet.model.LlamaService

Load Llama model from file.

loadLoraModelFromFile(String, float, String, int) - Static method in class chat.octet.model.LlamaService

Apply a LoRA adapter to a loaded model path_base_model is the path to a higher quality model to use as a base for the layers modified by the adapter.

logitsAll - Variable in class chat.octet.model.beans.LlamaContextParams

the llama_eval() call computes all logits, not just the last one.

LogitsProcessor - Interface in chat.octet.model.components.processor

Customize a processor to adjust the probability distribution of words and control the generation of model inference results.

LogitsProcessorList - Class in chat.octet.model.components.processor

Stopping criteria list

LogitsProcessorList() - Constructor for class chat.octet.model.components.processor.LogitsProcessorList

LogitsProcessorList(Collection<? extends LogitsProcessor>) - Constructor for class chat.octet.model.components.processor.LogitsProcessorList

M

MAC - Static variable in class chat.octet.model.utils.Platform

mainGpu - Variable in class chat.octet.model.beans.LlamaModelParams

the GPU that is used for scratch and small tensors.

MaxTimeCriteria - Class in chat.octet.model.components.criteria.impl

MaxTimeCriteria(long) - Constructor for class chat.octet.model.components.criteria.impl.MaxTimeCriteria

MaxTimeCriteria(long, long) - Constructor for class chat.octet.model.components.criteria.impl.MaxTimeCriteria

metrics() - Method in class chat.octet.model.Model

Print generation metrics.

Metrics - Class in chat.octet.model.beans

Generation metrics

Metrics() - Constructor for class chat.octet.model.beans.Metrics

METRICS_TEMPLATE - Static variable in class chat.octet.model.beans.Metrics

mlock - Variable in class chat.octet.model.beans.LlamaModelParams

force system to keep model in RAM.

mmap - Variable in class chat.octet.model.beans.LlamaModelParams

use mmap if possible.

Model - Class in chat.octet.model

LLama model, which provides functions for generating and chatting conversations.

Model(ModelParameter) - Constructor for class chat.octet.model.Model

Model(String) - Constructor for class chat.octet.model.Model

ModelException - Exception Class in chat.octet.model.exceptions

Model exception

ModelException(String) - Constructor for exception class chat.octet.model.exceptions.ModelException

ModelException(String, Throwable) - Constructor for exception class chat.octet.model.exceptions.ModelException

ModelParameter - Class in chat.octet.model.parameters

Llama model parameters

ModelParameter() - Constructor for class chat.octet.model.parameters.ModelParameter

ModelType - Enum Class in chat.octet.model.enums

Model type

mulMatQ - Variable in class chat.octet.model.beans.LlamaContextParams

if true, use experimental mul_mat_q kernels.

N

NETBSD - Static variable in class chat.octet.model.utils.Platform

NoBadWordsLogitsProcessor - Class in chat.octet.model.components.processor.impl

NoBadWordsLogitsProcessor(int[]) - Constructor for class chat.octet.model.components.processor.impl.NoBadWordsLogitsProcessor

NONE - Enum constant in enum class chat.octet.model.enums.FinishReason

Default type.

O

OASST_LLAMA - Enum constant in enum class chat.octet.model.enums.ModelType

OPEN_BUDDY - Enum constant in enum class chat.octet.model.enums.ModelType

OPENBSD - Static variable in class chat.octet.model.utils.Platform

output() - Method in class chat.octet.model.Generator

Stream outputs the generated text.

P

Platform - Class in chat.octet.model.utils

processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.impl.CustomBiasLogitsProcessor

processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.impl.NoBadWordsLogitsProcessor

processor(int[], float[], Object...) - Method in interface chat.octet.model.components.processor.LogitsProcessor

Logits processor

processor(int[], float[], Object...) - Method in class chat.octet.model.components.processor.LogitsProcessorList

PromptBuilder - Class in chat.octet.model.utils

Prompt builder

PromptBuilder() - Constructor for class chat.octet.model.utils.PromptBuilder

R

REDPAJAMA_INCITE - Enum constant in enum class chat.octet.model.enums.ModelType

release() - Static method in class chat.octet.model.LlamaService

Close model and release all resources.

removeAllChatStatus() - Method in class chat.octet.model.Model

Delete all user session states.

removeChatStatus(String) - Method in class chat.octet.model.Model

Delete the session state of the specified user.

reset() - Method in class chat.octet.model.beans.Status

result() - Method in class chat.octet.model.Generator

Return the generated complete text.

ropeFreqBase - Variable in class chat.octet.model.beans.LlamaContextParams

RoPE base frequency.

ropeFreqScale - Variable in class chat.octet.model.beans.LlamaContextParams

RoPE frequency scaling factor.

S

sampling(float[], int[], int, float, float, float, boolean, int, float, float, float, int, float, float, float, int, int) - Static method in class chat.octet.model.LlamaService

Inference sampling the next token.

seed - Variable in class chat.octet.model.beans.LlamaContextParams

RNG seed, -1 for random.

SNOOZY - Enum constant in enum class chat.octet.model.enums.ModelType

SOLARIS - Static variable in class chat.octet.model.utils.Platform

spliterator() - Method in class chat.octet.model.Generator

Status - Class in chat.octet.model.beans

Status() - Constructor for class chat.octet.model.beans.Status

Status(Status) - Constructor for class chat.octet.model.beans.Status

STOP - Enum constant in enum class chat.octet.model.enums.FinishReason

Generation stopped by StoppingCriteria.

StoppingCriteria - Interface in chat.octet.model.components.criteria

Customize a controller to implement stop rule control for model inference.

StoppingCriteriaList - Class in chat.octet.model.components.criteria

Stopping criteria list

StoppingCriteriaList() - Constructor for class chat.octet.model.components.criteria.StoppingCriteriaList

StoppingCriteriaList(Collection<? extends StoppingCriteria>) - Constructor for class chat.octet.model.components.criteria.StoppingCriteriaList

subInputIds(int) - Method in class chat.octet.model.beans.Status

subInputIds(int, int) - Method in class chat.octet.model.beans.Status

SYSTEM - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole

System prompt

T

tensorSplit - Variable in class chat.octet.model.beans.LlamaModelParams

how to split layers across multiple GPUs (size: LLAMA_MAX_DEVICES).

threads - Variable in class chat.octet.model.beans.LlamaContextParams

number of threads used for generation.

threadsBatch - Variable in class chat.octet.model.beans.LlamaContextParams

number of threads used for prompt and batch processing.

toAssistant(String) - Static method in class chat.octet.model.beans.ChatMessage

Token - Class in chat.octet.model.beans

Token

Token(int, LlamaTokenType, String) - Constructor for class chat.octet.model.beans.Token

TokenDecoder - Class in chat.octet.model

Token decoder

tokenize(byte[], int, int[], int, boolean, boolean) - Static method in class chat.octet.model.LlamaService

Convert the provided text into tokens.

tokenize(String, boolean, boolean) - Static method in class chat.octet.model.LlamaService

Convert the provided text into tokens.

tokenToPiece(int, byte[], int) - Static method in class chat.octet.model.LlamaService

Convert the token id to text piece.

toPrompt(ModelType, String) - Static method in class chat.octet.model.utils.PromptBuilder

Create prompt text

toPrompt(ModelType, String, String) - Static method in class chat.octet.model.utils.PromptBuilder

Create prompt text

toString() - Method in class chat.octet.model.beans.Metrics

toString() - Method in enum class chat.octet.model.enums.LlamaTokenType

toString() - Method in class chat.octet.model.Model

toSystem(String) - Static method in class chat.octet.model.beans.ChatMessage

toUser(String) - Static method in class chat.octet.model.beans.ChatMessage

TRUNCATED - Enum constant in enum class chat.octet.model.enums.FinishReason

Generation has exceeded the maximum context limit and has been truncated.

U

UNKNOWN - Enum constant in enum class chat.octet.model.enums.FinishReason

Unknown type, no available token state.

UNSPECIFIED - Static variable in class chat.octet.model.utils.Platform

updateFinishReason(FinishReason) - Method in class chat.octet.model.beans.Token

USER - Enum constant in enum class chat.octet.model.beans.ChatMessage.ChatRole

User role

V

V1 - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode

V2 - Enum constant in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode

valueOf(String) - Static method in enum class chat.octet.model.beans.ChatMessage.ChatRole

Returns the enum constant of this class with the specified name.

valueOf(String) - Static method in enum class chat.octet.model.enums.FinishReason

Returns the enum constant of this class with the specified name.

valueOf(String) - Static method in enum class chat.octet.model.enums.LlamaTokenType

Returns the enum constant of this class with the specified name.

valueOf(String) - Static method in enum class chat.octet.model.enums.ModelType

Returns the enum constant of this class with the specified name.

valueOf(String) - Static method in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode

Returns the enum constant of this class with the specified name.

valueOfType(int) - Static method in enum class chat.octet.model.enums.LlamaTokenType

values() - Static method in enum class chat.octet.model.beans.ChatMessage.ChatRole

Returns an array containing the constants of this enum class, in the order they are declared.

values() - Static method in enum class chat.octet.model.enums.FinishReason

Returns an array containing the constants of this enum class, in the order they are declared.

values() - Static method in enum class chat.octet.model.enums.LlamaTokenType

Returns an array containing the constants of this enum class, in the order they are declared.

values() - Static method in enum class chat.octet.model.enums.ModelType

Returns an array containing the constants of this enum class, in the order they are declared.

values() - Static method in enum class chat.octet.model.parameters.GenerateParameter.MirostatMode

Returns an array containing the constants of this enum class, in the order they are declared.

VICUNA - Enum constant in enum class chat.octet.model.enums.ModelType

vocabOnly - Variable in class chat.octet.model.beans.LlamaModelParams

only load the vocabulary, no weights.

W

WINDOWS - Static variable in class chat.octet.model.utils.Platform

WINDOWSCE - Static variable in class chat.octet.model.utils.Platform

A B C D E F G H I K L M N O P R S T U V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form