Package chat.octet.model

Class Model

java.lang.Object

chat.octet.model.Model

All Implemented Interfaces:

AutoCloseable

public class Model
extends Object
implements AutoCloseable

LLama model, which provides functions for generating and chatting conversations.

Author:

William

Constructor Summary

Constructors

Constructor	Description
Model(ModelParameter modelParams)
Model(String modelPath)

Method Summary

Modifier and Type	Method	Description
Iterable<Token>	chat(GenerateParameter generateParams, String question)	Start a conversation and chat in streaming format.
Iterable<Token>	chat(GenerateParameter generateParams, String system, String question)	Start a conversation and chat in streaming format.
Iterable<Token>	chat(String question)	Start a conversation and chat in streaming format.
Iterable<Token>	chat(String system, String question)	Start a conversation and chat in streaming format.
CompletionResult	chatCompletions(GenerateParameter generateParams, String question)	Start a conversation and chat.
CompletionResult	chatCompletions(GenerateParameter generateParams, String system, String question)	Start a conversation and chat.
CompletionResult	chatCompletions(String question)	Start a conversation and chat.
void	close()	Close the model and release resources.
CompletionResult	completions(GenerateParameter generateParams, String text)	Generate complete text.
CompletionResult	completions(String text)	Generate complete text.
Iterable<Token>	generate(GenerateParameter generateParams, String text)	Generate text in stream format.
Iterable<Token>	generate(String text)	Generate text in stream format.
void	metrics()	Print generation metrics.
void	removeAllChatStatus()
void	removeChatStatus(String user)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

Model

public Model(String modelPath)

Model

public Model(ModelParameter modelParams)

Method Details

removeChatStatus

public void removeChatStatus(String user)

removeAllChatStatus

public void removeAllChatStatus()

completions

public CompletionResult completions(String text)

Generate complete text.

Parameters:

text - Input text or prompt.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

completions

public CompletionResult completions(GenerateParameter generateParams,
 String text)

Generate complete text.

Parameters:

generateParams - Specify a generation parameter.

text - Input text or prompt.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

generate

public Iterable<Token> generate(String text)

Generate text in stream format.

Parameters:

text - Input text or prompt.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

generate

public Iterable<Token> generate(GenerateParameter generateParams,
 String text)

Generate text in stream format.

Parameters:

generateParams - Specify a generation parameter.

text - Input text or prompt.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

chatCompletions

public CompletionResult chatCompletions(String question)

Start a conversation and chat.

Parameters:

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chatCompletions

public CompletionResult chatCompletions(GenerateParameter generateParams,
 String question)

Start a conversation and chat.

Parameters:

generateParams - Specify a generation parameter.

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chatCompletions

public CompletionResult chatCompletions(GenerateParameter generateParams,
 String system,
 String question)

Start a conversation and chat.

Parameters:

generateParams - Specify a generation parameter.

system - System prompt.

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chat

public Iterable<Token> chat(String question)

Start a conversation and chat in streaming format.

Parameters:

question - User question.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

chat

public Iterable<Token> chat(String system,
 String question)

Start a conversation and chat in streaming format.

Parameters:

system - System prompt.

question - User question.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

chat

public Iterable<Token> chat(GenerateParameter generateParams,
 String question)

Start a conversation and chat in streaming format.

Parameters:

generateParams - Specify a generation parameter.

question - User question.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

chat

public Iterable<Token> chat(GenerateParameter generateParams,
 String system,
 String question)

Start a conversation and chat in streaming format.

Parameters:

generateParams - Specify a generation parameter.

system - System prompt.

question - User question.

Returns:

Iterable, Generation iterator.

See Also:

• Generator

metrics

public void metrics()

Print generation metrics.

Require verbose parameter to be true.

close

public void close()

Close the model and release resources.

Specified by:

close in interface AutoCloseable

toString

public String toString()

Overrides:

toString in class Object