Package chat.octet.model

Class Model

java.lang.Object

chat.octet.model.Model

All Implemented Interfaces:

AutoCloseable

public class Model
extends Object
implements AutoCloseable

LLama model, which provides functions for generating and chatting conversations.

Author:

William

Constructor Summary

Constructors

Constructor	Description
Model(ModelParameter modelParams)
Model(String modelPath)

Method Summary

Modifier and Type	Method	Description
Generator	chat(GenerateParameter generateParams, String question)	Start a conversation and chat in streaming format.
Generator	chat(GenerateParameter generateParams, String system, String question)	Start a conversation and chat in streaming format.
Generator	chat(String question)	Start a conversation and chat in streaming format.
Generator	chat(String system, String question)	Start a conversation and chat in streaming format.
CompletionResult	chatCompletions(GenerateParameter generateParams, String question)	Start a conversation and chat.
CompletionResult	chatCompletions(GenerateParameter generateParams, String system, String question)	Start a conversation and chat.
CompletionResult	chatCompletions(String question)	Start a conversation and chat.
void	close()	Close the model and release resources.
CompletionResult	completions(GenerateParameter generateParams, String text)	Generate complete text.
CompletionResult	completions(String text)	Generate complete text.
Generator	generate(GenerateParameter generateParams, String text)	Generate text in stream format.
Generator	generate(String text)	Generate text in stream format.
void	metrics()	Print generation metrics.
void	removeAllChatStatus()	Delete all user session states.
void	removeChatStatus(String session)	Delete the session state of the specified user.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

Model

public Model(String modelPath)

Model

public Model(ModelParameter modelParams)

Method Details

removeChatStatus

public void removeChatStatus(String session)

Delete the session state of the specified user.

Parameters:

session - User session key.

removeAllChatStatus

public void removeAllChatStatus()

Delete all user session states.

completions

public CompletionResult completions(String text)

Generate complete text.

Parameters:

text - Input text or prompt.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

completions

public CompletionResult completions(GenerateParameter generateParams,
 String text)

Generate complete text.

Parameters:

generateParams - Specify a generation parameter.

text - Input text or prompt.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

generate

public Generator generate(String text)

Generate text in stream format.

Parameters:

text - Input text or prompt.

Returns:

Inference generator.

See Also:

• Generator

generate

public Generator generate(GenerateParameter generateParams,
 String text)

Generate text in stream format.

Parameters:

generateParams - Specify a generation parameter.

text - Input text or prompt.

Returns:

Inference generator.

See Also:

• Generator

chatCompletions

public CompletionResult chatCompletions(String question)

Start a conversation and chat.

Parameters:

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chatCompletions

public CompletionResult chatCompletions(GenerateParameter generateParams,
 String question)

Start a conversation and chat.

Parameters:

generateParams - Specify a generation parameter.

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chatCompletions

public CompletionResult chatCompletions(GenerateParameter generateParams,
 String system,
 String question)

Start a conversation and chat.

Parameters:

generateParams - Specify a generation parameter.

system - System prompt.

question - User question.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

chat

public Generator chat(String question)

Start a conversation and chat in streaming format.

Parameters:

question - User question.

Returns:

Inference generator.

See Also:

• Generator

chat

public Generator chat(String system,
 String question)

Start a conversation and chat in streaming format.

Parameters:

system - System prompt.

question - User question.

Returns:

Inference generator.

See Also:

• Generator

chat

public Generator chat(GenerateParameter generateParams,
 String question)

Start a conversation and chat in streaming format.

Parameters:

generateParams - Specify a generation parameter.

question - User question.

Returns:

Inference generator.

See Also:

• Generator

chat

public Generator chat(GenerateParameter generateParams,
 String system,
 String question)

Start a conversation and chat in streaming format.

Parameters:

generateParams - Specify a generation parameter.

system - System prompt.

question - User question.

Returns:

Inference generator.

See Also:

• Generator

metrics

public void metrics()

Print generation metrics.

Require verbose parameter to be true.

close

public void close()

Close the model and release resources.

Specified by:

close in interface AutoCloseable

toString

public String toString()

Overrides:

toString in class Object