Package chat.octet.model

Class Generator

java.lang.Object

chat.octet.model.Generator

All Implemented Interfaces:

Iterable<Token>

public class Generator
extends Object
implements Iterable<Token>

Model inference generator, Supports streaming output text and generating complete text.

Author:

William

Constructor Summary

Constructors

Constructor	Description
Generator(GenerateParameter generateParams, String prompt)	Create inference generator.
Generator(GenerateParameter generateParams, String prompt, Status chatStatus)	Create inference generator.

Method Summary

Modifier and Type	Method	Description
void	close()	Close inference generator.
Iterator<Token>	iterator()	Return inference iterator.
void	output()	Stream outputs the generated text.
CompletionResult	result()	Return the generated complete text.
List<Token>	tokens()	Return the generated tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Details

Generator

public Generator(GenerateParameter generateParams,
 String prompt,
 Status chatStatus)

Create inference generator.

Parameters:

generateParams - Specify a generation parameter.

prompt - Prompt text.

chatStatus - Source status.

Generator

public Generator(GenerateParameter generateParams,
 String prompt)

Create inference generator.

Parameters:

generateParams - Specify a generation parameter.

prompt - Prompt text.

Method Details

iterator

@Nonnull
public Iterator<Token> iterator()

Return inference iterator.

Specified by:

iterator in interface Iterable<Token>

Returns:

Iterator

output

public void output()

Stream outputs the generated text.

result

public CompletionResult result()

Return the generated complete text.

Returns:

CompletionResult, generated text and completion reason.

See Also:

• CompletionResult

tokens

public List<Token> tokens()

Return the generated tokens.

Returns:

A list of tokens will be returned.

close

public void close()

Close inference generator.