public final class ImageSegmenter extends BaseVisionTaskApi

Performs segmentation on images.

The API expects a TFLite model with TFLite Model Metadata..

The API supports models with one image input tensor and one output tensor. To be more specific, here are the requirements.

An example of such model can be found on TensorFlow Hub..

Nested Class Summary

class ImageSegmenter.ImageSegmenterOptions Options for setting up an ImageSegmenter

Public Method Summary

static ImageSegmenter
createFromBuffer(ByteBuffer modelBuffer)
Creates an ImageSegmenter instance with a model buffer and the default ImageSegmenter.ImageSegmenterOptions.
static ImageSegmenter
static ImageSegmenter
createFromFile(File modelFile)
Creates an ImageSegmenter instance from the default ImageSegmenter.ImageSegmenterOptions.
static ImageSegmenter
createFromFile(Context context, String modelPath)
Creates an ImageSegmenter instance from the default ImageSegmenter.ImageSegmenterOptions.
static ImageSegmenter
static ImageSegmenter
List<Segmentation>
segment(long frameBufferHandle, ImageProcessingOptions options)
List<Segmentation>
segment(TensorImage image)
Performs actual segmentation on the provided image.
List<Segmentation>
segment(MlImage image)
Performs actual segmentation on the provided MlImage.
List<Segmentation>
segment(TensorImage image, ImageProcessingOptions options)
Performs actual segmentation on the provided image with ImageProcessingOptions.
List<Segmentation>
segment(MlImage image, ImageProcessingOptions options)
Performs actual segmentation on the provided MlImage with ImageProcessingOptions.

Protected Method Summary

void
deinit(long nativeHandle)

Inherited Method Summary

Public Methods

public static ImageSegmenter createFromBuffer (ByteBuffer modelBuffer)

Creates an ImageSegmenter instance with a model buffer and the default ImageSegmenter.ImageSegmenterOptions.

Parameters
modelBuffer a direct ByteBuffer or a MappedByteBuffer of the segmentation model
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the model buffer is not a direct ByteBuffer or a MappedByteBuffer

public static ImageSegmenter createFromBufferAndOptions (ByteBuffer modelBuffer, ImageSegmenter.ImageSegmenterOptions options)

Creates an ImageSegmenter instance with a model buffer and ImageSegmenter.ImageSegmenterOptions.

Parameters
modelBuffer a direct ByteBuffer or a MappedByteBuffer of the segmentation model
options
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the model buffer is not a direct ByteBuffer or a MappedByteBuffer

public static ImageSegmenter createFromFile (File modelFile)

Creates an ImageSegmenter instance from the default ImageSegmenter.ImageSegmenterOptions.

Parameters
modelFile the segmentation model File instance
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ImageSegmenter createFromFile (Context context, String modelPath)

Creates an ImageSegmenter instance from the default ImageSegmenter.ImageSegmenterOptions.

Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ImageSegmenter createFromFileAndOptions (File modelFile, ImageSegmenter.ImageSegmenterOptions options)

Parameters
modelFile the segmentation model File instance
options
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ImageSegmenter createFromFileAndOptions (Context context, String modelPath, ImageSegmenter.ImageSegmenterOptions options)

Parameters
context
modelPath path of the segmentation model with metadata in the assets
options
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public List<Segmentation> segment (long frameBufferHandle, ImageProcessingOptions options)

public List<Segmentation> segment (TensorImage image)

Performs actual segmentation on the provided image.

ImageSegmenter supports the following TensorImage color space types:

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
Returns
  • results of performing image segmentation. Note that at the time, a single Segmentation element is expected to be returned. The result is stored in a List for later extension to e.g. instance segmentation models, which may return one segmentation per object.
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the color space type of image is unsupported

public List<Segmentation> segment (MlImage image)

Performs actual segmentation on the provided MlImage.

Parameters
image an MlImage to segment.
Returns
  • results of performing image segmentation. Note that at the time, a single Segmentation element is expected to be returned. The result is stored in a List for later extension to e.g. instance segmentation models, which may return one segmentation per object.
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the storage type or format of the image is unsupported

public List<Segmentation> segment (TensorImage image, ImageProcessingOptions options)

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
options the options configure how to preprocess the image
Returns
  • results of performing image segmentation. Note that at the time, a single Segmentation element is expected to be returned. The result is stored in a List for later extension to e.g. instance segmentation models, which may return one segmentation per object.
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the color space type of image is unsupported

public List<Segmentation> segment (MlImage image, ImageProcessingOptions options)

Performs actual segmentation on the provided MlImage with ImageProcessingOptions.

ImageSegmenter supports the following options:

Parameters
image an MlImage to segment.
options the options configure how to preprocess the image.
Returns
  • results of performing image segmentation. Note that at the time, a single Segmentation element is expected to be returned. The result is stored in a List for later extension to e.g. instance segmentation models, which may return one segmentation per object.
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the color space type of image is unsupported

Protected Methods

protected void deinit (long nativeHandle)