Performs segmentation on images.
The API expects a TFLite model with TFLite Model Metadata..
The API supports models with one image input tensor and one output tensor. To be more specific, here are the requirements.
kTfLiteUInt8
/kTfLiteFloat32
)
[batch x height x width x channels]
.batch
is required to be 1).channels
is required to be 3).kTfLiteFloat32
, NormalizationOptions are required to be
attached to the metadata for input normalization.kTfLiteUInt8
/kTfLiteFloat32
)
[batch x mask_height x mask_width x num_classes]
, where
batch
is required to be 1, mask_width
and
mask_height
are the dimensions of the segmentation masks produced by the
model, and num_classes
is the number of classes supported by the
model.
ColoredLabel.getlabel()
of the results. The display name, i.e.
ColoredLabel.getDisplayName()
, is filled from the AssociatedFile (if any)
whose locale matches the `display_names_locale` field of the `ImageSegmenterOptions`
used at creation time ("en" by default, i.e. English). If none of these are available,
only the `index` field of the results will be filled.An example of such model can be found on TensorFlow Hub..
class | ImageSegmenter.ImageSegmenterOptions | Options for setting up an
ImageSegmenter . |
static ImageSegmenter |
createFromBuffer(ByteBuffer
modelBuffer)
Creates an
ImageSegmenter instance with a model buffer and the default
ImageSegmenter.ImageSegmenterOptions .
|
static ImageSegmenter |
createFromBufferAndOptions(ByteBuffer
modelBuffer,
ImageSegmenter.ImageSegmenterOptions options)
Creates an
ImageSegmenter instance with a model buffer and
ImageSegmenter.ImageSegmenterOptions .
|
static ImageSegmenter |
createFromFile(File modelFile)
Creates an
ImageSegmenter instance from the default
ImageSegmenter.ImageSegmenterOptions .
|
static ImageSegmenter |
createFromFile(Context
context, String modelPath)
Creates an
ImageSegmenter instance from the default
ImageSegmenter.ImageSegmenterOptions .
|
static ImageSegmenter |
createFromFileAndOptions(File modelFile,
ImageSegmenter.ImageSegmenterOptions options)
Creates an
ImageSegmenter instance from
ImageSegmenter.ImageSegmenterOptions .
|
static ImageSegmenter |
createFromFileAndOptions(Context
context, String modelPath,
ImageSegmenter.ImageSegmenterOptions options)
Creates an
ImageSegmenter instance from
ImageSegmenter.ImageSegmenterOptions .
|
List<Segmentation> |
segment(long frameBufferHandle, ImageProcessingOptions options)
|
List<Segmentation> |
segment(TensorImage image)
Performs actual segmentation on the provided image.
|
List<Segmentation> |
segment(MlImage image)
Performs actual segmentation on the provided
MlImage .
|
List<Segmentation> |
segment(TensorImage image, ImageProcessingOptions options)
Performs actual segmentation on the provided image with
ImageProcessingOptions .
|
List<Segmentation> |
segment(MlImage image, ImageProcessingOptions options)
Performs actual segmentation on the provided
MlImage with
ImageProcessingOptions .
|
void |
deinit(long nativeHandle)
|
Creates an ImageSegmenter
instance with a model buffer and the default
ImageSegmenter.ImageSegmenterOptions
.
modelBuffer | a direct ByteBuffer
or a MappedByteBuffer
of the segmentation model |
---|
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the model buffer is not a direct ByteBuffer
or a MappedByteBuffer |
Creates an ImageSegmenter
instance with a model buffer and
ImageSegmenter.ImageSegmenterOptions
.
modelBuffer | a direct ByteBuffer
or a MappedByteBuffer
of the segmentation model |
---|---|
options |
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the model buffer is not a direct ByteBuffer
or a MappedByteBuffer |
Creates an ImageSegmenter
instance from the default
ImageSegmenter.ImageSegmenterOptions
.
modelFile | the segmentation model File instance |
---|
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
Creates an ImageSegmenter
instance from the default
ImageSegmenter.ImageSegmenterOptions
.
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
Creates an ImageSegmenter
instance from
ImageSegmenter.ImageSegmenterOptions
.
modelFile | the segmentation model File instance |
---|---|
options |
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
Creates an ImageSegmenter
instance from
ImageSegmenter.ImageSegmenterOptions
.
context | |
---|---|
modelPath | path of the segmentation model with metadata in the assets |
options |
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
Performs actual segmentation on the provided image.
ImageSegmenter
supports the following TensorImage
color space types:
image | a UINT8 TensorImage
object that represents an RGB or YUV image |
---|
Segmentation
element is expected to be returned. The result is stored in a List
for later
extension to e.g. instance segmentation models, which may return one segmentation per
object.IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the color space type of image is unsupported |
Performs actual segmentation on the provided MlImage
.
image | an MlImage to segment. |
---|
Segmentation
element is expected to be returned. The result is stored in a List
for later
extension to e.g. instance segmentation models, which may return one segmentation per
object.IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the storage type or format of the image is unsupported |
Performs actual segmentation on the provided image with ImageProcessingOptions
.
ImageSegmenter
supports the following TensorImage
color space types:
ImageSegmenter
supports the following options:
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
).
It defaults to
ImageProcessingOptions.Orientation.TOP_LEFT
image | a UINT8 TensorImage
object that represents an RGB or YUV image |
---|---|
options | the options configure how to preprocess the image |
Segmentation
element is expected to be returned. The result is stored in a List
for later
extension to e.g. instance segmentation models, which may return one segmentation per
object.IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the color space type of image is unsupported |
Performs actual segmentation on the provided MlImage
with
ImageProcessingOptions
.
ImageSegmenter
supports the following options:
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
).
It defaults to
ImageProcessingOptions.Orientation.TOP_LEFT
. MlImage.getRotation()
is not effective.image | an MlImage to segment. |
---|---|
options | the options configure how to preprocess the image. |
Segmentation
element is expected to be returned. The result is stored in a List
for later
extension to e.g. instance segmentation models, which may return one segmentation per
object.IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the color space type of image is unsupported |