Package io.mishmash.opentelemetry.server.collector

Class AbstractCollector<REQ extends com.google.protobuf.AbstractMessage,RESP extends com.google.protobuf.AbstractMessage,RECORD>

java.lang.Object
java.util.concurrent.SubmissionPublisher<RECORD>
io.mishmash.opentelemetry.server.collector.AbstractCollector<REQ,RESP,RECORD>
Type Parameters:
REQ - the type of OTLP request being handled by this collector
RESP - the type of OTLP response
RECORD - the type of data items a request is broken into
All Implemented Interfaces:
AutoCloseable, Flow.Publisher<RECORD>
Direct Known Subclasses:
LogsCollector, MetricsCollector, TracesCollector
public abstract class AbstractCollector<REQ extends com.google.protobuf.AbstractMessage,RESP extends com.google.protobuf.AbstractMessage,RECORD> extends SubmissionPublisher<RECORD>

This class handles the OTLP protocol for gRPC and HTTP transports for a collector of a given signal type (like logs, metrics or traces).

As each collector attaches to own gRPC methods and own HTTP endpoints, the common functionality of receiving, parsing, 'queuing' signals for subscribing classes to process and responding back to clients is kept here.

This class provides methods to bind to Vert.x gRPC and HTTP servers and to process the OTLP requests and responses, leaving only the task of breaking up the data into individual items to be submitted to the subscribers. See LogsCollector, MetricsCollector and TracesCollector.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    DEFAULT_CLOSE_TIMETOUT_SEC
    Default timeout (in sec) to complete a call to close() this collector.
    static final int
    DEFAULT_MAX_BODY_LEN
    Default maximum HTTP request body size in bytes.
    static final int
    DEFAULT_SUBSCRIBER_QUEUE_SIZE
    Default size of a subscriber queue.
    static final int
    DEFAULT_TIMEOUT_SEC
    Default timeout (in sec) to process a single OTLP packet.
    protected static final int
    HTTP_BAD_GATEWAY
    HTTP status code 502.
    protected static final int
    HTTP_BAD_REQUEST
    HTTP status code 400.
    protected static final int
    HTTP_EXPECTATION_FAILED
    HTTP status code 417.
    protected static final int
    HTTP_GATEWAY_TIMEOUT
    HTTP status code 504.
    protected static final int
    HTTP_INTERNAL_ERROR
    HTTP status code 500.
    static final int
    HTTP_OK
    HTTP status code 200.
    protected static final int
    HTTP_PAYLOAD_TOO_LARGE
    HTTP status code 413.
    protected static final int
    HTTP_SERVICE_UNAVAILABLE
    HTTP status code 503.
    protected static final int
    HTTP_TOO_MANY_REQUESTS
    HTTP status code 429.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AbstractCollector(String httpUrlPath, io.grpc.MethodDescriptor<REQ,RESP> exportMethod, Instrumentation otelHelper, Supplier<com.google.protobuf.Parser<REQ>> requestParser, Supplier<com.google.protobuf.Message.Builder> requestBuilder, Supplier<com.google.protobuf.Parser<RESP>> responseParser, Supplier<com.google.protobuf.Message.Builder> responseBuilder, ForkJoinPool threadPool)
    Create a new collector.

  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    addDroppedRequestItems(long count, String transport, String encoding)
    Update the counter of how many OTLP items (logs, spans, metrics) have been dropped so far.
    protected void
    addFailedRequests(long count, String transport, String encoding, boolean retryable)
    Updates the number of failed requests counter.
    protected void
    addGrpcFailedRequests(long count, io.vertx.grpc.common.GrpcStatus status)
    Update the number of failed requests counter for a given gRPC error code.
    protected void
    addHttpFailedRequests(long count, int httpResponseStatusCode, String encoding)
    Update the number of failed requests counter for a given HTTP error code.
    protected void
    addPartiallySucceededRequests(long count, String transport, String encoding)
    Updates the number of partially successful requests counter.
    protected void
    addRequestItems(long count, String transport, String encoding)
    Update the counter of how many OTLP items (logs, spans, metrics) have been processed so far.
    protected void
    addRequestsInProcess(long count, String transport, String encoding)
    Update the counter of number of requests currently being processed.
    protected void
    addSucceededRequests(long count, String transport, String encoding)
    Updates the number of successful requests counter.
    void
    bind(io.vertx.ext.web.Router router)
    Bind this collector to the given HTTP Router and start accepting requests.
    void
    bind(io.vertx.grpc.server.GrpcServer server)
    Binds this collector to the gRPC server.
    protected void
    checkBody(io.vertx.ext.web.RoutingContext context)
    Checks the body of an incoming HTTP request for validity.
    protected void
    checkContentType(io.vertx.ext.web.RoutingContext ctx)
    Checks the Content-Type header of an incoming HTTP request for validity.
    protected void
    checkSubscribers(io.vertx.ext.web.RoutingContext ctx)
    Fail an incoming request if we have no subscribers.
    void
    close()
    void
    closeExceptionally(Throwable error)
    protected abstract RESP
    getBatchResponse(REQ request, Batch<RECORD> completedBatch, String transport, String encoding)
    Called to compute the correct response to the client after an entire batch (or packet) of OpenTelemetry logs, metrics or spans has been processed.
    protected int
    getErrorCode(io.vertx.ext.web.RoutingContext ctx)
    Returns the previously set error code for the given HTTP request.
    protected Instrumentation
    getInstrumentation()
    Get the OpenTelemetry instrumentation helper.
    protected io.opentelemetry.context.Context
    getOtelContext(io.vertx.ext.web.RoutingContext ctx)
    Get the telemetry context previously associated with this HTTP routing context.
    protected REQ
    getOtelRequest(io.vertx.ext.web.RoutingContext ctx)
    Get the OTLP request object from a HTTP request.
    protected RESP
    getOtelResponse(io.vertx.ext.web.RoutingContext ctx)
    Get the OTLP response object from the routing context.
    protected void
    handle(io.vertx.grpc.server.GrpcServerRequest<REQ,RESP> request)
    Handles an incoming gRPC request.
    protected String
    httpEncoding(io.vertx.ext.web.RoutingContext ctx)
    Gets the encoding of a HTTP OTLP request.
    protected void
    initInstrumentation()
    Create own instrumentation metrics.
    protected boolean
    isError(io.vertx.ext.web.RoutingContext ctx)
    Checks if the processing of a HTTP request encountered an error.
    protected boolean
    isJsonContent(io.vertx.core.http.HttpServerRequest request)
    Check if the HTTP request is JSON formatted.
    protected boolean
    isJsonRequest(io.vertx.ext.web.RoutingContext ctx)
    Check if the HTTP routing context was previously marked as for a JSON-encoded OTLP request.
    protected boolean
    isProtobufContent(io.vertx.core.http.HttpServerRequest request)
    Check if the HTTP request is protobuf formatted.
    protected boolean
    isRetryable(int httpResponseStatusCode)
    Check if a given HTTP error can be retried by the client.
    protected boolean
    isRetryable(io.vertx.grpc.common.GrpcStatus status)
    Check if a given gRPC error can be retried by the client.
    protected abstract Batch<RECORD>
    loadBatch(REQ request, String transport, String encoding, io.opentelemetry.context.Context otelContext)
    Creates a new batch from a client's request.
    protected long
    parseContentLengthHeader(io.vertx.core.http.HttpServerRequest request)
    Parses the Content-Length header of an incoming HTTP request.
    protected REQ
    parseHttpJson(InputStream is)
    Parses an incoming HTTP json-encoded request.
    protected REQ
    parseHttpProtobuf(InputStream is)
    Parses an incoming HTTP protobuf-encoded request.
    protected void
    parseHttpRequest(io.vertx.ext.web.RoutingContext ctx)
    Parse an incoming HTTP request.
    protected void
    publish(io.vertx.ext.web.RoutingContext ctx, REQ request)
    Publish an incoming HTTP request to subscribers.
    protected Batch<RECORD>
    publish(REQ request, String transport, String encoding, io.opentelemetry.context.Context otelContext)
    Publishes an OTLP packet (request) to all current subscribers.
    protected void
    respond(io.vertx.ext.web.RoutingContext ctx)
    Send the HTTP response back to the client.
    protected void
    respond(io.vertx.grpc.server.GrpcServerRequest<REQ,RESP> grpcRequest, REQ request, Batch<RECORD> batch, Throwable err)
    Responds to a processed gRPC OTLP request.
    protected String
    responseToJson(RESP response)
    Converts a response object to JSON before sending back to client.
    protected byte[]
    responseToProtobuf(RESP response)
    Converts a response object to protobuf before sending back to client.
    protected void
    setErrorCode(io.vertx.ext.web.RoutingContext ctx, int errorCode)
    Sets an error in a HTTP routing context so that a proper response can be returned to the client.
    protected void
    setJsonRequest(io.vertx.ext.web.RoutingContext ctx, boolean isJson)
    Mark the HTTP routing context as handling a JSON-encoded request.
    protected void
    setOtelContext(io.vertx.ext.web.RoutingContext ctx, io.opentelemetry.context.Context otelContext)
    Configure the telemetry context to be used when processing this HTTP request.
    protected void
    setOtelRequest(io.vertx.ext.web.RoutingContext ctx, REQ request)
    Sets the OTLP request object within the routing context for later use.
    protected void
    setOtelResponse(io.vertx.ext.web.RoutingContext ctx, RESP response)
    Sets the OTLP response object within the routing context for later use.
    protected abstract String
    telemetrySignalType()
    Get the type of telemetry signals processed by this collector.

    Methods inherited from class java.util.concurrent.SubmissionPublisher

    consume, estimateMaximumLag, estimateMinimumDemand, getClosedException, getExecutor, getMaxBufferCapacity, getNumberOfSubscribers, getSubscribers, hasSubscribers, isClosed, isSubscribed, offer, offer, submit, subscribe

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DEFAULT_TIMEOUT_SEC

      public static final int DEFAULT_TIMEOUT_SEC
      Default timeout (in sec) to process a single OTLP packet.
      See Also:

    • DEFAULT_CLOSE_TIMETOUT_SEC

      public static final int DEFAULT_CLOSE_TIMETOUT_SEC
      Default timeout (in sec) to complete a call to close() this collector.
      See Also:

    • DEFAULT_MAX_BODY_LEN

      public static final int DEFAULT_MAX_BODY_LEN
      Default maximum HTTP request body size in bytes.
      See Also:

    • DEFAULT_SUBSCRIBER_QUEUE_SIZE

      public static final int DEFAULT_SUBSCRIBER_QUEUE_SIZE
      Default size of a subscriber queue.
      See Also:

    • HTTP_OK

      public static final int HTTP_OK
      HTTP status code 200.
      See Also:

    • HTTP_BAD_REQUEST

      protected static final int HTTP_BAD_REQUEST
      HTTP status code 400.
      See Also:

    • HTTP_PAYLOAD_TOO_LARGE

      protected static final int HTTP_PAYLOAD_TOO_LARGE
      HTTP status code 413.
      See Also:

    • HTTP_EXPECTATION_FAILED

      protected static final int HTTP_EXPECTATION_FAILED
      HTTP status code 417.
      See Also:

    • HTTP_TOO_MANY_REQUESTS

      protected static final int HTTP_TOO_MANY_REQUESTS
      HTTP status code 429.
      See Also:

    • HTTP_INTERNAL_ERROR

      protected static final int HTTP_INTERNAL_ERROR
      HTTP status code 500.
      See Also:

    • HTTP_BAD_GATEWAY

      protected static final int HTTP_BAD_GATEWAY
      HTTP status code 502.
      See Also:

    • HTTP_SERVICE_UNAVAILABLE

      protected static final int HTTP_SERVICE_UNAVAILABLE
      HTTP status code 503.
      See Also:

    • HTTP_GATEWAY_TIMEOUT

      protected static final int HTTP_GATEWAY_TIMEOUT
      HTTP status code 504.
      See Also:

  • Constructor Details

    • AbstractCollector

      public AbstractCollector(String httpUrlPath, io.grpc.MethodDescriptor<REQ,RESP> exportMethod, Instrumentation otelHelper, Supplier<com.google.protobuf.Parser<REQ>> requestParser, Supplier<com.google.protobuf.Message.Builder> requestBuilder, Supplier<com.google.protobuf.Parser<RESP>> responseParser, Supplier<com.google.protobuf.Message.Builder> responseBuilder, ForkJoinPool threadPool)
      Create a new collector.
      Parameters:
      httpUrlPath - the URL path for HTTP requests
      exportMethod - the gRPC method to bind to
      otelHelper - a helper for own instrumentation
      requestParser - to get a parser for the request type
      requestBuilder - to get a builder for the request type
      responseParser - to get a parser for the response type
      responseBuilder - to get a builder for the response type
      threadPool - the thread pool to use for processing

  • Method Details

    • loadBatch

      protected abstract Batch<RECORD> loadBatch(REQ request, String transport, String encoding, io.opentelemetry.context.Context otelContext)
      Creates a new batch from a client's request.
      Parameters:
      request - the client's OTLP request
      transport - the OTLP transport used
      encoding - the encoding of the OTLP packet
      otelContext - context to use for processing
      Returns:
      a new Batch with all items of the request

    • getBatchResponse

      protected abstract RESP getBatchResponse(REQ request, Batch<RECORD> completedBatch, String transport, String encoding)
      Called to compute the correct response to the client after an entire batch (or packet) of OpenTelemetry logs, metrics or spans has been processed.
      Parameters:
      request - the original request from the client
      completedBatch - the completed batch of records
      transport - the OTLP transport used - "grpc" or "http"
      encoding - - the OTLP encoding - "protobuf" or "json"
      Returns:
      the correct OTLP reposne to be sent back to the client

    • telemetrySignalType

      protected abstract String telemetrySignalType()
      Get the type of telemetry signals processed by this collector. Used to format some strings like metric names, logs, etc.
      Returns:
      the signal type

    • initInstrumentation

      protected void initInstrumentation()
      Create own instrumentation metrics.

    • getInstrumentation

      protected Instrumentation getInstrumentation()
      Get the OpenTelemetry instrumentation helper.
      Returns:
      the instrumentation helper

    • parseHttpProtobuf

      protected REQ parseHttpProtobuf(InputStream is) throws Exception
      Parses an incoming HTTP protobuf-encoded request.
      Parameters:
      is - an input stream of the data
      Returns:
      the parsed request object
      Throws:
      Exception - if an error is encountered

    • parseHttpJson

      protected REQ parseHttpJson(InputStream is) throws Exception
      Parses an incoming HTTP json-encoded request.
      Parameters:
      is - an input stream of the data
      Returns:
      the parsed request object
      Throws:
      Exception - if an error is encountered

    • responseToJson

      protected String responseToJson(RESP response) throws Exception
      Converts a response object to JSON before sending back to client.
      Parameters:
      response - the response to format
      Returns:
      a JSON string
      Throws:
      Exception - if an error is encountered

    • responseToProtobuf

      protected byte[] responseToProtobuf(RESP response) throws Exception
      Converts a response object to protobuf before sending back to client.
      Parameters:
      response - the response to format
      Returns:
      a JSON string
      Throws:
      Exception - if an error is encountered

    • publish

      protected Batch<RECORD> publish(REQ request, String transport, String encoding, io.opentelemetry.context.Context otelContext)
      Publishes an OTLP packet (request) to all current subscribers.
      Parameters:
      request - the client's request
      transport - the OTLP transport used
      encoding - the encoding of the OTLP transport
      otelContext - a context to be used for own telemetry
      Returns:
      a Batch of all items

    • respond

      protected void respond(io.vertx.grpc.server.GrpcServerRequest<REQ,RESP> grpcRequest, REQ request, Batch<RECORD> batch, Throwable err)
      Responds to a processed gRPC OTLP request.
      Parameters:
      grpcRequest - the gRPC request
      request - the OTLP request
      batch - the processing Batch
      err - set if an error occurred during processing

    • handle

      protected void handle(io.vertx.grpc.server.GrpcServerRequest<REQ,RESP> request)
      Handles an incoming gRPC request.
      Parameters:
      request - the gRPC request

    • bind

      public void bind(io.vertx.grpc.server.GrpcServer server)
      Binds this collector to the gRPC server.
      Parameters:
      server - the gRPC server to bind to

    • checkContentType

      protected void checkContentType(io.vertx.ext.web.RoutingContext ctx)
      Checks the Content-Type header of an incoming HTTP request for validity.
      Parameters:
      ctx - the routing context of the request

    • checkBody

      protected void checkBody(io.vertx.ext.web.RoutingContext context)
      Checks the body of an incoming HTTP request for validity.
      Parameters:
      context - the routing context of the request

    • checkSubscribers

      protected void checkSubscribers(io.vertx.ext.web.RoutingContext ctx)
      Fail an incoming request if we have no subscribers.
      Parameters:
      ctx - the request's context

    • parseHttpRequest

      protected void parseHttpRequest(io.vertx.ext.web.RoutingContext ctx)
      Parse an incoming HTTP request.
      Parameters:
      ctx - the request context

    • publish

      protected void publish(io.vertx.ext.web.RoutingContext ctx, REQ request)
      Publish an incoming HTTP request to subscribers.
      Parameters:
      ctx - the request's context
      request - the request to publish

    • respond

      protected void respond(io.vertx.ext.web.RoutingContext ctx)
      Send the HTTP response back to the client.
      Parameters:
      ctx - the routing context of this HTTP request

    • bind

      public void bind(io.vertx.ext.web.Router router)
      Bind this collector to the given HTTP Router and start accepting requests.
      Parameters:
      router - the router

    • isJsonContent

      protected boolean isJsonContent(io.vertx.core.http.HttpServerRequest request)
      Check if the HTTP request is JSON formatted.
      Parameters:
      request - the request
      Returns:
      true if JSON was used by the client

    • isProtobufContent

      protected boolean isProtobufContent(io.vertx.core.http.HttpServerRequest request)
      Check if the HTTP request is protobuf formatted.
      Parameters:
      request - the request
      Returns:
      true if protobuf was used by the client

    • getOtelRequest

      protected REQ getOtelRequest(io.vertx.ext.web.RoutingContext ctx)
      Get the OTLP request object from a HTTP request.
      Parameters:
      ctx - the request context
      Returns:
      the OTLP request object

    • setOtelRequest

      protected void setOtelRequest(io.vertx.ext.web.RoutingContext ctx, REQ request)
      Sets the OTLP request object within the routing context for later use.
      Parameters:
      ctx - the routing context
      request - the OTLP request object

    • getOtelResponse

      protected RESP getOtelResponse(io.vertx.ext.web.RoutingContext ctx)
      Get the OTLP response object from the routing context.
      Parameters:
      ctx - the context
      Returns:
      the response object

    • setOtelResponse

      protected void setOtelResponse(io.vertx.ext.web.RoutingContext ctx, RESP response)
      Sets the OTLP response object within the routing context for later use.
      Parameters:
      ctx - the routing context
      response - the OTLP response object

    • isJsonRequest

      protected boolean isJsonRequest(io.vertx.ext.web.RoutingContext ctx)
      Check if the HTTP routing context was previously marked as for a JSON-encoded OTLP request.
      Parameters:
      ctx - the routing context
      Returns:
      true if OTLP request was JSON-encoded

    • setJsonRequest

      protected void setJsonRequest(io.vertx.ext.web.RoutingContext ctx, boolean isJson)
      Mark the HTTP routing context as handling a JSON-encoded request.
      Parameters:
      ctx - the routing context
      isJson - set to true to mark this request as JSON-encoded

    • httpEncoding

      protected String httpEncoding(io.vertx.ext.web.RoutingContext ctx)
      Gets the encoding of a HTTP OTLP request.
      Parameters:
      ctx - the routing context
      Returns:
      "json" or "protobuf"

    • isError

      protected boolean isError(io.vertx.ext.web.RoutingContext ctx)
      Checks if the processing of a HTTP request encountered an error.
      Parameters:
      ctx - the routing context
      Returns:
      true if an error was encountered

    • setErrorCode

      protected void setErrorCode(io.vertx.ext.web.RoutingContext ctx, int errorCode)
      Sets an error in a HTTP routing context so that a proper response can be returned to the client.
      Parameters:
      ctx - the routing context
      errorCode - the error code

    • getErrorCode

      protected int getErrorCode(io.vertx.ext.web.RoutingContext ctx)
      Returns the previously set error code for the given HTTP request.
      Parameters:
      ctx - the routing context
      Returns:
      the error code previously set

    • setOtelContext

      protected void setOtelContext(io.vertx.ext.web.RoutingContext ctx, io.opentelemetry.context.Context otelContext)
      Configure the telemetry context to be used when processing this HTTP request.
      Parameters:
      ctx - the HTTP routing context
      otelContext - the telemetry context to be set

    • getOtelContext

      protected io.opentelemetry.context.Context getOtelContext(io.vertx.ext.web.RoutingContext ctx)
      Get the telemetry context previously associated with this HTTP routing context.
      Parameters:
      ctx - the routing context
      Returns:
      the associated telemetry context

    • parseContentLengthHeader

      protected long parseContentLengthHeader(io.vertx.core.http.HttpServerRequest request)
      Parses the Content-Length header of an incoming HTTP request.
      Parameters:
      request - the HTTP request
      Returns:
      the parsed value or -1 if header is missing or unparseable

    • addSucceededRequests

      protected void addSucceededRequests(long count, String transport, String encoding)
      Updates the number of successful requests counter.
      Parameters:
      count - a count of how many to add
      transport - the OTLP transport used
      encoding - the OTLP transport encoding of the requests

    • addPartiallySucceededRequests

      protected void addPartiallySucceededRequests(long count, String transport, String encoding)
      Updates the number of partially successful requests counter.
      Parameters:
      count - a count of how many to add
      transport - the OTLP transport used
      encoding - the OTLP transport encoding of the requests

    • isRetryable

      protected boolean isRetryable(io.vertx.grpc.common.GrpcStatus status)
      Check if a given gRPC error can be retried by the client.
      Parameters:
      status - the gRPC error code
      Returns:
      true if error is retriable

    • isRetryable

      protected boolean isRetryable(int httpResponseStatusCode)
      Check if a given HTTP error can be retried by the client.
      Parameters:
      httpResponseStatusCode - the HTTP error code
      Returns:
      true if error is retriable

    • addGrpcFailedRequests

      protected void addGrpcFailedRequests(long count, io.vertx.grpc.common.GrpcStatus status)
      Update the number of failed requests counter for a given gRPC error code.
      Parameters:
      count - the count to add
      status - the gRPC error code

    • addFailedRequests

      protected void addFailedRequests(long count, String transport, String encoding, boolean retryable)
      Updates the number of failed requests counter.
      Parameters:
      count - a count of how many to add
      transport - the OTLP transport used
      encoding - the OTLP transport encoding of the requests
      retryable - true if the error allows the client to retry

    • addHttpFailedRequests

      protected void addHttpFailedRequests(long count, int httpResponseStatusCode, String encoding)
      Update the number of failed requests counter for a given HTTP error code.
      Parameters:
      count - the count to add
      httpResponseStatusCode - the HTTP error code
      encoding - the OTLP encoding used in the requests

    • addRequestsInProcess

      protected void addRequestsInProcess(long count, String transport, String encoding)
      Update the counter of number of requests currently being processed.
      Parameters:
      count - the count of requests to add or subtract (if negative)
      transport - the transport used by the requests
      encoding - the transport encoding

    • addRequestItems

      protected void addRequestItems(long count, String transport, String encoding)
      Update the counter of how many OTLP items (logs, spans, metrics) have been processed so far.
      Parameters:
      count - the number of items to add
      transport - the OTLP transport used
      encoding - the OTLP encoding used

    • addDroppedRequestItems

      protected void addDroppedRequestItems(long count, String transport, String encoding)
      Update the counter of how many OTLP items (logs, spans, metrics) have been dropped so far.
      Parameters:
      count - the number of items to add
      transport - the OTLP transport used
      encoding - the OTLP encoding used

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Overrides:
      close in class SubmissionPublisher<RECORD>

    • closeExceptionally

      public void closeExceptionally(Throwable error)
      Overrides:
      closeExceptionally in class SubmissionPublisher<RECORD>