Package io.debezium.connector.mysql

Class RowDeserializers

  • public class RowDeserializers
extends Object
    Custom deserializers for the MySQL Binlog Client library.

    A few of the MySQL Binlog Client row deserializers convert MySQL raw row data into Date, Time, and Timestamp values using Calendar instances (and thus implicitly use the local timezone to calculate the milliseconds past epoch. Rather than perform this conversion, we prefer to convert the raw MySQL row values directly into LocalDate, LocalTime, LocalDateTime, and OffsetDateTime.

    Unfortunately, all of the methods used to deserialize individual values are defined on the AbstractRowsEventDataDeserializer base class, and inherited by the DeleteRowsEventDataDeserializer, UpdateRowsEventDataDeserializer, and WriteRowsEventDataDeserializer subclasses. Since we can't provide a new base class, the simplest way to override these methods is to subclass each of these 3 subclasses and override the methods on all 3 classes. It's ugly, but it works.

    See the MySQL Date Time documentation.

    Author:
    Randall Hauch

    • Constructor Detail

      • RowDeserializers

        private RowDeserializers()

    • Method Detail

      • deserializeString

        protected static Serializable deserializeString​(int length,
                                                com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                         throws IOException
        Converts a MySQL string to a byte[].
        Parameters:
        length - the number of bytes used to store the length of the string
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the byte[] object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeVarString

        protected static Serializable deserializeVarString​(int meta,
                                                   com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                            throws IOException
        Converts a MySQL string to a byte[].
        Parameters:
        meta - the meta value containing the number of bytes in the length field
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the byte[] object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeDate

        protected static Serializable deserializeDate​(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                       throws IOException
        Converts a MySQL DATE value to a LocalDate.

        This method treats all zero values for DATE columns as NULL, since they cannot be accurately represented as valid LocalDate objects.

        Parameters:
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the LocalDate object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeTime

        protected static Serializable deserializeTime​(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                       throws IOException
        Converts a MySQL TIME value without fractional seconds to a Duration.
        Parameters:
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the LocalTime object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeTimeV2

        protected static Serializable deserializeTimeV2​(int meta,
                                                com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                         throws IOException
        Converts a MySQL TIME value with fractional seconds to a Duration.
        Parameters:
        meta - the meta value containing the fractional second precision, or fsp
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the Duration object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeDatetime

        protected static Serializable deserializeDatetime​(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                           throws IOException
        Converts a MySQL DATETIME value without fractional seconds to a LocalDateTime.

        This method treats all zero values for DATETIME columns as NULL, since they cannot be accurately represented as valid LocalDateTime objects.

        Parameters:
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the LocalDateTime object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeDatetimeV2

        protected static Serializable deserializeDatetimeV2​(int meta,
                                                    com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                             throws IOException
        Converts a MySQL DATETIME value with fractional seconds to a LocalDateTime.

        This method treats all zero values for DATETIME columns as NULL, since they cannot be accurately represented as valid LocalDateTime objects.

        Parameters:
        meta - the meta value containing the fractional second precision, or fsp
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the LocalDateTime object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeTimestamp

        protected static Serializable deserializeTimestamp​(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                            throws IOException
        Converts a MySQL TIMESTAMP value without fractional seconds to a OffsetDateTime. MySQL stores the TIMESTAMP values as seconds past epoch in UTC, but the resulting OffsetDateTime will be in the local timezone.
        Parameters:
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the OffsetDateTime object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeTimestampV2

        protected static Serializable deserializeTimestampV2​(int meta,
                                                     com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                              throws IOException
        Converts a MySQL TIMESTAMP value with fractional seconds to a OffsetDateTime. MySQL stores the TIMESTAMP values as seconds + fractional seconds past epoch in UTC, but the resulting OffsetDateTime will be in the local timezone.
        Parameters:
        meta - the meta value containing the fractional second precision, or fsp
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the OffsetDateTime object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeYear

        protected static Serializable deserializeYear​(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                       throws IOException
        Converts a MySQL YEAR value to a Year object.
        Parameters:
        inputStream - the binary stream containing the raw binlog event data for the value
        Returns:
        the Year object
        Throws:
        IOException - if there is an error reading from the binlog event data

      • split

        protected static int[] split​(long value,
                             int divider,
                             int length)
        Split the integer into multiple integers.

        We can't use/access the private split method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

        Parameters:
        value - the long value
        divider - the value used to separate the individual values (e.g., 10 to separate each digit into a separate value, 100 to separate each pair of digits into a separate value, 1000 to separate each 3 digits into a separate value, etc.)
        length - the expected length of the integer array
        Returns:
        the integer values

      • bigEndianLong

        protected static long bigEndianLong​(byte[] bytes,
                                    int offset,
                                    int length)
        Read a big-endian long value.

        We can't use/access the private bigEndianLong method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

        Parameters:
        bytes - the bytes containing the big-endian representation of the value
        offset - the offset within the bytes byte array where the value starts
        length - the length of the byte representation within the bytes byte array
        Returns:
        the long value

      • bitSlice

        protected static int bitSlice​(long value,
                              int bitOffset,
                              int numberOfBits,
                              int payloadSize)
        Slice an integer out of a portion of long value.

        We can't use/access the private bitSlice method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

        Parameters:
        value - the long containing the integer encoded within it
        bitOffset - the number of bits where the integer value starts
        numberOfBits - the number of bits in the integer value
        payloadSize - the total number of bits used in the value
        Returns:
        the integer value

      • deserializeFractionalSecondsInNanos

        protected static int deserializeFractionalSecondsInNanos​(int fsp,
                                                         com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                                  throws IOException
        Read the binary input stream to obtain the number of nanoseconds given the fractional seconds precision, or fsp.

        We can't use/access the deserializeFractionalSeconds method in the AbstractRowsEventDataDeserializer class, so we replicate it here with modifications to support nanoseconds rather than microseconds. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

        Parameters:
        fsp - the fractional seconds precision describing the number of digits precision used to store the fractional seconds (e.g., 1 for storing tenths of a second, 2 for storing hundredths, 3 for storing milliseconds, etc.)
        inputStream - the binary data stream
        Returns:
        the number of nanoseconds
        Throws:
        IOException - if there is an error reading from the binlog event data

      • deserializeFractionalSecondsInNanosNegative

        protected static int deserializeFractionalSecondsInNanosNegative​(int fsp,
                                                                 com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream)
                                                          throws IOException
        Read the binary input stream to obtain the number of nanoseconds given the fractional seconds precision, or fsp.

        We can't use/access the deserializeFractionalSeconds method in the AbstractRowsEventDataDeserializer class, so we replicate it here with modifications to support nanoseconds rather than microseconds and negative values. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

        Parameters:
        fsp - the fractional seconds precision describing the number of digits precision used to store the fractional seconds (e.g., 1 for storing tenths of a second, 2 for storing hundredths, 3 for storing milliseconds, etc.)
        inputStream - the binary data stream
        Returns:
        the number of nanoseconds
        Throws:
        IOException - if there is an error reading from the binlog event data