Module aya.ij.parsing.core
Package com.intellij.lexer

Class StringLiteralLexer

java.lang.Object
com.intellij.lexer.StringLiteralLexer
All Implemented Interfaces:
Lexer, LexerBase
public class StringLiteralLexer extends Object implements LexerBase

  • Field Details

    • NO_QUOTE_CHAR

      public static final char NO_QUOTE_CHAR
      See Also:

    • myBuffer

      protected CharSequence myBuffer

    • myStart

      protected int myStart

    • myEnd

      protected int myEnd

    • myBufferEnd

      protected int myBufferEnd

    • myQuoteChar

      protected final char myQuoteChar

    • myOriginalLiteralToken

      protected final IElementType myOriginalLiteralToken

  • Constructor Details

    • StringLiteralLexer

      public StringLiteralLexer(char quoteChar, IElementType originalLiteralToken)

    • StringLiteralLexer

      public StringLiteralLexer(char quoteChar, IElementType originalLiteralToken, boolean canEscapeEolOrFramingSpaces, String additionalValidEscapes)

    • StringLiteralLexer

      public StringLiteralLexer(char quoteChar, IElementType originalLiteralToken, boolean canEscapeEolOrFramingSpaces, String additionalValidEscapes, boolean allowOctal, boolean allowHex)
      Parameters:
      canEscapeEolOrFramingSpaces - true if following sequences are acceptable:
      * '\' at the end of the buffer (meaning escaped end of line)
      * '\ ' (escaped space) at the beginning and ath the end of the buffer (meaning escaped space, to avoid auto trimming on load)

  • Method Details

    • start

      public void start(@NotNull @NotNull CharSequence buffer, int startOffset, int endOffset, int initialState)
      Description copied from interface: Lexer
      Prepare for lexing character data from buffer passed. Internal lexer state is supposed to be initialState. It is guaranteed that the value of initialState is the same as returned by Lexer.getState() method of this lexer at condition startOffset=getTokenStart(). This method is used to incrementally re-lex changed characters using lexing data acquired from this particular lexer sometime in the past.
      Specified by:
      start in interface Lexer
      Parameters:
      buffer - character data for lexing.
      startOffset - offset to start lexing from
      endOffset - offset to stop lexing at
      initialState - the initial state of the lexer.

    • getState

      public int getState()
      Description copied from interface: Lexer
      Returns the current state of the lexer.
      Specified by:
      getState in interface Lexer
      Returns:
      the lexer state.

    • getTokenType

      public IElementType getTokenType()
      Description copied from interface: Lexer
      Returns the token at the current position of the lexer or null if lexing is finished.
      Specified by:
      getTokenType in interface Lexer
      Returns:
      the current token.

    • shouldAllowSlashZero

      protected boolean shouldAllowSlashZero()

    • handleSingleSlashEscapeSequence

      @NotNull protected @NotNull IElementType handleSingleSlashEscapeSequence()

    • getHexCodedEscapeSeq

      protected IElementType getHexCodedEscapeSeq()

    • getUnicodeEscapeSequenceType

      @NotNull protected @NotNull IElementType getUnicodeEscapeSequenceType()

    • getStandardLimitedHexCodedEscapeSeq

      @NotNull protected @NotNull IElementType getStandardLimitedHexCodedEscapeSeq(int offsetLimit)

    • getTokenStart

      public int getTokenStart()
      Description copied from interface: Lexer
      Returns the start offset of the current token.
      Specified by:
      getTokenStart in interface Lexer
      Returns:
      the current token start offset.

    • getTokenEnd

      public int getTokenEnd()
      Description copied from interface: Lexer
      Returns the end offset of the current token.
      Specified by:
      getTokenEnd in interface Lexer
      Returns:
      the current token end offset.

    • locateHexEscapeSequence

      protected int locateHexEscapeSequence(int start, int i)

    • locateUnicodeEscapeSequence

      protected int locateUnicodeEscapeSequence(int start, int i)

    • locateAdditionalEscapeSequence

      protected int locateAdditionalEscapeSequence(int start, int indexOfCharAfterSlash)

      Locates the end of an additional (non-standard) escape sequence. The sequence is considered to begin with a backslash symbol located at the start index of the lexer's buffer.

      Override this method if your language supports non-standard escape sequences. For example, in Go language Unicode escapes look like '\U12345678'. To locate this escape sequence, the implementation should check that indexOfCharAfterSlash points to the 'U' symbol in the buffer and return start + 8 (or the end index of the buffer if it is too short). Otherwise, the implementations should return -1 to indicate that the current buffer starting at the index start doesn't represent an additional escape sequence.

      When overriding this method, you most likely will need to also override getTokenType() to return proper type for the sequences located here.

    • advance

      public void advance()
      Description copied from interface: Lexer
      Advances the lexer to the next token.
      Specified by:
      advance in interface Lexer

    • getBufferSequence

      @NotNull public @NotNull CharSequence getBufferSequence()
      Description copied from interface: Lexer
      Returns the buffer sequence over which the lexer is running. This method should return the same buffer instance which was passed to the start() method.
      Specified by:
      getBufferSequence in interface Lexer
      Returns:
      the lexer buffer.

    • getBufferEnd

      public int getBufferEnd()
      Description copied from interface: Lexer
      Returns the offset at which the lexer will stop lexing. This method should return the length of the buffer or the value passed in the endOffset parameter to the start() method.
      Specified by:
      getBufferEnd in interface Lexer
      Returns:
      the lexing end offset

    • toString

      public String toString()
      Overrides:
      toString in class Object