Module aya.ij.parsing.core
Package com.intellij.lexer

Interface Lexer

All Known Subinterfaces:
LexerBase
All Known Implementing Classes:
DelegateLexer, FlexAdapter, LayeredLexer, LookAheadLexer, MergingLexerAdapter, MergingLexerAdapterBase, StringLiteralLexer
public interface Lexer
Interface for breaking a file into a sequence of tokens.
See Also:

  • Field Details

  • Method Details

    • start

      void start(@NotNull @NotNull CharSequence buffer, int startOffset, int endOffset, int initialState)
      Prepare for lexing character data from buffer passed. Internal lexer state is supposed to be initialState. It is guaranteed that the value of initialState is the same as returned by getState() method of this lexer at condition startOffset=getTokenStart(). This method is used to incrementally re-lex changed characters using lexing data acquired from this particular lexer sometime in the past.
      Parameters:
      buffer - character data for lexing.
      startOffset - offset to start lexing from
      endOffset - offset to stop lexing at
      initialState - the initial state of the lexer.

    • start

      default void start(@NotNull @NotNull CharSequence buf, int start, int end)

    • start

      default void start(@NotNull @NotNull CharSequence buf)

    • getTokenSequence

      @NotNull default @NotNull CharSequence getTokenSequence()

    • getTokenText

      @NotNull default @NotNull String getTokenText()

    • getState

      int getState()
      Returns the current state of the lexer.
      Returns:
      the lexer state.

    • getTokenType

      @Nullable @Nullable IElementType getTokenType()
      Returns the token at the current position of the lexer or null if lexing is finished.
      Returns:
      the current token.

    • getTokenStart

      int getTokenStart()
      Returns the start offset of the current token.
      Returns:
      the current token start offset.

    • getTokenEnd

      int getTokenEnd()
      Returns the end offset of the current token.
      Returns:
      the current token end offset.

    • advance

      void advance()
      Advances the lexer to the next token.

    • getCurrentPosition

      @NotNull @NotNull LexerPosition getCurrentPosition()
      Returns the current position and state of the lexer.
      Returns:
      the lexer position and state.

    • restore

      void restore(@NotNull @NotNull LexerPosition position)
      Restores the lexer to the specified state and position.
      Parameters:
      position - the state and position to restore to.

    • getBufferSequence

      @NotNull @NotNull CharSequence getBufferSequence()
      Returns the buffer sequence over which the lexer is running. This method should return the same buffer instance which was passed to the start() method.
      Returns:
      the lexer buffer.

    • getBufferEnd

      int getBufferEnd()
      Returns the offset at which the lexer will stop lexing. This method should return the length of the buffer or the value passed in the endOffset parameter to the start() method.
      Returns:
      the lexing end offset