Module aya.ij.parsing.core
Package com.intellij.lexer

Class Lexer

java.lang.Object
com.intellij.lexer.Lexer
Direct Known Subclasses:
LexerBase
public abstract class Lexer extends Object
Interface for breaking a file into a sequence of tokens.
See Also:

  • Constructor Details

    • Lexer

      public Lexer()

  • Method Details

    • start

      public abstract void start(@NotNull @NotNull CharSequence buffer, int startOffset, int endOffset, int initialState)
      Prepare for lexing character data from buffer passed. Internal lexer state is supposed to be initialState. It is guaranteed that the value of initialState is the same as returned by getState() method of this lexer at condition startOffset=getTokenStart(). This method is used to incrementally re-lex changed characters using lexing data acquired from this particular lexer sometime in the past.
      Parameters:
      buffer - character data for lexing.
      startOffset - offset to start lexing from
      endOffset - offset to stop lexing at
      initialState - the initial state of the lexer.

    • start

      public final void start(@NotNull @NotNull CharSequence buf, int start, int end)

    • start

      public final void start(@NotNull @NotNull CharSequence buf)

    • getTokenSequence

      @NotNull public @NotNull CharSequence getTokenSequence()

    • getTokenText

      @NotNull public @NotNull String getTokenText()

    • getState

      public abstract int getState()
      Returns the current state of the lexer.
      Returns:
      the lexer state.

    • getTokenType

      @Nullable public abstract @Nullable IElementType getTokenType()
      Returns the token at the current position of the lexer or null if lexing is finished.
      Returns:
      the current token.

    • getTokenStart

      public abstract int getTokenStart()
      Returns the start offset of the current token.
      Returns:
      the current token start offset.

    • getTokenEnd

      public abstract int getTokenEnd()
      Returns the end offset of the current token.
      Returns:
      the current token end offset.

    • advance

      public abstract void advance()
      Advances the lexer to the next token.

    • getCurrentPosition

      @NotNull public abstract @NotNull LexerPosition getCurrentPosition()
      Returns the current position and state of the lexer.
      Returns:
      the lexer position and state.

    • restore

      public abstract void restore(@NotNull @NotNull LexerPosition position)
      Restores the lexer to the specified state and position.
      Parameters:
      position - the state and position to restore to.

    • getBufferSequence

      @NotNull public abstract @NotNull CharSequence getBufferSequence()
      Returns the buffer sequence over which the lexer is running. This method should return the same buffer instance which was passed to the start() method.
      Returns:
      the lexer buffer.

    • getBufferEnd

      public abstract int getBufferEnd()
      Returns the offset at which the lexer will stop lexing. This method should return the length of the buffer or the value passed in the endOffset parameter to the start() method.
      Returns:
      the lexing end offset