import Token from 'perplex/lib/token.js'

public class | source

Token

Represents a token instance

Constructor Summary

Public Constructor
public	constructor(type: T, match: string, groups: string[], start: number, end: number, lexer: Lexer<T>) Constructs a token

Public Members
public	end: number The string position where this match ends
public	groups: string[] Any RegExp groups that accrued during the match
public	lexer: Lexer<T> The parent Lexer
public	match: string The string that the lexer consumed to create this token
public	start: number The string position where this match started
public	type: T The token type

Public Methods
public	isEof(): boolean
public	strpos(): TokenPosition Returns the bounds of this token, each in `{line, column}` format

Constructs a token

Name	Type	Attribute	Description
type	T		The token type
match	string		The string that the lexer consumed to create this token
groups	string[]		Any RegExp groups that accrued during the match
start	number		The string position where this match started
end	number		The string position where this match ends
lexer	Lexer<T>		The parent Lexer

The string position where this match ends

Any RegExp groups that accrued during the match

The parent Lexer

The string that the lexer consumed to create this token

The string position where this match started

The token type

Returns the bounds of this token, each in {line, column} format