Class Lexer

Namespace

Elegy.Common.Text

Assembly

Elegy.Common.dll

Lexicological parser, parses tokens out of a continuous piece of text. Ported from adm-utils: https://github.com/Admer456/adm-utils/blob/master/src/Text/Lexer.hpp

public class Lexer : ILexer

Inheritance

object

Lexer

Implements

ILexer

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Right now it works with string, it may be worth investigating a ReadOnlySpan<T> version.

Constructors

Lexer(string, string)

public Lexer(string text, string delimiters = "()[]{}.:,;")

Parameters

text string

delimiters string

Fields

DelimitersFull

public const string DelimitersFull = "()[]{}.:,;=+-*/&@'?!#%"

Field Value

string

DelimitersSimple

public const string DelimitersSimple = "()[]{}.:,;"

Field Value

string

Properties

IgnoreDelimiters

Whether or not to ignore delimiters while scanning for characters.

public bool IgnoreDelimiters { get; set; }

Property Value

bool

Methods

Clear()

Clears the text buffer.

public void Clear()

Expect(string, bool)

Peeks at the next string and, optionally, advances seeking.

public bool Expect(string expectedToken, bool advanceIfTrue = false)

Parameters

expectedToken string

The token to expect.

advanceIfTrue bool

Advance the seeking upon success? Behaves about the same as Next then.

Returns

bool

true if the next token and expectedToken match.

ExpectAnythingUntilNewline()

Is there anything other than whitespaces until the newline?

public bool ExpectAnythingUntilNewline()

Returns

bool

GetLineInfo()

Gets the current line and column, formatted in a string.

public string GetLineInfo()

Returns

string

GetLineNumber()

Gets the current line number.

public int GetLineNumber()

Returns

int

IsEnd()

Whether or not the end of the text has been reached.

public bool IsEnd()

Returns

bool

Load(string)

Loads the given text

public void Load(string text)

Parameters

text string

NewLine()

Skip the current line.

public void NewLine()

Next()

Continues parsing the string and extracts the next token from it.

public string Next()

Returns

string

The next token in line, "" if EOF is reached.

ParsingException(string)

Throws an exception with the attached errorMessage. Includes line and column info.

public Exception ParsingException(string errorMessage)

Parameters

errorMessage string

Returns

Exception

Peek()

Peeks at the next string.

public string Peek()

Returns

string

The next token in line, "" if EOF is reached.

PeekUntil(string, bool, bool)

Returns all characters that were encountered between the current position and what. If skipPeeked is true, the cursor will skip the result too.

public string PeekUntil(string what, bool skipPeeked = false, bool skipWhatToo = true)

Parameters

what string

skipPeeked bool

skipWhatToo bool

Returns

string

SetDelimiters(string)

Sets new delimiters.

public void SetDelimiters(string delimiters)

Parameters

delimiters string

SkipUntil(string, bool)

Skips characters until a given string what has been found. If skipThatToo is true, it'll skip that string as well.

public void SkipUntil(string what, bool skipThatToo = false)

Parameters

what string

skipThatToo bool

TokenListBeforeNewline()

An array of tokens until the newline.

public IList<string> TokenListBeforeNewline()

Returns

IList<string>

TokensBeforeNewline()

Returns a string of tokens until the newline.

public string TokensBeforeNewline()

Returns

string