Tokenizer Class

Namespace: Contracts.Expressions
Assembly: Contracts (in Contracts.dll)

C++

JScript

Copy

Public Class Tokenizer

This language is not supported or no code example is available.

C++

JScript

Copy

public static class Tokenizer

This language is not supported or no code example is available.

C++

JScript

Copy

public ref class Tokenizer

This language is not supported or no code example is available.

C++

JScript

Copy

public class Tokenizer

This language is not supported or no code example is available.

	Name	Description
	Equals(Object)	Determines whether the specified object is equal to the current object. (inherited from Object).
	Finalize	Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (inherited from Object).
	GetHashCode	Serves as the default hash function. (inherited from Object).
	GetType	Gets the Type of the current instance. (inherited from Object).
	GobbleLiteralToken(StringBuilder, Queue<Token>, long)	Gobbles a literal token used in a command
	MemberwiseClone	Creates a shallow copy of the current Object. (inherited from Object).
	ToString	Returns a string that represents the current object. (inherited from Object).
	Tokenize(string, string, string)	Tokenize some data
	TokenizeCommandParameters(string)	Tokenize comand parameters - this is actually used in the parsing phrase currently - this is done here, rather than in the initial tokenize, so we know context

	Name	Description
	TokenState

Contracts.Expressions.Tokenizer

Supported in: 4.5, 4.5.1, 4.5.2, 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1