ZestyParser.Tokens

Package ZestyParser :: Module Tokens

[show private | hide private]

Module ZestyParser.Tokens

Version: 0.7.0

Author: Adam Atlas

Contact: adam@atlas.st

Classes
`AbstractToken`	Base class from which most tokens defined in this module derive.
`TokenWrapper`	If you write your own token type in a way other than subclassing AbstractToken, e.g.
Basic Tokens
`RawToken`	A class whose instances match only a particular string.
`Token`	A class whose instances match Python regular expressions.
`TakeToken`	A class whose instances match and return a given number of characters from the parser's `data`.
Complex Tokens
`CompositeToken`	A class whose instances match any of a number of tokens.
`TokenSequence`	A class whose instances match a sequence of tokens.
`TokenSeries`	A particularly versatile class whose instances match one token multiple times (with a great degree of customizability).
Special Tokens
`Defer`	A token which takes a callable (generally a lambda) which takes no arguments and itself returns a token.
`Default`	A class whose instances always return `desc` and do not advance the parser's cursor.
`Lookahead`	Scans for another token and returns its result as usual, but doesn't actually advance the parser's cursor.
`Negative`	Scans for another token, and only matches (returning True) if that token did not match.
TokenSequence Flags
`Omit`	See `TokenSequence`.
`Skip`	See `TokenSequence`.
`Only`	See `TokenSequence`.

Function Summary
	`Const(value)` This is a factory function used for when you want an `AbstractToken`-compatible callback that returns a constant.

Variable Summary
`Default`	`EmptyToken`: A `Default` instance initialized with the empty string.
`_EOF`	`EOF`: A token which matches (and returns `None`) if the parser is at the end of its `data` sequence.

Function Details

Const(value)

This is a factory function used for when you want an AbstractToken-compatible callback that returns a constant. Example:

   RawToken('foo') >> Const('bar')

This token matches 'foo' as usual, but always returns 'bar'.

Variable Details

EmptyToken

A Default instance initialized with the empty string.

Type:: Default
Value:: Default ''

EOF

A token which matches (and returns None) if the parser is at the end of its data sequence.

In ZestyParser, a token object must, at minimum, be a callable taking a ZestyParser instance and its current cursor as parameters. It can do whatever it needs with the parser's data and cursor properties before returning. It may raise NotMatched to indicate to the ZestyParser instance that it failed to match; it may also raise ParseError to indicate, for instance, that it began matching successfully but encountered an unrecoverable error.

The Tokens module contains a variety of predefined token classes (instances of which are callable) and other valid token objects which should cover most parsing situations.

Type:: _EOF
Value:: _EOF

Generated by Epydoc 2.1 on Thu Mar 22 02:13:39 2007

http://epydoc.sf.net