com.svincent.moksa
Class PrologTokenizer

java.lang.Object
  |
  +--com.svincent.moksa.WamObject
        |
        +--com.svincent.moksa.PrologTokenizer

public class PrologTokenizer

extends WamObject

This class encapsulates the tokenization of Prolog.

A PrologTokenizer has a single method, 'getToken', which returns a PrologTerm. This term is one of:

• atom ('name') - an atom token (i.e. - foo, a32, '$').
• variable ('Name') - a variable token (i.e. - Foo, _, and A32).
• integer (value)
- an integral value token (i.e. - 42, -3, 7).
• float (value)
- a floating-point value token (i.e. - 3.141592, 42.0).
• string (value)
- a string value token (i.e. - "Four score and seven years ago...", "hello").
• open
- an open parenthesis token (i.e. - ().
• close
- a close parenthesis token (i.e. - )).
• open_list
- an open square token (i.e. - [).
• close_list
- a close square token (i.e. - ]).
• open_curly
- an open curly token (i.e. - {).
• close_curly
- a close curly token (i.e. - }).
• head_tail_separator
- a list head/tail seperator token (i.e. - |).
• end
- an end token (i.e. - .).
• end_of_file
- an end of file token, returned when we run out of tokens on the input stream.
• error(reason)
- an error occurred (i.e. - .).

All strings, etc, have quote marks stripped and escape sequences expanded, so there is no need to do these things later on.

I believe that MoksaProlog will ship with two complete parsers: one written in Prolog, which supports the entire Prolog language, and one written completely in Java, which supports a subset.

Both of these parsers, I believe, shall be accessable to Prolog users, through various and sundry APIs.

Constructor Summary

PrologTokenizer(PrologFactory _factory)

Method Summary

void	consume(Io.PrologInput in, CompoundTerm expected)
static void	main(java.lang.String[] args) Test code.
CompoundTerm	parseCompoundTerm(java.lang.String value) From something of the form "Shawn" Make something of the form [83,104,97,119,110]
CompoundTerm	peek(Io.PrologInput in)
void	pushback(Io.PrologInput in, CompoundTerm token)
CompoundTerm	readToken(Io.PrologInput in) Read a single token from 'in'.

Methods inherited from class com.svincent.moksa.WamObject

tag, tag

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PrologTokenizer

public PrologTokenizer(PrologFactory _factory)

Method Detail

pushback

public void pushback(Io.PrologInput in,
                     CompoundTerm token)

peek

public CompoundTerm peek(Io.PrologInput in)
                  throws java.io.IOException

consume

public void consume(Io.PrologInput in,
                    CompoundTerm expected)
             throws java.io.IOException,
                    PrologParseException

readToken

public CompoundTerm readToken(Io.PrologInput in)
                       throws java.io.IOException

Read a single token from 'in'. This is the only public API on PrologTokenizer: all the rest of its methods exist solely to service it.

parseCompoundTerm

public CompoundTerm parseCompoundTerm(java.lang.String value)

From something of the form "Shawn" Make something of the form [83,104,97,119,110]

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Test code.