The Standard Java Streams

Your operating system will typically define three standard streams that are accessible through members of the System class in Java:

• A standard input stream that usually corresponds to the keyboard by default. This is encapsulated by the in member of the System class, and is of type InputStream.
• A standard output stream that corresponds to output on the command line. This is encapsulated by the out member of the System class, and is of type PrintStream.
• A standard error output stream for error messages that usually maps to the command line output by default. This is encapsulated by the err member of the System class, and is also of type PrintStream.
  

Tokenizing a Stream

The StreamTokenizer class defines objects that can read an input stream and parse it into tokens. The input stream is read and treated as a series of separate bytes, and each byte is regarded as a character in the range 'u\0000' to 'u\00FF'. A StreamTokenizer object in its default state can recognize the following kinds of tokens:

To retrieve a token from the stream, you call the nextToken() method for the StreamTokenizer object:

int tokenType = 0;
try {
  while(tokenType = tokenizer.nextToken() != tokenizer.TT_EOF) {
     // Do something with the token...
  }
} catch (IOException e) {
  e.printStackTrace(System.err);
  System.exit(1);
}

Customizing a Stream Tokenizer

The default tokenizing mode can be modified by calling one or other of the following methods:

 resetSyntax()

 Resets the state of the tokenizer object so no characters have any special significance. This has the effect that all characters are regarded as ordinary, and will be read from the stream as single characters. The value of each character will be stored in the ttype field.

ordinaryChar (int ch)

Sets the character, ch, as an ordinary character. An ordinary character is a character that has no special significance. It will be read as a single character whose value will be stored in the ttype field. Calling this method will not alter the state of characters other than the argument value.
 
ordinaryChars (int low, int hi)

Causes all characters from low to hi inclusive to be treated as ordinary characters. Calling this method will not alter the state of characters other than those specified by the argument values.
 
whitespaceChars (int low, int hi)

Causes all characters from low to hi inclusive to be treated as whitespace characters. Unless they appear in a string, whitespace characters are treated as delimiters between tokens. Calling this method will not alter the state of characters other than those specified by the argument values.
 
wordChars (int low, int hi)

Specifies that the characters from low to hi inclusive are word characters. A word is at least one of these characters. Calling this method will not alter the state of characters other than those specified by the argument values.
 
commentChar(int ch)

Specifies that ch is a character that indicates the start of a comment. All characters to the end of the line following the character, ch, will be ignored. Calling this method will not alter the state of characters other than the argument value.
 
quoteChar(int ch)

Specifies that matching pairs of the character, ch, enclose a string. Calling this method will not alter the state of characters other than the argument value.
 
slashStarComments (boolean flag)

If the argument is false, this switches off recognizing comments between /* and */. A true argument switches it on again.
 
slashSlashComments (boolean flag)

If the argument is false, this switches off recognizing comments starting will a double slash. A true argument switches it on again.
 
lowerCaseMode (boolean flag)

An argument of true causes strings to be converted to lower case before being stored in sval. An argument of false switches off lower case mode.
 
pushback()

Calling this method causes the next call of the nextToken() method to return the ttype value that was set by the previous nextToken() call and to leave sval and nval unchanged.