kirk.io
Class FileTokenizer

java.lang.Object
  kirk.io.FileTokenizer

public class FileTokenizer

extends Object

This class reads a file token by token. Reading files token by token is only useful with text based files.

Methods in this class may generate an IOException.

Construction is done in one of the following ways:

 		new TokenFileReader(String file);
 			Most standard. Opens the passed file or throws an IOEception. Uses the
 			standard delimiters (" ") and any line terminating delimiters, and no
 			delimiting token returning.
 
 		new TokenFileReader(String file, String delims);
 			Opens the passed file or throws an IOEception. Uses the passed delimiters
 			and any line terminating delimiters, and no delimiting token returning.
 
 		new TokenFileReader(String file, String delims, boolean delimtokens);
 			Opens the passed file or throws an IOEception. Uses the passed delimiters
 			and any line terminating delimiters, and delimiting token returning.
 
 		new TokenFileReader(String file, boolean delimtokens);
 			Opens the passed file or throws an IOEception. Uses the standard
 			delimiters (" ") and any line terminating delimiters, and delimiting
 			token returning.
 

Reading the tokens in a file usually goes like this:

 		FileTokenizer ft = new FileTokenizer("file.txt");
 		while(ft.fileHasMoreTokens()) {
 			String token = ft.nextToken();
 			// Do something with token.
 		}
 

Because of the presence of the methods that deal with the current token (see below), you might want to act on the current token instead.
Below "String token = ft.currentToken();" can be replaced with any of the other token reading methods.

 		FileTokenizer ft = new FileTokenizer("file.txt");
 		while(ft.fileHasMoreTokens()) {
 			ft.nextToken();
 
 			String token = ft.currentToken();
 			// Do something with token.
 		}
 

Be aware that any call to currentToken() before any nextToken() is called after object construction, results in a null result.

Here are the other ways to retrieve the token:

 		getTokenAsString() will return the same result as currenttoken().
 		getTokenAsByte() will return the token as an Byte object, or null if that's
 			not possible.
 		getTokenAsCharacter() will return the token as an Character object, or null
 			if that's not possible.
 		getTokenAsDouble() will return the token as an Double object, or null if
 			that's not possible.
 		getTokenAsFloat() will return the token as an Float object, or null if that's
 			 not possible.
 		getTokenAsInteger() will return the token as an Integer object, or null if
 			that's not possible.
 		getTokenAsLong() will return the token as an Long object, or null if that's
 			not possible.
 		getTokenAsShort() will return the token as an Short object, or null if that's
 			not possible.
 

There are also two methods for checking if there are any tokens left.

 		fileHasMoreTokens() will return true iff the file has more tokens.
 			If !fileHasMoreTokens() then nextToken() will return null. After this
 			call any current token accessors will return null too.
 		lineHasMoreTokens() will return true iff the current line in the file has
 			more tokens.
 

Two methods are provided for checking on lines.

 		firsttoken() will return true iff the current token is the first one on the
 			current line.
 		lasttoken() will return true iff the current token is the last one on the
 			current line.
 

Naturally a method is included to close the file: close().

Since:

1.20

Version:

2.10

Author:

Berend "Kirk" Wouda

Field Summary

protected String	currentline The current line.
protected String	currenttoken The current token.
protected String	delimiters The tokenizer delimiters.
protected String	nextline The next line in the file.
protected BufferedReader	reader The BufferedReader that wraps the file.
protected boolean	returndelims The return delimiters flag.
protected StringTokenizer	tokenizer The StringTokenizer that is used to tokenize the current line.

Constructor Summary

FileTokenizer(String file)
Creates a new instance of this class with: file as the file this class works upon.

FileTokenizer(String file, boolean delimtokens)
Creates a new instance of this class with: file as the file this class works upon and delimtokens to indicate if tokens should be returned.

FileTokenizer(String file, String delims)
Creates a new instance of this class with: file as the file this class works upon and delims as the tokenizing delimiters.

FileTokenizer(String file, String delims, boolean delimtokens)
Creates a new instance of this class with: file as the file this class works upon, delims as the tokenizing delimiters and delimtokens to indicate if tokens should be returned.

Method Summary

void	close() Closes the file reader is reading from.
String	currentToken() Returns the current token, or null if there is none.
boolean	fileHasMoreTokens() Returns whether there are more tokens to be read in this file.
boolean	firstToken() Returns whether the current token is the first one on the line.
Byte	getTokenAsByte() Returns the current token as a Byte, or null if there is none/it isn't a Byte.
Character	getTokenAsCharacter() Returns the current token as a Character, or null if there is none/it isn't a Character.
Double	getTokenAsDouble() Returns the current token as a Double, or null if there is none/it isn't a Double.
Float	getTokenAsFloat() Returns the current token as a Float, or null if there is none/it isn't a Float.
Integer	getTokenAsInteger() Returns the current token as an Integer, or null if there is none/it isn't an Integer.
Long	getTokenAsLong() Returns the current token as a Long, or null if there is none/it isn't a Long.
Short	getTokenAsShort() Returns the current token as a Short, or null if there is none/it isn't a Short.
String	getTokenAsString() Returns the current token as a String (which it already is), or null if there is none.
boolean	lastToken() Returns whether the current token is the last one on the line.
boolean	lineHasMoreTokens() Returns whether there are more tokens to be read on this line.
protected void	nextLine() Advances to the next line.
String	nextToken() Returns the next token in the file, or null if there is none.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

currenttoken

protected String currenttoken

The current token.

currentline

protected String currentline

The current line.

nextline

protected String nextline

The next line in the file.

delimiters

protected String delimiters

The tokenizer delimiters.

returndelims

protected boolean returndelims

The return delimiters flag.

reader

protected BufferedReader reader

The BufferedReader that wraps the file.

tokenizer

protected StringTokenizer tokenizer

The StringTokenizer that is used to tokenize the current line.

Constructor Detail

FileTokenizer

public FileTokenizer(String file,
                     String delims,
                     boolean delimtokens)
              throws IOException

Creates a new instance of this class with:

file as the file this class works upon,
 delims as the tokenizing delimiters and
 delimtokens to indicate if tokens should be returned.

The actual FileReader isn't used really, a much nicer class called BufferedReader is used.

Parameters:

file - The file to be tokenized.

delims - The delimiters to be used for tokenizing.

delimtokens - Whether to tokenize the delimiters too.

See Also:

StringTokenizer

FileTokenizer

public FileTokenizer(String file,
                     String delims)
              throws IOException

Creates a new instance of this class with:

file as the file this class works upon and
 delims as the tokenizing delimiters.

 
 
The actual FileReader isn't used really, a much nicer class
 called BufferedReader is used.
 
 
This constructor overloads the first one.

Parameters:
file - The file to be tokenized.
delims - The delimiters to be used for tokenizing.

FileTokenizer

public FileTokenizer(String file,
                     boolean delimtokens)
              throws IOException

Creates a new instance of this class with:

file as the file this class works upon and
 delimtokens to indicate if tokens should be returned.

The actual FileReader isn't used really, a much nicer class called BufferedReader is used.

This constructor overloads the first one.

Parameters:

file - The file to be tokenized.

delimtokens - Whether to tokoenize the delimiters too.

FileTokenizer

public FileTokenizer(String file)
              throws IOException

Creates a new instance of this class with:

file as the file this class works upon.

The actual FileReader isn't used really, a much nicer class called BufferedReader is used.

This constructor overloads the first one.

Parameters:

file - The file to be tokenized.

Method Detail

close

public void close()
           throws IOException

Closes the file reader is reading from. This should be done before the object is dismissed.

Throws:

IOException

nextLine

protected void nextLine()
                 throws IOException

Advances to the next line.

Throws:

IOException

nextToken

public String nextToken()
                 throws IOException

Returns the next token in the file, or null if there is none. Also advances to the next token.

Returns:

The next token in the file, or null if there is none.

Throws:

IOException

currentToken

public String currentToken()

Returns the current token, or null if there is none.

Returns:

The current token, or null if there is none.

getTokenAsString

public String getTokenAsString()

Returns the current token as a String (which it already is), or null if there is none.

Returns:

The current token as a String (which it already is), or null if there is none.

getTokenAsByte

public Byte getTokenAsByte()

Returns the current token as a Byte, or null if there is none/it isn't a Byte.

Returns:

The current token as a Byte, or null if there is none/it isn't a Byte.

getTokenAsCharacter

public Character getTokenAsCharacter()

Returns the current token as a Character, or null if there is none/it isn't a Character.

Returns:

The current token as a Character, or null if there is none/it isn't a Character.

getTokenAsDouble

public Double getTokenAsDouble()

Returns the current token as a Double, or null if there is none/it isn't a Double.

Returns:

The current token as a Double, or null if there is none/it isn't a Double.

getTokenAsFloat

public Float getTokenAsFloat()

Returns the current token as a Float, or null if there is none/it isn't a Float.

Returns:

The current token as a Float, or null if there is none/it isn't a Float.

getTokenAsInteger

public Integer getTokenAsInteger()

Returns the current token as an Integer, or null if there is none/it isn't an Integer.

Returns:

The current token as an Integer, or null if there is none/it isn't an Integer.

getTokenAsLong

public Long getTokenAsLong()

Returns the current token as a Long, or null if there is none/it isn't a Long.

Returns:

The current token as a Long, or null if there is none/it isn't a Long.

getTokenAsShort

public Short getTokenAsShort()

Returns the current token as a Short, or null if there is none/it isn't a Short.

Returns:

The current token as a Short, or null if there is none/it isn't a Short.

fileHasMoreTokens

public boolean fileHasMoreTokens()

Returns whether there are more tokens to be read in this file.

Returns:

Whether there are more tokens to be read in this file.

lineHasMoreTokens

public boolean lineHasMoreTokens()

Returns whether there are more tokens to be read on this line.

Returns:

Whether there are more tokens to be read on this line.

firstToken

public boolean firstToken()

Returns whether the current token is the first one on the line.

Returns:

Whether the current token is the first one on the line.

lastToken

public boolean lastToken()

Returns whether the current token is the last one on the line.

Returns:

Whether the current token is the last one on the line.