kirk.io
Class FileTokenizer

java.lang.Object
  extended bykirk.io.FileTokenizer

public class FileTokenizer
extends Object

This class reads a file token by token. Reading files token by token is only useful with text based files.

Methods in this class may generate an IOException.

Construction is done in one of the following ways:

 		new TokenFileReader(String file);
 			Most standard. Opens the passed file or throws an IOEception. Uses the
 			standard delimiters (" ") and any line terminating delimiters, and no
 			delimiting token returning.
 
 		new TokenFileReader(String file, String delims);
 			Opens the passed file or throws an IOEception. Uses the passed delimiters
 			and any line terminating delimiters, and no delimiting token returning.
 
 		new TokenFileReader(String file, String delims, boolean delimtokens);
 			Opens the passed file or throws an IOEception. Uses the passed delimiters
 			and any line terminating delimiters, and delimiting token returning.
 
 		new TokenFileReader(String file, boolean delimtokens);
 			Opens the passed file or throws an IOEception. Uses the standard
 			delimiters (" ") and any line terminating delimiters, and delimiting
 			token returning.
 

Reading the tokens in a file usually goes like this:

 		FileTokenizer ft = new FileTokenizer("file.txt");
 		while(ft.fileHasMoreTokens()) {
 			String token = ft.nextToken();
 			// Do something with token.
 		}
 

Because of the presence of the methods that deal with the current token (see below), you might want to act on the current token instead.
Below "String token = ft.currentToken();" can be replaced with any of the other token reading methods.

 		FileTokenizer ft = new FileTokenizer("file.txt");
 		while(ft.fileHasMoreTokens()) {
 			ft.nextToken();
 
 			String token = ft.currentToken();
 			// Do something with token.
 		}
 

Be aware that any call to currentToken() before any nextToken() is called after object construction, results in a null result.

Here are the other ways to retrieve the token:

 		getTokenAsString() will return the same result as currenttoken().
 		getTokenAsByte() will return the token as an Byte object, or null if that's
 			not possible.
 		getTokenAsCharacter() will return the token as an Character object, or null
 			if that's not possible.
 		getTokenAsDouble() will return the token as an Double object, or null if
 			that's not possible.
 		getTokenAsFloat() will return the token as an Float object, or null if that's
 			 not possible.
 		getTokenAsInteger() will return the token as an Integer object, or null if
 			that's not possible.
 		getTokenAsLong() will return the token as an Long object, or null if that's
 			not possible.
 		getTokenAsShort() will return the token as an Short object, or null if that's
 			not possible.
 

There are also two methods for checking if there are any tokens left.

 		fileHasMoreTokens() will return true iff the file has more tokens.
 			If !fileHasMoreTokens() then nextToken() will return null. After this
 			call any current token accessors will return null too.
 		lineHasMoreTokens() will return true iff the current line in the file has
 			more tokens.
 

Two methods are provided for checking on lines.

 		firsttoken() will return true iff the current token is the first one on the
 			current line.
 		lasttoken() will return true iff the current token is the last one on the
 			current line.
 

Naturally a method is included to close the file: close().

Version:
1
Author:
Berend "Kirk" Wouda

Field Summary
protected  String currentline
          The current line.
protected  String currenttoken
          The current token.
protected  String delimiters
          The tokenizer delimiters.
protected  String nextline
          The next line in the file.
protected  BufferedReader reader
          The BufferedReader that wraps the file.
protected  boolean returndelims
          The return delimiters flag.
protected  StringTokenizer tokenizer
          The StringTokenizer that is used to tokenize the current line.
 
Constructor Summary
FileTokenizer(String file)
          Action: Creates a new instance of this class with: file as the file this class works upon.
FileTokenizer(String file, boolean delimtokens)
          Action: Creates a new instance of this class with: file as the file this class works upon and delimtokens to indicate if tokens should be returned.
FileTokenizer(String file, String delims)
          Creates a new instance of this class with: file as the file this class works upon and delims as the tokenizing delimiters.
FileTokenizer(String file, String delims, boolean delimtokens)
          Creates a new instance of this class with: file as the file this class works upon, delims as the tokenizing delimiters and delimtokens to indicate if tokens should be returned.
 
Method Summary
 void close()
          Closes the file reader is reading from.
 String currentToken()
          Returns the current token, or null if there is none.
 boolean fileHasMoreTokens()
          Returns whether there are more tokens to be read in this file.
 boolean firstToken()
          Returns whether the current token is the first one on the line.
 Byte getTokenAsByte()
          Returns the current token as a Byte, or null if there is none/it isn't a Byte.
 Character getTokenAsCharacter()
          Returns the current token as a Character, or null if there is none/it isn't a Character.
 Double getTokenAsDouble()
          Returns the current token as a Double, or null if there is none/it isn't a Double.
 Float getTokenAsFloat()
          Returns the current token as a Float, or null if there is none/it isn't a Float.
 Integer getTokenAsInteger()
          Returns the current token as an Integer, or null if there is none/it isn't an Integer.
 Long getTokenAsLong()
          Returns the current token as a Long, or null if there is none/it isn't a Long.
 Short getTokenAsShort()
          Returns the current token as a Short, or null if there is none/it isn't a Short.
 String getTokenAsString()
          Returns the current token as a String (which it already is), or null if there is none.
 boolean lastToken()
          Returns whether the current token is the last one on the line.
 boolean lineHasMoreTokens()
          Returns whether there are more tokens to be read on this line.
protected  void nextLine()
          Advances to the next line.
 String nextToken()
          Returns the next token in the file, or null if there is none.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

currenttoken

protected String currenttoken
The current token.


currentline

protected String currentline
The current line.


nextline

protected String nextline
The next line in the file.


delimiters

protected String delimiters
The tokenizer delimiters.


returndelims

protected boolean returndelims
The return delimiters flag.


reader

protected BufferedReader reader
The BufferedReader that wraps the file.


tokenizer

protected StringTokenizer tokenizer
The StringTokenizer that is used to tokenize the current line.

Constructor Detail

FileTokenizer

public FileTokenizer(String file,
                     String delims,
                     boolean delimtokens)
              throws IOException
Creates a new instance of this class with:
 file as the file this class works upon,
 delims as the tokenizing delimiters and
 delimtokens to indicate if tokens should be returned.
 
The actual FileReader isn't used really, a much nicer class called BufferedReader is used.

Parameters:
file - The file to be tokenized.
delims - The delimiters to be used for tokenizing.
delimtokens - Whether to tokoenize the delimiters too.
See Also:
StringTokenizer

FileTokenizer

public FileTokenizer(String file,
                     String delims)
              throws IOException
Creates a new instance of this class with:
 file as the file this class works upon and
 delims as the tokenizing delimiters.
 
 The actual FileReader isn't used really, a much nicer class
 called BufferedReader is used.
 This constructor overloads the first one.

Parameters:
file -
delims -

FileTokenizer

public FileTokenizer(String file,
                     boolean delimtokens)
              throws IOException
Action: Creates a new instance of this class with: file as the file this class works upon and delimtokens to indicate if tokens should be returned. The actual FileReader isn't used really, a much nicer class called BufferedReader is used. This constructor overloads the first one.

Parameters:
file -
delimtokens -

FileTokenizer

public FileTokenizer(String file)
              throws IOException
Action: Creates a new instance of this class with: file as the file this class works upon. The actual FileReader isn't used really, a much nicer class called BufferedReader is used. This constructor overloads the first one.

Parameters:
file -
Method Detail

close

public void close()
           throws IOException
Closes the file reader is reading from. This should be done before the object is dismissed.

Throws:
IOException

nextLine

protected void nextLine()
                 throws IOException
Advances to the next line.

Throws:
IOException

nextToken

public String nextToken()
                 throws IOException
Returns the next token in the file, or null if there is none. Also advances to the next token.

Returns:
The next token in the file, or null if there is none.
Throws:
IOException

currentToken

public String currentToken()
Returns the current token, or null if there is none.

Returns:
The current token, or null if there is none.

getTokenAsString

public String getTokenAsString()
Returns the current token as a String (which it already is), or null if there is none.

Returns:
The current token as a String (which it already is), or null if there is none.

getTokenAsByte

public Byte getTokenAsByte()
Returns the current token as a Byte, or null if there is none/it isn't a Byte.

Returns:
The current token as a Byte, or null if there is none/it isn't a Byte.

getTokenAsCharacter

public Character getTokenAsCharacter()
Returns the current token as a Character, or null if there is none/it isn't a Character.

Returns:
The current token as a Character, or null if there is none/it isn't a Character.

getTokenAsDouble

public Double getTokenAsDouble()
Returns the current token as a Double, or null if there is none/it isn't a Double.

Returns:
The current token as a Double, or null if there is none/it isn't a Double.

getTokenAsFloat

public Float getTokenAsFloat()
Returns the current token as a Float, or null if there is none/it isn't a Float.

Returns:
The current token as a Float, or null if there is none/it isn't a Float.

getTokenAsInteger

public Integer getTokenAsInteger()
Returns the current token as an Integer, or null if there is none/it isn't an Integer.

Returns:
The current token as an Integer, or null if there is none/it isn't an Integer.

getTokenAsLong

public Long getTokenAsLong()
Returns the current token as a Long, or null if there is none/it isn't a Long.

Returns:
The current token as a Long, or null if there is none/it isn't a Long.

getTokenAsShort

public Short getTokenAsShort()
Returns the current token as a Short, or null if there is none/it isn't a Short.

Returns:
The current token as a Short, or null if there is none/it isn't a Short.

fileHasMoreTokens

public boolean fileHasMoreTokens()
Returns whether there are more tokens to be read in this file.

Returns:
Whether there are more tokens to be read in this file.

lineHasMoreTokens

public boolean lineHasMoreTokens()
Returns whether there are more tokens to be read on this line.

Returns:
Whether there are more tokens to be read on this line.

firstToken

public boolean firstToken()
Returns whether the current token is the first one on the line.

Returns:
Whether the current token is the first one on the line.

lastToken

public boolean lastToken()
Returns whether the current token is the last one on the line.

Returns:
Whether the current token is the last one on the line.