File I/O: old I/O or NIO. Which is better?

Java introduced the New IO API in the 1.4 JDK release. The NIO API has been around for some time although in my experience, I have rarely seen the NIO APIs being used regularly. Developers have shown the tendency to prefer the older IO APIs of NIO.

There has been a healthy debate around whether the NIO is faster than IO. I have seen a number of articles contradicting each other. Therefore I decided to take the APIs for a spin.

First I need a file with large content to read. Below is my Java class LargeFileWriter which creates @300 MB file. The source code of LargeFileWriter is as below:

package com.vinraj.file.io;

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;

public class LargeFileWriter {

	private static final String FILE_CONTENT = "This is line number: ";

	public LargeFileWriter() {

	}

	private File createFile() {
		File f = new File(FileConstants.FILE_FOLDER + FileConstants.FILE_NAME);
		if(f.exists()) {
			f.delete();
		}
		try {
			boolean fileCreated = f.createNewFile();
			if (!fileCreated) {
				throw new RuntimeException("Cannot create File");
			}
		} catch (IOException e) {
			e.printStackTrace();
			throw new RuntimeException("Cannot create File");
		}
		return f;
	}

	private void writeContent(File f){
		try {
			FileWriter fw = new FileWriter(f);
			int i = 1;
			do {
				fw.write(FILE_CONTENT + i +" \r\n");
				i++;
			} while(i <= 11000000);
			fw.close();

		} catch (IOException e) {
			e.printStackTrace();
			throw new RuntimeException("Unable to write contents to file.");
		}

	}

	public static void main(String[] args) {
		LargeFileWriter lfw = new LargeFileWriter();
		File f = lfw.createFile();
		lfw.writeContent(f);
	}
}

Now that we are done with creating a large file. Let’s evaluate the file read process. I intend to use two read approaches. One is to read the file using a single reader thread and the other is to use 20 threads to read the same file. The reason for evaluating the multi-threaded approach is to check how NIO responds to file’s multi-threaded access. In old file I/O version, I had observed that the performance degraded due to possible file locking conflicts.

Below are three files FileConstants maintain some constant values. ConcurrentFileReader which is our test class file and NewFileReader which is responsible for spawning threads to read the large file concurrently.

package com.vinraj.file.io;

public class FileConstants {
	
	public static final String FILE_NAME = "largefile.txt";
	public static final String FILE_FOLDER = "D:/";

}
package com.vinraj.file.io;

import java.util.ArrayList;
import java.util.List;

public class ConcurrentFileReader {
	
	public static void main(String args[]) {
		
		System.out.println("System warmup.");
		List<Thread> thds = new ArrayList<Thread>(20);
		for(int i=0; i<=19; i++) {
			Thread t = new Thread(new NewFileReader(i+1));
			thds.add(t);
			t.start();
		}	
		checkIfThreadIsAlive(thds);
		System.out.println("System warmup completed.");
		
		System.out.println("Starting single thread Reader.");
		long startTime = System.nanoTime();
		Thread t = new Thread(new NewFileReader(999));
		t.start();
		while(t.isAlive()) {
		}
		long endTime = System.nanoTime();
		System.out.println("Ending single thread reader.");
		System.out.println("Single Thread Reading time: " + (float)(endTime-startTime)/1000000000 + " secs.");
		
		System.out.println("Starting multiple thread reader.");
		startTime = System.nanoTime();
		thds = new ArrayList<Thread>(20);
		for(int i=0; i<=19; i++) {
			t = new Thread(new NewFileReader(i+1));
			thds.add(t);
			t.start();
		}
		checkIfThreadIsAlive(thds);
		endTime = System.nanoTime();
		System.out.println("Multiple Thread Reading time: " + (float)(endTime-startTime)/1000000000 + " secs.");
		System.out.println("Ending multiple thread reader.");
	}

	private static void checkIfThreadIsAlive(List<Thread> thds) {
		boolean isAlive = true;
		while(isAlive) {
			boolean check = false;
			for (Thread thread : thds) {
				if (thread.isAlive()){
					check = true;
				}
			}
			isAlive = check;
		}
	}
}
package com.vinraj.file.io;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;

public class NewFileReader implements Runnable {

	private int i = 0;
	
	public NewFileReader(int i) {
		this.i = i;
	}
	
	@Override
	public void run() {
		try {
			BufferedReader br = new BufferedReader(new FileReader(new File(FileConstants.FILE_FOLDER 
					+ FileConstants.FILE_NAME)));
			String s = null;
			boolean readAllowed = true;
			while((s = br.readLine()) != null) {
				if (readAllowed) {
					readAllowed = false;
				}	
			}
			br.close();
			
		} catch (FileNotFoundException e) {
			e.printStackTrace();
			throw new RuntimeException("Unable to find file.");
		} catch (IOException e) {
			e.printStackTrace();
			throw new RuntimeException("Unable to read file.");
		}
	}
}

To replicate the same approach using NIO prefer the two classes below ConcurrentNIOFileReader2 and NewNIOFileReaderV3.

package com.vinraj.file.nio;

import java.util.ArrayList;
import java.util.List;

public class ConcurrentNIOFileReader2 {

	public static void main(String[] args) {
		System.out.println("System warmup.");
		List<Thread> thds = new ArrayList<Thread>(20);
		for(int i=0; i<=19; i++) {
			Thread t = new Thread(new NewNIOFileReaderV3());
			thds.add(t);
			t.start();
		}	
		checkIfThreadIsAlive(thds);
		System.out.println("System warmup completed.");
		
		System.out.println("Starting single thread Reader.");
		long startTime = System.nanoTime();
		Thread t = new Thread(new NewNIOFileReaderV3());
		t.start();
		while(t.isAlive()) {
		}
		long endTime = System.nanoTime();
		System.out.println("Ending single thread reader.");
		System.out.println("Single Thread Reading time: " + (float)(endTime-startTime)/1000000000 + " secs.");
		
		System.out.println("Starting multiple thread reader.");
		startTime = System.nanoTime();
		thds = new ArrayList<Thread>(20);
		for(int i=0; i<=19; i++) {
			t = new Thread(new NewNIOFileReaderV3());
			thds.add(t);
			t.start();
		}
		checkIfThreadIsAlive(thds);
		endTime = System.nanoTime();
		System.out.println("Multiple Thread Reading time: " + (float)(endTime-startTime)/1000000000 + " secs.");
		System.out.println("Ending multiple thread reader.");
	}

	private static void checkIfThreadIsAlive(List<Thread> thds) {
		boolean isAlive = true;
		while(isAlive) {
			boolean check = false;
			for (Thread thread : thds) {
				if (thread.isAlive()){
					check = true;
				}
			}
			isAlive = check;
		}
	}
}
package com.vinraj.file.nio;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

import com.vinraj.file.io.FileConstants;

public class NewNIOFileReaderV3 implements Runnable {

	@Override
	public void run() {
		try {
			FileInputStream f = new FileInputStream(FileConstants.FILE_FOLDER + FileConstants.FILE_NAME);
			FileChannel ch = f.getChannel();
			int MAXSIZE = 100000;
			int SIZE    = 50000;
			ByteBuffer bb = ByteBuffer.allocateDirect(MAXSIZE);
			byte[] barray = new byte[SIZE];
			int nRead, nGet;
			/*File file = new File("D:/sample.txt");
			file.delete();
			file.createNewFile();
			BufferedWriter bw = new BufferedWriter(new FileWriter(file));*/
			
			while ((nRead=ch.read(bb)) != -1) {
				bb.flip();
				bb.position(0);
			    bb.limit(nRead);
			    while(bb.hasRemaining()) {
			        nGet = Math.min(bb.remaining(), SIZE);
			        //This step cleans up existing content of the byte[]
			        barray = new byte[nGet];
			        bb.get(barray, 0, nGet); 
			        //Update:04/21/2011 Code changed to ensure 
			        //fairness by converting byte to character
			        String s = new String(barray).trim();
			        //bw.write(new String(barray).trim());
			    }			  
			    bb.clear();
			}
			
			//bw.flush();
			//bw.close();
			ch.close();
		} catch (FileNotFoundException e) {
			e.printStackTrace();
			throw new RuntimeException("File not found");
		} catch (IOException e) {
			e.printStackTrace();
			throw new RuntimeException("File IO Error");
		}
	}
}

There are some lines commented out. To validate if the behavior is as expected I wrote test code to copy all data from source file to another file. Note there is a more standard NIO way of transferring data between files as explained here.

Update(21-Apr-2011) As per vonbean’s suggestion have levelled the field by incorporating conversion of bytes to characters.

The run output of old I/O is:

System warmup.
System warmup completed.
Starting single thread Reader.
Ending single thread reader.
Single Thread Reading time: 3.9117332 secs.
Starting multiple thread reader.
Multiple Thread Reading time: 37.83342 secs.
Ending multiple thread reader.

The run output of NIO is:

System warmup.
System warmup completed.
Starting single thread Reader.
Ending single thread reader.
Single Thread Reading time: 7.60425 secs.
Starting multiple thread reader.
Multiple Thread Reading time: 31.793577 secs.
Ending multiple thread reader.

Update incorporating vonbean’s suggestion(21-Apr-2011)
There is a some benefit reading the file in a multi threaded mode using NIO vis-a-vis IO and however the same does not apply for single-threaded mode. I am sure there will be other factors such as buffer size and file size which might affect the overall performance but from my simplistic test. Appears that NIO is a winner in multi-threaded mode. Multi-threaded mode NIO seems to have lower lock contention.

Advertisements

3 thoughts on “File I/O: old I/O or NIO. Which is better?

  1. Your test code for single thread NIO can not correctly measure NIO’s characters processing performance. The NewNIOFileReaderV3 does not decode bytes to characters. The benchmark is unfair to BufferedReader. If you employ character decoding in your NewNIOFileReaderV3 thread, a significant performance drop can be observed.
    As my experience, the old java I/O outperformed than NIO when reading characters in single thread.

  2. Hi Vonbean,

    Many thanks for your critique. I have made suitable changes in the code and posted the revised benchmarks. I hope I have done justice to your suggestion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s