non-blocking IO vs async IO and implementation in Java

Solution 1:

I see this is an old question, but I think something was missed here, that @nickdu attempted to point out but wasn't quite clear.

There are four types of IO pertinent to this discussion:

Blocking IO

Non-Blocking IO

Asynchronous IO

Asynchronous Non-Blocking IO

The confusion arises I think because of ambiguous definitions. So let me attempt to clarify that.

First Let's talk about IO. When we have slow IO this is most apparent, but IO operations can either be blocking or non-blocking. This has nothing to do with threads, it has to do with the interface to the operating system. When I ask the OS for an IO operation I have the choice of waiting for all the data to be ready (blocking), or getting what is available right now and moving on (non-blocking). The default is blocking IO. It is much easier to write code using blocking IO as the path is much clearer. However, your code has to stop and wait for IO to complete. Non-Blocking IO requires interfacing with the IO libraries at a lower level, using select and read/write instead of the higher level libraries that provide convenient operations. Non-Blocking IO also implies that you have something you need to work on while the OS works on doing the IO. This might be multiple IO operations or computation on the IO that has completed.

Blocking IO - The application waits for the OS to gather all the bytes to complete the operation or reach the end before continuing. This is default. To be more clear for the very technical, the system call that initiates the IO will install a signal handler waiting for a processor interrupt that will occur when the IO operation makes progress. Then the system call will begin a sleep which suspends operation of the current process for a period of time, or until the process interrupt occurs.

Non-Blocking IO - The application tells the OS it only wants what bytes are available right now, and moves on while the OS concurrently gathers more bytes. The code uses select to determine what IO operations have bytes available. In this case the system call will again install a signal handler, but rather than sleep, it will associate the signal handler with the file handle, and immediately return. The process will become responsible for periodically checking the file handle for the interrupt flag having been set. This is usually done with a select call.

Now Asynchronous is where the confusion begins. The general concept of asynchronous only implies that the process continues while the background operation is performed, the mechanism by which this occurs is not specific. The term is ambiguous as both non-blocking IO and threaded blocking IO can be considered to be asynchronous. Both allow concurrent operations, however the resource requirements are different, and the code is substantially different. Because you have asked a question "What is Non-Blocking Asynchronous IO", I am going to use a stricter definition for asynchronous, a threaded system performing IO which may or may not be non-blocking.

The general definition

Asynchronous IO - Programmatic IO which allows multiple concurrent IO operations to occur. IO operations are happening simultaneously, so that code is not waiting for data that is not ready.

The stricter definition

Asynchronous IO - Programmatic IO which uses threading or multiprocessing to allow concurrent IO operations to occur.

Now with those clearer definitions we have the following four types of IO paradigms.

Blocking IO - Standard single threaded IO in which the application waits for all IO operations to complete before moving on. Easy to code, no concurrency and so slow for applications that require multiple IO operations. The process or thread will sleep while waiting for the IO interrupt to occur.

Asynchronous IO - Threaded IO in which the application uses threads of execution to perform Blocking IO operations concurrently. Requires thread safe code, but is generally easier to read and write than the alternative. Gains the overhead of multiple threads, but has clear execution paths. May require the use of synchronized methods and containers.

Non-Blocking IO - Single threaded IO in which the application uses select to determine which IO operations are ready to advance, allowing the execution of other code or other IO operations while the OS processes concurrent IO. The process does not sleep while waiting for the IO interrupt, but takes on the responsibility to check for the IO flag on the filehandle. Much more complicated code due to the need to check the IO flag with select, though does not require thread-safe code or synchronized methods and containers. Low execution over-head at the expense of code complexity. Execution paths are convoluted.

Asynchronous Non-Blocking IO - A hybrid approach to IO aimed at reducing complexity by using threads, while maintaining scalability by using non-blocking IO operations where possible. This would be the most complex type of IO requiring synchronized methods and containers, as well as convoluted execution paths. This is not the type of IO that one should consider coding lightly, and is most often only used when using a library that will mask the complexity, something like Futures and Promises.

Solution 2:

So what is actually "non-blocking async IO"?

To answer that, you must first understand that there's no such thing as blocking async I/O. The very concept of asynchronism dictates that there's no waiting, no blocking, no delay. When you see non-blocking asynchronous I/O, the non-blocking bit only serves to further qualify the async adjective in that term. So effectively, non-blocking async I/O might be a bit of a redundancy.

There are mainly two kinds of I/O. Synchronous and Asynchronous. Synchronous blocks the current thread of execution until processing is complete, while Asynchronous doesn't block the current thread of execution, rather passing control to the OS Kernel for further processing. The kernel then advises the async thread when the submitted task is complete

Asynchronous Channel Groups

The concept of Async Channels in java is backed by Asynchronous Channel Groups. An async channel group basically pools a number of channels for reuse. Consumers of the async api retrieve a channel from the group (the JVM creates one by default) and the channel automatically puts itself back into the group after it's completed its read/write operation. Ultimately, Async Channel Groups are backed by surprise, threadpools. Also, Asynchronous channels are threadsafe.

The size of the threadpool that backs an async channel group is configured by the following JVM property

java.nio.channels.DefaultThreadPool.initialSize

which, given an integer value will setup a threadpool of that size, to back the channel group. The channel group is created and maintained transparently to the developer otherwise.

And how all them can be implemented in Java

Well, I'm glad you asked. Here's an example of an AsynchronousSocketChannel (used to open a non-blocking client Socket to a listening server.) This sample is an excerpt from Apress Pro Java NIO.2, commented by me:

//Create an Asynchronous channel. No connection has actually been established yet
AsynchronousSocketChannel asynchronousSocketChannel = AsynchronousSocketChannel.open(); 

/**Connect to an actual server on the given port and address. 
   The operation returns a type of Future, the basis of the all 
   asynchronous operations in java. In this case, a Void is 
   returned because nothing is returned after a successful socket connection
  */
Void connect = asynchronousSocketChannel.connect(new InetSocketAddress("127.0.0.1", 5000)).get();


//Allocate data structures to use to communicate over the wire
ByteBuffer helloBuffer = ByteBuffer.wrap("Hello !".getBytes()); 

//Send the message

Future<Integer> successfullyWritten=  asynchronousSocketChannel.write(helloBuffer);

//Do some stuff here. The point here is that asynchronousSocketChannel.write() 
//returns almost immediately, not waiting to actually finish writing 
//the hello to the channel before returning control to the currently executing thread

doSomethingElse();

//now you can come back and check if it was all written (or not)

System.out.println("Bytes written "+successfullyWritten.get());

EDIT: I should mention that support for Async NIO came in JDK 1.7

Solution 3:

I would say there are three types of io:

synchronous blocking
synchronous non-blocking
asynchronous

Both synchronous non-blocking and asynchronous would be considered non-blocking as the calling thread is not waiting on the IO to complete. So while non-blocking asynchronous io might be redundant, they are not one in the same. When I open a file I can open it in non-blocking mode. What does this mean? It means when I issue a read() it won't block. It will either return me the bytes that are available or indicate that there are no bytes available. If I didn't enable non-blocking io the read() would block until data was available. I might want to enable non-blocking io if I want a thread to handle multiple io requests. For instance, I could use select() to find out what file descriptors, or maybe sockets, have data available to read. I then do synchronous reads on those file descriptors. None of those reads should block because I already know data is available, plus I have opened the file descriptors in non-blocking mode.

Asynchronous io is where you issue an io request. That request is queued, and thus doesn't block the issuing thread. You are notified when either the request failed or has completed successfully.

Solution 4:

Non blocking IO is when the call to perform IO returns immediately, and does not block your thread.

The only way to know if the IO is done, is to poll its status or block. Think of it as a Future. You start an IO operation, and it returns you a Future. You can call isDone() on it to check if its done, if it is, do what you want with it, otherwise keep doing other stuff until the next time you want to check if its done. Or, if you're out of things to do, you can call get on it, which will block until its done.

Async IO is when the call to perform IO notifies you it is done through an event, not through its return value.

This can be blocking or non-blocking.

Blocking Async IO

What is meant by blocking async IO is that the call to perform IO is a normal blocking call, but the thing you called wrapped that call inside a thread which will block until the IO is done and then delegate the handling of the result of the IO to your callback. That is, there is still a thread lower down the stack which is blocked on the IO, but your thread isn't.

Non-blocking Async IO

This is actually the more common one, and it means that the non-blocking IO does not need to be polled for its status, as with standard non-blocking IO, instead it will call your callback when its done. As opposed to blocking async IO, this one has no threads blocked anywhere down the stack, thus its faster and uses less resources, as the asynchronous behavior is managed without blocking threads.

You can think of it as a CompletableFuture. It requires that your program has some form of async event framework, which can be multi-threaded or not. So its possible the callback is executed in another thread, or that it is scheduled for execution on an existing thread once the current task is done.

I explain the distinction more thoroughly here.

Solution 5:

Synchronous vs. asynchronous

Asynchronous is a relative term that applies to all kinds of computation, not just IO. Something can not be asynchronous by itself but always to something else. Usually, asynchronicity means that some operation is happening in a different thread of execution relative to the thread that requested the IO computation, and there is no explicit synchronization (waiting) between a requesting and a computing threads. If a requesting thread waits (sleeps, blocks) while the computing thread is doing its work, we call such an operation synchronous. There are also mixed cases. Sometimes a requesting thread doesn't wait immediately and performs some fixed amount of useful work asynchronously after issuing an IO request, but later blocks (synchronizes) to await for the IO results if they are not yet readily available.

Blocking vs. non-blocking

In the broader sense, "blocking" and "non-blocking" can roughly be used to denote "synchronous" and "asynchronous" correspondingly. You will often encounter "blocking" to be used interchangeably with "synchronous" and "non-blocking" with "asynchronous". In this sense, "non-blocking asynchronous" is redundant as other folks mentioned above.

However, in a more narrow sense "blocking" and "non-blocking" may refer to different kernel IO interfaces. It's worth saying here that all IO operations these days are performed by the OS kernel because access to IO hardware devices such as disks or network interface cards is abstracted away by the OS. It means that every IO operation that you request from your userspace code will end up being executed by the kernel via either blocking or non-blocking interface.

When called via the blocking interface, the kernel will assume that your thread wants to obtain results synchronously and will put it to sleep (deschedule, block) until the IO results are available. Therefore that thread will not be able to do any other useful work while the kernel is fulfilling the IO request. As an example, all disk IO on Linux is blocking.

Non-blocking kernel interfaces work differently. You tell the kernel which IO operations you want. The kernel doesn't block (deschedule) your thread and returns from the IO call immediately. Your thread can then move on and do some useful work. Kernel threads will fulfill the IO requests asynchronously. Your code then needs to check occasionally if the kernel has already done its job, after which you can consume the results. As an example, Linux provides the epoll interface for the non-blocking IO. There are also older poll and select system calls for the same purpose. It's worth noting that non-blocking interfaces mostly apply and are used for networking.

Please, note that the fact that some higher-level IO APIs use blocking kernel IO under the hood doesn't mean that your thread will necessarily block when calling that API. Such an API may implement a mechanism to spawn a new or use a different existing thread to perform that blocking IO. It will notify your calling thread later through some means (a callback, an event, or by letting your thread poll) that it has completed the IO request. I.e., non-blocking IO semantics can be implemented in userspace by third-party libraries or runtimes on top of the blocking OS kernel interfaces by using additional threads.

Conclusion

To understand how each particular runtime or library achieves IO asynchronicity, you will have to go deeper and find out if it spawns new threads or relies upon asynchronous kernel interfaces.

Afterword

Realistically, there is very little chance you will encounter genuinely single-threaded systems these days.

As en example, most people will refer to Node.js as having a "single-threaded non-blocking" IO. However, this is a simplification. On Linux, truly non-blocking IO is only available for network operations through the epoll interface. For disk IO, the kernel will always block the calling thread. To achieve asynchronicity for disk IO (which is relatively slow), Node.js runtime (or libuv to be precise) maintains a dedicated thread pool. Whenever an asynchronous disk IO operation is requested, the runtime assigns the work to one of the threads from that pool. That thread will do standard blocking disk IO, while the main (calling) thread will go on asynchronously. Not to mention numerous threads, which are maintained separately by V8 runtime for garbage collection and other managed runtime tasks.