Docstoc

Java NIO

Document Sample
Java NIO Powered By Docstoc
					NIO
By Mohit Kumar

13.10.2004

Designed By : Mohit Kumar

Contents
The need for NIO  Limitations of traditional Java I/O  What was missing  What’s new  The makeup of NIO  Buffers  Channels  Selectors  Making use of NIO  Multiplexing  Gotchas  Protoplex  The future of NIO 13.10.2004 Designed By : Mohit Kumar  JSR 203


Why Do We Need NIO?
Efficiency – The Need For Speed  Byte/char-oriented pipelines are flexible but relatively inefficient  The OS provides high-performance I/O services - the JVM gets in the way  Scalability – Livin' Large  Big applications have big appetites, they consume large amounts of data  Traditional method of handling large numbers of I/O streams does not scale  Multiplexing can only be done effectively with OS support  Reliability – Less Code Written = Fewer Bugs  These I/O needs are generic and should be provided by the Java platform  Application programmers should write application code, not 13.10.2004 Designed By : Mohit Kumar infrastructure


Why Do We Need NIO?




No Longer CPU Bound  Moving data has become the bottleneck, not bytecode execution speed JSR 51 (http://www.jcp.org/en/jsr/detail?id=51)  Requested I/O features widely available on most OSs but missing from Java

13.10.2004

Designed By : Mohit Kumar

Should I Stop Using java.io?
 

 

Nope java.nio is not a replacement for java.io  NIO addresses different needs  java.nio does not re-implement java.io java.io is not going away NIO is not the right tool for every job

13.10.2004

Designed By : Mohit Kumar

How quick is NIO?

13.10.2004

Designed By : Mohit Kumar

How quick is NIO?

13.10.2004

Designed By : Mohit Kumar

How quick is NIO?

13.10.2004

Designed By : Mohit Kumar

How quick is NIO?

13.10.2004

Designed By : Mohit Kumar

How quick is NIO?
1. The max throughput achieved by C Netpipe driver is ~ 900 Mbits/s The max throughput achieved by Java Netpipe driver is ~ 680 Mbits/s The Java driver should be right up with the C driver! Java on Gigabit Ethernet

2.

3.

13.10.2004

Designed By : Mohit Kumar

This is ridiculous!!!!

1.

mpjdev (represented by red line) is communicating through sockets (the time is dictated by memory bus bandwidth) ‘native mpjdev’ is communicating through shared memory

2.

13.10.2004

Designed By : Mohit Kumar

How is NIO so quick?




The traditional I/O package of Java is an blocking API:  A separate thread to handle each socket connection. Java New I/O package:  Adds non-blocking I/O to the Java language:  C select () like functionality.  Direct Buffers:  Conventional Java objects are allocated on JVM heap,  Unlike conventional Java objects, direct buffers are allocated in the native OS memory,  Provides faster I/O, not subject to garbage collection.  (explain all this in a while)
Designed By : Mohit Kumar

13.10.2004

What makes up NIO?






13.10.2004

New Abstractions • Buffers • Channels • Selectors New I/O Capabilities • Non-Blocking Sockets • Readiness Selection • File Locking • Memory Mapped Files New Non-I/O Features • Regular Expressions • Pluggable Charset Transcoders

Designed By : Mohit Kumar

What makes up NIO?








13.10.2004


Move large amounts of data efficiently  NIO is primarily block oriented – java.io uses streams  Direct buffers to do raw, overlapped I/O – bypassing the JVM Multiplex large numbers of open sockets  NIO sockets can operate in non-blocking mode  One thread can manage huge numbers of socket channels  Better resource utilization Use OS-level file locking or memory mapping  Locking: Integration with legacy/non-Java applications  Mapped Files: Non-traditional I/O model - leverages virtual memory Do custom character set Transcoding
Designed By : Mohit Kumar

What makes up NIO?










Buffers  Data container objects Channels  Transfer data between buffers and I/O services Selectors  Provide status information about channels Regular Expressions  Perform pattern matching against character sequences Character Set Coding  Perform encoding/decoding of character sequences to/from byte streams
Designed By : Mohit Kumar

13.10.2004

Why is Traditional IO Slower?
 

Trdaditional IO required multiple copies for various reasons. To facilitate GC mechanism and to do IO in OS independent way.

13.10.2004

Designed By : Mohit Kumar

Why is Traditional IO Slower?




 

Processes perform I/O by requesting of the operating system that data be drained from a buffer (write) or that a buffer be filled with data (read). That's really all it boils down to. All data moves in or out of a process by this mechanism. The machinery inside the operating system that performs these transfers can be incredibly complex, but conceptually, it's very straightforward. The process requests that its buffer be filled by making the read( ) system call. This results in the kernel issuing a command to the disk controller hardware to fetch the data from disk. The disk controller writes the data directly into a kernel memory buffer by DMA without further assistance from the main CPU. Once the disk controller finishes filling the buffer, the kernel copies the data from the temporary buffer in kernel space to the buffer specified by the process when it requested the read( ) operation.
Designed By : Mohit Kumar

13.10.2004

Why is Traditional IO Slower?




When a process requests an I/O operation, it performs a system call, sometimes known as a trap, which transfers control into the kernel. The low-level open( ), read( ), write( ), and close( ) functions so familiar to C/C++ coders do nothing more than set up and perform the appropriate system calls. When the kernel is called in this way, it takes whatever steps are necessary to find the data the process is requesting and transfer it into the specified buffer in user space. The kernel tries to cache and/or prefetch data, so the data being requested by the process may already be available in kernel space. If so, the data requested by the process is copied out. If the data isn't available, the process is suspended while the kernel goes about bringing the data into memory.
Designed By : Mohit Kumar

13.10.2004

Why is Traditional IO Slower?


 



it's probably occurred to you that copying from kernel space to the final user buffer seems like extra work. Why not tell the disk controller to send it directly to the buffer in user space? There are a couple of problems with this. First, hardware is usually not able to access user space directly.' Second, block-oriented hardware devices such as disk controllers operate on fixed-size data blocks. The user process may be requesting an oddly sized or misaligned chunk of data. The kernel plays the role of intermediary, breaking down and reassembling data as it moves between user space and storage devices.

13.10.2004

Designed By : Mohit Kumar

Why is Traditional IO Slower?
    

Only the base set of techniques were used that would be supported by all OSes. So IO lost out on the newer techniques like scatter-gather, DMA, Memory mapping through virtual memory. Also in the traditional techniques the buffers were created in the JVM heap. NIO was on the cards as more and more OSes adopted these techniques. IO was designed as a API that can address the base set of techniques and that almost always boiled down to sequential data transfer..slow but easy….

13.10.2004

Designed By : Mohit Kumar

Why is NIO so quick?
 

For starters it uses the techniques described earlier. Scatter Gather (also known as vectored IO)  Many operating systems can make the assembly/disassembly process even more efficient. The notion of scatter/gather allows a process to pass a list of buffer addresses to the operating system in one system call. The kernel can then fill or drain the multiple buffers in sequence, scattering the data to multiple user space buffers on a read, or gathering from several buffers on a write  This saves the user process from making several system calls (which can be expensive) and allows the kernel to optimize handling of the data because it has information about the total transfer. If multiple CPUs are available, it may even be possible to fill or drain several buffers simultaneously.
Designed By : Mohit Kumar

13.10.2004

Scatter Gather

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files






OS(Linux and other as well) processes execute in a virtual environment that makes it appear as if each process had the entire address space of the CPU available to itself. This virtual address space extends from address 0 all the way to the maximum address. On a 32-bit platform, such as IA-32, the maximum address is 232 −1 or 0xffffffff. On a 64-bit platform, such as IA-64, this is 264−1 or 0xffffffffffffffff. While it is obviously convenient for a process to be able to access such a huge address space, there are really three distinct, but equally important, reasons for using virtual memory.

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files
1. Resource virtualization. On a system with virtual memory, a process does not have to concern itself with the details of how much physical memory is available or which physical memory locations are already in use by some other process. In other words, virtual memory takes a limited physical resource (physical memory) and turns it into an infinite, or at least an abundant, resource (virtual memory).  2. Information isolation. Because each process runs in its own address space, it is not possible for one process to read data that belongs to another process. This improves security because it reduces the risk of one process being able to spy on another process and, e.g., steal a password.  3. Fault isolation. Processes with their own virtual address spaces cannot overwrite each other’s memory. This greatly reduces the risk of a failure in one process triggering a failure in another process. That is, when a process crashes, the problem is generally limited to that process alone and does Designed By :the entire machine to go down. not cause Mohit Kumar 13.10.2004


Virtual Memory and memory Mapped Files


The basic Idea

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files




All modem operating systems make use of virtual memory. Virtual memory means that artificial, or virtual, addresses are used in place of physical (hardware RAM) memory addresses. This provides many advantages, which fall into two basic categories:  1. More than one virtual address can refer to the same physical memory location.  2. A virtual memory space can be larger than the actual hardware memory available. The previous section said that device controllers cannot do DMA directly into user space, but the same effect is achievable by exploiting item 1 above. By mapping a kernel space address to the same physical address as a virtual address in user space, the DMA hardware (which can access only physical memory addresses) can fill a buffer that is simultaneously visible to both the kernel and a user space process.
Designed By : Mohit Kumar

13.10.2004

Virtual Memory and memory Mapped Files




This is great because it eliminates copies between kernel and user space, but requires the kernel and user buffers to share the same page alignment. Buffers must also be a multiple of the block size used by the disk controller (usually 512 byte disk sectors). Operating systems divide their memory address spaces into pages, which are fixedsize groups of bytes. These memory pages are always multiples of the disk block size and are usually powers of 2 (which simplifies addressing). Typical memory page sizes are 1,024, 2,048, and 4,096 bytes. The virtual and physical memory page sizes are always the same.

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files


A Simple Picture

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files


Memory Paging  To support the second attribute of virtual memory (having an addressable space larger than physical memory), it's necessary to do virtual memory paging (often referred to as swapping, though true swapping is done at the process level, not the page level).  This is a scheme whereby the pages of a virtual memory space can be persisted to external disk storage to make room in physical memory for other virtual pages. Essentially, physical memory acts as a cache for a paging area, which is the space on disk where the content of memory pages is stored when forced out of physical memory.

13.10.2004

Designed By : Mohit Kumar

Virtual Memory and memory Mapped Files

13.10.2004

Designed By : Mohit Kumar

Page Fault Handling
 



if there is no mapping currently in effect between that virtual page and a physical memory page, the MMU raises a page fault to the CPU. A page fault results in a trap, similar to a system call, which vectors control into the kernel along with information about which virtual address caused the fault. The kernel then takes steps to validate the page. The kernel will schedule a pagein operation to read the content of the missing page back into physical memory. This often results in another page being stolen to make room for the incoming page. In such a case, if the stolen page is dirty (changed since its creation or last pagein) a pageout must first be done to copy the stolen page content to the paging area on disk.

13.10.2004

Designed By : Mohit Kumar

Page Fault Handling






If the requested address is not a valid virtual memory address (it doesn't belong to any of the memory segments of the executing process), the page cannot be validated, and a segmentation fault is generated. This vectors control to another part of the kernel and usually results in the process being killed. Once the faulted page has been made valid, the MMU is updated to establish the new virtual-to-physical mapping (and if necessary, break the mapping of the stolen page), and the user process is allowed to resume. The process causing the page fault will not be aware of any of this; it all happens transparently. This dynamic shuffling of memory pages based on usage is known as demand paging. Some sophisticated algorithms exist in the kernel to optimize this process and to prevent thrashing, a pathological condition in which paging demands become so great that nothing else can get done.
Designed By : Mohit Kumar

13.10.2004

Virtual Memory and memory Mapped Files

13.10.2004

Designed By : Mohit Kumar

Memory Mapped Files:
 

Memory Mapped Files: For conventional file 1/0, in which user processes issue read( ) and write( ) system calls to transfer data, there is almost always one or more copy operations to move the data between these filesystem pages in kernel space and a memory area in user space. This is because there is not usually a one-to-one alignment between filesystem pages and user buffers. There is, however, a special type of I/O operation supported by most operating systems that allows user processes to take maximum advantage of the page-oriented nature of system I/O and completely avoid buffer copies.

13.10.2004

Designed By : Mohit Kumar

Memory Mapped Files:


Memory-mapped 1/0 uses the filesystem to establish a virtual memory mapping from user space directly to the applicable filesystem pages. This has several advantages: • The user process sees the file data as memory, so there is no need to issue read( ) or write( ) system calls. • As the user process touches the mapped memory space, page faults will be generated automatically to bring in the file data from disk. If the user modifies the mapped memory space, the affected page is automatically marked as dirty and will be subsequently flushed to disk to update the file. • The virtual memory subsystem of the operating system will perform intelligent caching of the pages, automatically managing memory according to system load.
Designed By : Mohit Kumar

13.10.2004

Memory Mapped Files:

13.10.2004

Designed By : Mohit Kumar

Memory Mapped Files:




The data is always page-aligned, and no buffer copying is ever needed. • Very large files can be mapped without consuming large amounts of memory to copy the data. Virtual memory and disk I/O are intimately linked and, in many respects, are simply two aspects of the same thing. Keep this in mind when handling large amounts of data. Most operating systems are far more efficient when handling data buffers that are page-aligned and are multiples of the native page size.

13.10.2004

Designed By : Mohit Kumar

Stream IO
 



Stream I/0 Not all I/O is block-oriented, as described in previous sections. There is also stream IO, which is modeled on a pipeline. The bytes of an IO stream must be accessed sequentially. TTY (console) devices, printer ports, and network connections are common examples of streams. Streams are generally, but not necessarily, slower than block devices and are often the source of intermittent input. Most operating systems allow streams to be placed into nonblocking mode, which permits a process to check if input is available on the stream without getting stuck if none is available at the moment. Such a capability allows a process to handle input as it arrives but perform other functions while the input stream is idle.
Designed By : Mohit Kumar

13.10.2004

Stream IO




A step beyond nonblocking mode is the ability to do readiness selection. This is similar to nonblocking mode (and is often built on top of nonblocking mode), but offloads the checking of whether a stream is ready to the operating system. The operating system can be told to watch a collection of streams and return an indication to the process of which of those streams are ready. This ability permits a process to multiplex many active streams using common code and a single thread by leveraging the readiness information returned by the operating system. This is widely used in network servers to handle large numbers of network connections. Readiness selection is essential for high-volume scaling.

13.10.2004

Designed By : Mohit Kumar

NIO Contructs
  

Buffers Channels Selectors

13.10.2004

Designed By : Mohit Kumar

13.10.2004

Designed By : Mohit Kumar

NIO Buffers






 

13.10.2004

Fixed size containers of primitive data types  ByteBuffer, CharBuffer, FloatBuffer, etc. Byte buffers are special  Used with channels to perform I/O  Can provide alternate views of contained data Direct and Non-direct ByteBuffers  Direct ByteBuffers address native memory  OS-level I/O accesses direct buffer content directly Buffers can be views of other buffers or wrap arrays Byte order (endian-ness)  Settable for ByteBuffers  Immutable for non-ByteBuffer objects
Designed By : Mohit Kumar

Buffer Classes

13.10.2004

Designed By : Mohit Kumar

Buffer Objects (Empty/Fill)

13.10.2004

Designed By : Mohit Kumar

Buffer Objects (Flip)

13.10.2004

Designed By : Mohit Kumar

Buffer Views (Dupe/Slice)

13.10.2004

Designed By : Mohit Kumar

Buffer Views (Char View)

13.10.2004

Designed By : Mohit Kumar

Buffer Compact

13.10.2004

Designed By : Mohit Kumar

Buffer Marking

13.10.2004

Designed By : Mohit Kumar

Byte Buffer
public abstract class ByteBuffer extends Buffer implements Comparable { public static ByteBuffer allocate (int capacity) public static ByteBuffer allocateDirect (int capacity) public abstract Boolean isDirect(); public static ByteBuffer wrap (byte() array, int offset, int length) public static ByteBuffer wrap (byte() array) public abstract ByteBuffer duplicate(); public abstract ByteBuffer asReadonly8uffer(); public abstract ByteBuffer slice(); public final Boolean hasArray() public final byte [] array() public final int arrayOffset() public abstract byte get(); public abstract byte get (int index); public ByteBuffer get (byte[] dst, int offset, int length) public 13.10.2004 ByteBuffer get (byte[] dst, int By : Mohit Kumar Designed offset, int length)

Byte Buffer
public abstract ByteBuffer put (byte b); public abstract ByteBuffer put (int index, byte b); public ByteBuffer put (ByteBuffer src) public ByteBuffer put (byte[] src, int offset, int length) public final ByteBuffer put (byte[] src) public final ByteOrder order() public final ByteBuffer order (ByteOrder bo) public abstract CharBuffer asCharBuffer(); public abstract ShortBuffer asShortBuffer(); public abstract IntBuffer asIntBuffer(); public abstract LongBuffer aslongBuffer(); public abstract FloatBuffer asFloatBuffer(); public abstract DoubleBuffer asDoubleBuffer();
13.10.2004 Designed By : Mohit Kumar

Byte Buffer
public abstract char getChar(); public abstract char getChar (int index); public abstract ByteBuffer putChar (char value); public abstract ByteBuffer putChar (int index, char value); public abstract short getShort(); public abstract short getShort (int index); public abstract ByteBuffer putShort (short value); public abstract ByteBuffer putShort (int index, short value); public abstract int getInt(); public abstract int getInt (int index); public abstract ByteBuffer putInt (int value); public abstract ByteBuffer putInt (int index, int value);

13.10.2004

Designed By : Mohit Kumar

Byte Buffer
public abstract long getLong(); public abstract long getLong (int index); public abstract ByteBuffer putlong (long value); public abstract ByteBuffer putlong (int index, long value); public abstract float getFloat(); public abstract float getFloat (int index); public abstract ByteBuffer putFloat (float value); public abstract ByteBuffer putFloat (int index, float value); public abstract double getDouble(); public abstract double getDouble (int index); public abstract ByteBuffer putDouble (double value); public abstract ByteBuffer putDouble (int index, double value);

13.10.2004

Designed By : Mohit Kumar

Byte Buffer
public abstract ByteBuffer compact(); public Boolean equals (Object ob); public int compareTo (Object ob); public String toString(); public int hashCode(); }

13.10.2004

Designed By : Mohit Kumar

NIO Channels






New I/O metaphor  Conduit to an I/O service (“nexus”)  Channels are not streams Channels do bulk data transfers to and from buffers  channel.write (buffer) ~= buffer.get (byteArray)  channel.read (buffer) ~= buffer.put (byteArray)  Data is logically transferred in blocks, not byte/char at a time More capable APIs  Scatter/gather  Channel-to-channel transfers

13.10.2004

Designed By : Mohit Kumar

NIO Channels




Three primary channel implementations  FileChannel: File locks, memory mapping, cross-connect transfers  Sockets: Non-blocking, selectable, async connections, socket peers  Pipe: loopback channel pair, selectable, generic channels Selectable Channel Implementations are pluggable (SPI)  Specialized implementations are possible

13.10.2004

Designed By : Mohit Kumar

File Channels
 







They address the scatter/gather, file locks, and memory mapped files File channels are always blocking and cannot be placed into nonblocking mode. Modem operating systems have sophisticated caching and prefetch algorithms that usually give local disk I/O very low latency. Network filesystems generally have higher latencies but often benefit from the same optimizations. The nonblocking paradigm of stream-oriented I/O doesn't make as much sense for file-oriented operations because of the fundamentally different nature of file I/O. For file I/O, the to winner is asynchronous 1/O, which lets a process request one or more I/O operation from the operating system but does not wait for them to complete. The process notified at a later time that the requested I/O has completed. Asynchronous I/O is advanced capability not available on many operating systems. It is under consideration as a future NIO enhancement.

13.10.2004

Designed By : Mohit Kumar

File Channels

13.10.2004

Designed By : Mohit Kumar

File Channels


Creating Channels
FileChannel fc = new RandomAccessFile( new File("temp.tmp"), "rw").getChannel(); //or FileOutputStream fos = new FileOutputStream (DEMOGRAPHIC); GatheringByteChannel gatherChannel = fos.getChannel(); //or FileInputStream fos = new FileInputStream (DEMOGRAPHIC); GatheringByteChannel gatherChannel = fos.getChannel();

13.10.2004

Designed By : Mohit Kumar

File Channels


Scatter/Gather with file

13.10.2004

Designed By : Mohit Kumar

File Channels
FileOutputStream fos = new FileOutputStream (DEMOGRAPHIC); GatheringByteChannel gatherChannel = fos.getChannel(); // generate some brilliant marcom, er, repurposed content ByteBuffer [] bs = utterBS (reps); // deliver the message to the waiting market while (gatherChannel.write (bs) > 0) { // empty body // loop until write() returns zero } System.out.println ("Mindshare paradigms synergized to " + DEMOGRAPHIC); fos.close();
13.10.2004 Designed By : Mohit Kumar

File Channels
 

Scatter/Gather Most modern operating systems support native vectored 1/0. When you request a scatter/gather operation on a channel, the request will be translated into appropriate native calls to fill or drain the buffers directly. This is a big win, because buffer copies and system calls are reduced or eliminated. Scatter/gather should be used with direct ByteBuffers to gain the greatest advantage from native I/O, especially if the buffers are long-lived.

13.10.2004

Designed By : Mohit Kumar

File Channels
 

Memory-Mapped Files The new FileChannel class provides a method, map( ), that establishes a virtual mem ory mapping between an open file and a special type of ByteBuffer. Calling map( ) on a FileChannel creates a virtual memory mapping, backed by a disk file and wraps a MappedByteBuffer object around that virtual memory space. The MappedByteBuffer object returned from map( ) behaves like a memory-base buffer in most respects, but its data elements are stored in a file on disk. Calling get will fetch data from the disk file, and this data reflects the current content of the file even if the file has been modified by an external process since the mapping was established.
Designed By : Mohit Kumar



13.10.2004

File Channels






The data visible through a file mapping is exactly the same as you would se by reading the file conventionally. Likewise, doing a put( ) to the mapped buffer will update the file on disk (assuming you have write permission), and your changes will be visible to other readers of the file. Accessing a file through the memory-mapping mechanism can be far more efficient than reading or writing data by conventional means, even when using channels. No explicit system calls need to be made, which can be time-consuming. More importantly, the virtual memory system of the operating system automatically cache memory pages. These pages will be cached using system memory and will not consume space from the JVM's memory heap.
Designed By : Mohit Kumar

13.10.2004

File Channels




Once a memory page has been made valid (brought in from disk), it can be accessed again at full hardware speed(as if it is an array in the memory) without the need to make another system call to get the data. Large, structured files that contain indexes or other sections that are referenced or updated frequently can benefit tremendously from memory mapping When combined with file locking to protect critical sections and control transactional atomicity, you begin to see how memory mapped buffers can be put to goo use.

13.10.2004

Designed By : Mohit Kumar

File Channels
RandomAccessFile file = new RandomAccessFile (tempFile, "rw"); FileChannel channel = file.getChannel(); MappedByteBuffer ro = channel.map (FileChannel.MapMode.READ_ONLY, 0, channel.size());

13.10.2004

Designed By : Mohit Kumar

File Channels


Mapped buffers are used as if they were just like other buffers, which is how you would use them most of the time. But MappedByteBuffer also defines a few unique methods of its own:

public abstract class MappedByteBuffer extends ByteBuffer { // This is a partial API listing public final MappedByteBuffer load() public final Boolean isloaded() public final MappedByteBuffer force() }

13.10.2004

Designed By : Mohit Kumar

File Channels


Channel-to-channel  The FileChannel class adds certain method to do channel to channel transfer. This would mean that there would be no need to transfer data to an intermediate buffer  The transferTo() and transferFrom() method allow you to cross connect two channels. You cant do direct transfer between two socket channels. But a socket Channel can be connected to a file channel.

13.10.2004

Designed By : Mohit Kumar

Socket Channels


 



Let's move on to the channel classes that model network sockets. Socket channels have different characteristics than file channels. The new socket channels can operate in nonblocking mode and are selectable. These two capabilities enable tremendous scalability and flexibility in large applications, such as web servers and middleware components. It's no longer necessary to dedicate a thread to each socket connection (and suffer the contextswitching overhead of managing large numbers of threads). Using the new NIO classes, one or a few threads can manage hundreds or even thousands of active socket connections with little or no performance loss.
Designed By : Mohit Kumar

13.10.2004

Socket Channels


Picture

13.10.2004

Designed By : Mohit Kumar

Non-Blocking Sockets – Simple Really

ByteBuffer buffer = ByteBuffer.allocate (1024); SocketChannel socketChannel = SocketChannel.open(); socketChannel.configureBlocking (false); ... while (true) { ... if (socketChannel.read (buffer) != 0) { processInput (buffer); } ... }

13.10.2004

Designed By : Mohit Kumar

Non-Blocking Server Socket
ServerSocketChannel ssc = ServerSocketChannel.open(); ssc.socket().bind (new InetSocketAddress (port)); ssc.configureBlocking (false); while (true) { SocketChannel newConnection = ssc.accept(); if (newConnection == null) { doSomethingToKeepBusy(); } else { doSomethingWithSocket (newConnection); }

}

13.10.2004

Designed By : Mohit Kumar

NIO Selectors
Multiplexing Channels – Readiness Selection  Channels are registered with Selectors  A SelectionKey object encapsulates one selector/channel relationship  Each Selector maintains a set of channels registered with it (Registered Set)  A subset of ready channels is selected from the Selector's Registered Set of channels (Selector.select())  Selected Set returned by select() contains keys with non-empty Ready Sets  Each SelectionKey has Interest & Ready Sets  Possible members of Interest Set: accept, read, write, connect  Interest Set (bit mask) is specified at registration, can be changed at any

13.10.2004

time

Designed By : Mohit Kumar

NIO Selectors


Ready Set is a subset of the Interest Set as-of the last select() call



Readiness Selection means less work  Ignore idle channels  Sockets are decoupled from threads  Less synchronization required  Far simpler code - less debugging, more reliable

13.10.2004

Designed By : Mohit Kumar

Selectors, Keys and Channels

13.10.2004

Designed By : Mohit Kumar

Selectors, Keys and Channels

13.10.2004

Designed By : Mohit Kumar

Multiplexing: The Selection Process
Create a Selector and register channels with it  The register() method is on SelectableChannel, not Selector  Invoke select() on the Selector object  Fetch the Selected Set of keys from the Selector  Selected set: Registered keys with non-empty Ready Sets  keys = selector.selectedKeys()  Iterate over the Selected Set  Check each key's Ready Set (set of operations ready to go as-of last select())  Remove the key from the Selected Set (iterator.remove())  Bits in the Ready Sets are never reset while the key is in the Selected Set

13.10.2004 Designed By : Mohit Kumar

Multiplexing: The Selection Process


The Selector never removes keys from the Selected Set – you must do so  Service the channel (key.channel()) as appropriate (read, write, etc)

13.10.2004

Designed By : Mohit Kumar

Multiplexing: The Selection Process


A selection operation is performed by a selector when one of the three forms of select( ) is invoked. Whichever is called, the following three steps are performed:  1. The cancelled key set is checked. If it's nonempty, each key in the cancelled set is removed from the other two sets, and the channel associated with the cancelled key is deregistered. When this step is complete, the cancelled key set is empty.  2. The operation interest sets of each key in the registered key set are examined. Changes made to the interest sets after they've been examined in this step will not be seen during the remainder of the selection operation.

13.10.2004

Designed By : Mohit Kumar

Multiplexing: The Selection Process




Once readiness criteria have been determined, the underlying operating system is queried to determine the actual readiness state of each channel for its operations of interest. Depending on the specific select( ) method called, the thread may block at this point if no channels are currently ready, possibly with a timeout value. Upon completion of the system calls, which may have caused the invoking thread to be put to sleep for a while, the current readiness status of each channel will have been determined. Nothing further happens to any channel not found to be currently ready. For each channel that the operating system indicates is ready for at least one of the operations in its interest set, one of the following two things happens:  a. If the key for the channel is not already in the selected key set, the key's
Designed By : Mohit Kumar

13.10.2004

Multiplexing: The Selection Process
 



ready set is cleared, and the bits representing the operations determined to . be currently ready on the channel are set. b. Otherwise, the key is already in the selected key set. The key's ready set is updated by setting bits representing the operations found to be currently ready. Any previously set bits representing operations that are no longer ready are not cleared. In fact, no bits are cleared. The ready set as determined by the operating system is bitwise-disjoined into the previous ready set.' Once a key has been placed in the selected key set of the selector, its ready set is cumulative. Bits are set but never cleared. 3. Step 2 can potentially take a long time, especially if the invoking thread sleeps. Keys associated with this selector could have been cancelled in the meantime.

13.10.2004

Designed By : Mohit Kumar

Registering With a Selector
ServerSocketChannel serverChannel = ServerSocketChannel.open(); Selector selector = Selector.open(); serverChannel.socket().bind (new InetSocketAddress (port)); serverChannel.configureBlocking (false); serverChannel.register (selector, SelectionKey.OP_ACCEPT);

13.10.2004

Designed By : Mohit Kumar

Running a Selection Loop
while (true) { selector.select(); Iterator it = selector.selectedKeys().iterator(); while (it.hasNext()) { SelectionKey key = (SelectionKey) it.next(); it.remove(); if (key.isAcceptable()) { ServerSocketChannel server = (ServerSocketChannel) key.channel(); SocketChannel channel = server.accept(); channel.configureBlocking (false); channel.register (selector, SelectionKey.OP_READ); } } if (key.isReadable()) readDataFromSocket (key);
Designed By : Mohit Kumar

13.10.2004

}

Scalability With Selectors
One Thread to Rule Them All  More threads != More Efficient  More threads means more context switching overhead  Actual concurrency is limited by the number of CPUs available  The OS and/or JVM do the hard work for you  Only the kernel can efficiently do Readiness Selection  No need to write and debug tricky socket management code  No more thread-per-socket nonsense  Simpler, easier to maintain code  Less concurrency hassles – locking overhead, thread races  Single point of dispatch

13.10.2004 Designed By : Mohit Kumar

Scalability With Selectors


Not necessarily single-threaded  Single selection thread can dispatch to multiple worker threads  You can have multiple selection dispatchers managing distinct sets of sockets

13.10.2004

Designed By : Mohit Kumar

Protoplex: NIO Made Easy
 







13.10.2004


Protocol-Independent Channel Multiplexer Open Source  Available soon on SourceForge  Watch http://javanio.info for availability (or contact me) Handles all details of channel management  Queues outbound data  Manages Interest Sets appropriately  Manages both inbound and outbound connections Multi-thread friendly  Multiple acceptor threads per server socket can be configured  Configurable worker thread pool  Uses Doug Lea’s concurrency package Application implements the Protocol interface
Designed By : Mohit Kumar

Design Using NIO


Classic Service Design

13.10.2004

Designed By : Mohit Kumar

Classic Service Design
class Server implements Runnable {
public void run() { try { ServerSocket ss = new ServerSocket(PORT); while (!Thread.interrupted()) new Thread(new Handler(ss.accept())).start(); // or, single-threaded, or a thread pool } catch (IOException ex) { /* ... */ } } static class Handler implements Runnable { final Socket socket; Handler(Socket s) { socket = s; } public void run() { try { byte[] input = new byte[MAX_INPUT]; socket.getInputStream().read(input); byte[] output = process(input); socket.getOutputStream().write(output); } catch (IOException ex) {Designed By : Mohit Kumar /* ... */ } 13.10.2004 }

Design with NIO




Scalability Goals  " Graceful degradation under increasing load(more clients)  " Continuous improvement with increasing resources (CPU, memory, disk, bandwidth)  " Also meet availability and performance goals  Short latencies  Meeting peak demand  Tunable quality of service  " Divide-and-conquer is usually the best approach for achieving any scalability goal Divide and Conquer  " Divide processing into small tasks  Each task performs an action without blocking
Designed By : Mohit Kumar

13.10.2004

Design with NIO
" Execute each task when it is enabled  Here, an IO event usually serves as trigger  " Basic mechanisms supported in java.nio  Non-blocking reads and writes  Dispatch tasks associated with sensed IO events  " Endless variation possible Event-driven Designs  " Usually more efficient than alternatives  Fewer resources  " Don't usually need a thread per client  Less overhead  " Less context switching, often less locking




13.10.2004

Designed By : Mohit Kumar

Design with NIO




 

But dispatching can be slower  " Must manually bind actions to events " Usually harder to program  Must break up into simple non-blocking actions  " Similar to GUI event-driven actions " Cannot eliminate all blocking: GC, page faults, etc Must keep track of logical state of service

13.10.2004

Designed By : Mohit Kumar

Design with NIO


Basic Reactor Design

13.10.2004

Designed By : Mohit Kumar

Design with NIO
class Reactor implements Runnable { final Selector selector; final ServerSocketChannel serverSocket; Reactor(int port) throws IOException { selector = Selector.open(); serverSocket = ServerSocketChannel.open(); serverSocket.socket().bind(new InetSocketAddress(port)); serverSocket.configureBlocking(false); SelectionKey sk =serverSocket.register(selector, SelectionKey.OP_ACCEPT, new Acceptor()); } /* Alternatively, use explicit SPI provider: SelectorProvider p = SelectorProvider.provider(); selector = p.openSelector(); serverSocket = p.openServerSocketChannel(); */ 13.10.2004 Designed By : Mohit Kumar

Design with NIO
// class Reactor continued public void run() { // normally in a new Thread try { while (!Thread.interrupted()) { selector.select(); Set selected = selector.selectedKeys(); Iterator it = selected.iterator(); while (it.hasNext()) dispatch((SelectionKey)(it.next()); selected.clear(); } } catch (IOException ex) { /* ... */ } } void dispatch(SelectionKey k) { Runnable r = (Runnable)(k.attachment()); if (r != null) 13.10.2004 r.run(); Designed By : Mohit Kumar

Design with NIO
// class Reactor continued class Acceptor implements Runnable { // inner public void run() { try { SocketChannel c = serverSocket.accept(); if (c != null) new Handler(selector, c); } catch(IOException ex) { /* ... */ } } } }

13.10.2004

Designed By : Mohit Kumar

Design with NIO

13.10.2004

Designed By : Mohit Kumar

Design with NIO
final class Handler implements Runnable { final SocketChannel socket; final SelectionKey sk; ByteBuffer input = ByteBuffer.allocate(MAXIN); ByteBuffer output = ByteBuffer.allocate(MAXOUT); static final int READING = 0, SENDING = 1; int state = READING; Handler(Selector sel, SocketChannel c)throws IOException { socket = c; c.configureBlocking(false); // Optionally try first read now sk = socket.register(sel, 0); sk.attach(this); sk.interestOps(SelectionKey.OP_READ); sel.wakeup(); } boolean inputIsComplete() { /* ... */ } boolean 13.10.2004 outputIsComplete() { /* ... */ }By : Mohit Kumar Designed

Design with NIO
void process() { /* ... */ } // class Handler continued public void run() { try { if (state == READING) read(); else if (state == SENDING) send(); } catch (IOException ex) { /* ... */ } } void read() throws IOException { socket.read(input); if (inputIsComplete()) { process(); state = SENDING; // Normally not required sk.interestOps(SelectionKey.OP_WRITE); } } 13.10.2004 Designed By : Mohit Kumar

Design with NIO
void send() throws IOException { socket.write(output); if (outputIsComplete()) sk.cancel(); } }

13.10.2004

Designed By : Mohit Kumar

Design with NIO


Multithreaded Designs  " Strategically add threads for scalability  Mainly applicable to multiprocessors  " Worker Threads  Reactors should quickly trigger handlers  " Handler processing slows down Reactor  Offload non-IO processing to other threads  " Multiple Reactor Threads  Reactor threads can saturate doing IO  Distribute load to other reactors  " Load-balance to match CPU and IO rates
Designed By : Mohit Kumar

13.10.2004

Design with NIO


Worker Threads  " Offload non-IO processing to speed up  Reactor thread  Similar to POSA2 Proactor designs  " Simpler than reworking compute-bound processing into eventdriven form  Should still be pure nonblocking computation  " Enough processing to outweigh overhead  " But harder to overlap processing with IO  Best when can first read all input into a buffer  " Use thread pool so can tune and control  Normally need many fewer threads than clients
Designed By : Mohit Kumar

13.10.2004

Design with NIO


Multi Threaded Design

13.10.2004

Designed By : Mohit Kumar

Design with NIO


Using Multiple Reactors

13.10.2004

Designed By : Mohit Kumar

Gotcha: Things To Watch Out For






Multithreaded access to Selectors  select() holds a lock while it’s sleeping  Causes calls to methods on SelectionKey to block  More stringent locking in JDK 1.4.2 Empty channels are always ready to write  Registering interest in write may cause busy spinning  Turn off write interest when you have nothing to write Don’t memory map beyond the end of a file  On some OSs, this may expand the size of the file  OK to lock beyond the end of a file

13.10.2004

Designed By : Mohit Kumar

Gotcha: Things To Watch Out For


In general, NIO is more efficient for reading and writing large chunks of data  No buffering is done, each call incurs system call overhead

13.10.2004

Designed By : Mohit Kumar

What Did They Leave Out?






Formatted I/O (ala printf/scanf) • Will leverage Regular Expressions Enhanced Filesystem Interface • More consistent across OS platforms • Better access to file/directory attributes • Pluggable to support new filesystem types True Asynchronous I/O • Under consideration • Very hard to do across all OS platforms

13.10.2004

Designed By : Mohit Kumar

What’s Next for NIO?




 

JSR 203 - More NIO  Originally planned as part of JDK 1.5 (Tiger)  Deferred because of other priorities - 1.6? Enhanced filesystem interface  Bulk access to file attributes  Escape to filesystem-specific APIs  SPI for pluggable filesystem API implementations True Asynchronous I/O  Async data transfer for both socket and file I/O Extend the SocketChannel APIs  Bind and configure sockets directly (not through Socket peer)  Support for multi-cast
13.10.2004 Designed By : Mohit Kumar

What’s Next for NIO?


Formatted I/O?  Demonstrated at JavaOne 2003  Not mentioned in JSR 203

13.10.2004

Designed By : Mohit Kumar

Final Thoughts
 





NIO is not for all I/O uses Some things can only be done with NIO  Use it when you need it (when you need it, you’ll know)  Don’t change code that’s working fine without it NIO is still maturing  Implementation is complex  API is not as elegant as it could be  Performance still needs improvement  Expect improvements in future releases Contact me if you still have questions

13.10.2004

Designed By : Mohit Kumar


				
DOCUMENT INFO
Shared By:
Tags: java, Input, Output
Stats:
views:738
posted:9/19/2009
language:English
pages:109
Description:  java.nio is not a replacement for java.io  NIO addresses different needs  java.nio does not re-implement java.io  java.io is not going away  NIO is not the right tool for every job
Vinothkumar Vinothkumar Engineer
About