Shared Memory between two JVMs

Solution 1:

Solution 1:

The best solution in my opinion is to use memory mapped files. This allows you to share a region of memory between any number of process, including other non java programs. You can't place java objects into a memory mapped file, unless you serialize them. The following example shows that you can communicate between two different process, but you would need to make it much more sophisticated to allow better communication between the processes. I suggest you look at Java's NIO package, specifically the classes and methods used in the below examples.

Server:

public class Server {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );

        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        char[] string = "Hello client\0".toCharArray();
        charBuf.put( string );

        System.out.println( "Waiting for client." );
        while( charBuf.get( 0 ) != '\0' );
        System.out.println( "Finished waiting." );
    }
}

Client:

public class Client {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );
        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        // Prints 'Hello server'
        char c;
        while( ( c = charBuf.get() ) != 0 ) {
            System.out.print( c );
        }
        System.out.println();

        charBuf.put( 0, '\0' );
    }

}

Solution 2:

Another solution is to use Java Sockets to communicate back and forth between processes. This has the added benefit of allowing communication over a network very easily. It could be argued that this is slower than using memory mapped files, but I do not have any benchmarks to back that statement up. I won't post code to implementing this solution, as it can become very complicated to implement a reliable network protocol and is fairly application specific. There are many good networking sites that can be found with quick searches.

Now the above examples are if you want to share memory between two different process. If you just want to read/write to arbitrary memory in the current process, there are some warnings you should know first. This goes against the entire principle of the JVM and you really really should not do this in production code. You violate all safety and can very easily crash the JVM if you are not very careful.

That being said, it is quite fun to experiment with. To read/write to arbitrary memory in the current process you can use the sun.misc.Unsafe class. This is provided on all JVMs that I am aware of and have used. An example on how to use the class can be found here.

Solution 2:

There are some IPC libraries which facilitate use of shared memory via memory-mapped files in Java.

Chronicle-Queue

Chronicle Queue is similar to a non-blocking Java Queue, except you could offer a message in one JVM and poll it in another JVM.

In both JVMs you should create a ChronicleQueue instance in the same FS directory (locate this directory in a memory-mounted FS if you don't need message persistence):

ChronicleQueue ipc = ChronicleQueueBuilder.single("/dev/shm/queue-ipc").build();

Write a message in one JVM:

ExcerptAppender appender = ipc.acquireAppender();
appender.writeDocument(w -> {
    w.getValueOut().object(message);
});

Read a message in another JVM:

ExcerptTailer tailer = ipc.createTailer();
// If there is no message, the lambda, passed to the readDocument()
// method is not called.
tailer.readDocument(w -> {
    Message message = w.getValueIn().object(Message.class);
    // process the message here
});

// or avoid using lambdas
try (DocumentContext dc = tailer.readingDocument()) {
    if (dc.isPresent()) {
        Message message = dc.wire().getValueIn().object(Message.class);
        // process the message here
    } else {
        // no message
    }
}

Aeron IPC

Aeron is more than just IPC queue (it is a network communication framework), but it provides an IPC functionality as well. It is similar to Chronicle Queue, one important difference is that it uses SBE library for message marshalling/demarshalling, while Chronicle Queue uses Chronicle Wire.

Chronicle Map

Chronicle Map allows IPC communication by some key. In both JVMs, you should create a map with identical configurations and persisted to the same file (the file should be localed in memory-mounted FS if you don't need actual disk persistence, e. g. in /dev/shm/):

Map<Key, Message> ipc = ChronicleMap
    .of(Key.class, Message.class)
    .averageKey(...).averageValue(...).entries(...)
    .createPersistedTo(new File("/dev/shm/jvm-ipc.dat"));

Then in one JVM you could write:

ipc.put(key, message); // publish a message

On the reciever JVM:

Message message = ipc.remove(key);
if (message != null) {
    // process the message here
}

Solution 3:

Distributed_cache is best solution to address your requirements.

In computing, a distributed cache is an extension of the traditional concept of cache used in a single locale. A distributed cache may span multiple servers so that it can grow in size and in transnational capacity.

Few options:

Terracotta allows threads in a cluster of JVMs to interact with each other across JVM boundaries using the same built-in JVM facilities extended to have a cluster-wide meaning

Oracle_Coherence is a proprietary1 Java-based in-memory data grid, designed to have better reliability, scalability and performance than traditional relational database management systems

Ehcache is a widely used open source Java distributed cache for general purpose caching, Java EE and light-weight containers. It features memory and disk stores, replicate by copy and invalidate, listeners, cache loaders, cache extensions, cache exception handlers, a gzip caching servlet filter, RESTful and SOAP APIs

Redis is a data structure server. It is open-source, networked, in-memory, and stores keys with optional durability.

Couchbase_Server is an open-source, distributed (shared-nothing architecture) multi-model NoSQL document-oriented database software package that is optimized for interactive applications. These applications may serve many concurrent users by creating, storing, retrieving, aggregating, manipulating and presenting data.

Useful posts:

What is Terracotta?

Is Terracotta a distributed cache?

infoq article

Solution 4:

Honestly, you don't want to share the same memory. You should send only the data that you need to the other JVM. That being said, in the case you do need the shared memory, other solutions exist.

Sending Data Two JVMs do not share the same memory access points, so it is impossible to use a reference from one JVM to use in another. A new reference will simply be create because they don't know about each other.

However, you may ship the data to the other JVM, and back in a variety of ways:

1) Using RMI, you can setup a remote server to parse data. I found it a bit of a hassle to set up because it requires security changes and that the data be Serializable. You can find out more at the link.

2) Using a server is the age-old method of sending data to different places. One way to implement this is using a ServerSocket and connecting with a Socket on localhost. Objects still need to be Serializable if you want to use ObjectOutputStream.

Sharing Data This is very dangerous and volatile, low-level, and, well, unsafe (literally).

If you want to use Java code, you can take a look at using s.m.Unsafe, using the correct memory addresses, you will be able to retrieve Objects stored by the backing C/C++ arrays in the OS.

Otherwise, you can use native methods to access the C/C++ arrays yourself, although I have no clue how this could be implemented.