Does the native linker change the status-quo for JVM low latency?

This article discusses if the Java 22 foreign memory API and value classes change my view of the status quo for building applications that communicate with C++ components and the possible latencies in that approach.

In the past, with JNI as the C++ go between, there were enough disadvantages that sufficiently down voted this option to the basement. Let’s investigate a few, firstly JNI required complex arrangements to call out into native code, you had to either use a third party library, or alternatively implement a lot of native shell code that implemented each native method declared in Java. Even this was not the biggest drawback. The biggest drawback was until recently, JNI calls prevented GC - see GC region pinning, and therefore carried the risk of introducing latency (the very thing we are trying to reduce anyway), and additionally, every thread that wanted to interact with the JVM had to register with it. Further, to access Java objects the JNI C++ code had to “inspect” the object itself, let’s just say the performance was not particularly good. The last obvious disadvantage is that the C++ code does not run in a VM, so any major error can crash the process.

Next, I look at the foreign memory API and also value classes, in order to see if there is another way. For this argument let’s take the worst possible case, an extremely high volume price feed that throughout a working day has to deal with a high volume of price data. In this article we’ll ignore the costs of sharing memory access across cores.

Why not use a C++ library with a Java application that handles the less critical parts?

However, enter modern C++, the native linker and foreign memory API, and many of the above issues simply vanish or reduce significantly. Let’s first take a look at a simple of use of the foreign memory API where I used it in TcMenu Generator UI and also to access data (it was when I wrote this code that led to this article).

So in summary, if we look at the way things are written there is a real paradigm shift, we define our function calls in terms of function names and their parameters exported from a shared object, along with this the JVM is in control of the memory directly, and no objects are passed outside the virtual machine. This means that most of the major flaws above are either reduced or completely removed.