TornadoVM 7 brings native GPU support to Java

TornadoVM 7.3 arrives with a fresh runtime that swaps the old JNI bridge for the Java Foreign Function & Memory (FFM) API. The change removes the experimental JVM Compiler Interface (JVMCI) dependency and forces all compiler metadata through a single reflection-ASM-Unsafe path. Because the FFM API ships with every JDK 21+ release, the framework now runs unchanged on OpenJDK, GraalVM, Temurin and Zulu without any extra native glue code.

The release also tightens CUDA integration, adds multi-GPU support, and expands library bindings for popular Java ecosystems. This post walks through the most visible architectural shifts, shows how a typical data-parallel workload can be accelerated, and points out the concrete steps a developer needs to take to start using TornadoVM today.

A concrete scenario

Imagine a data-processing service that runs Apache Flink queries on a fleet of cloud VMs. Most VMs have an integrated GPU, but a few high-throughput nodes are equipped with Nvidia cards. The team wants the same query code to run on both kinds of hardware and to get a speed boost on the Nvidia devices without maintaining separate code paths.

With TornadoVM 7.3 the service can keep its existing Flink Table API code, flip a single configuration flag, and let the framework decide at runtime whether the query should be executed on the CPU, the integrated GPU, or the Nvidia accelerator. The rest of this article explains how that works under the hood.

New FFM API replaces JNI

The previous version relied on JNI to load native libraries and forward calls to OpenCL or CUDA. JNI required a platform-specific shared object, manual memory pinning, and a separate build step for each target OS.

TornadoVM 7.3 now uses the standard Java foreign-memory mechanisms provided by the FFM API. Buffers are allocated with MemorySegment.allocateNative, pointers are passed as MemoryAddress objects, and backend functions are invoked through Linker.nativeLinker().downcallHandle.

Because these calls are pure Java, the same JAR can be executed on a laptop with an Intel Iris Xe GPU and later on a server with an Nvidia accelerator. The source notes that this “single, uniform path” simplifies deployment and eliminates the need for a separate compiler plug-in.

Example: allocating a buffer for an OpenCL kernel

import java.lang.foreign.*;
import static java.lang.foreign.ValueLayout.*;

MemorySegment buffer = MemorySegment.allocateNative(64 * Float.BYTES, ResourceScope.globalScope());
MemoryAddress ptr = buffer.address();

The snippet replaces the previous JNI malloc call and works on any JDK 21+ runtime.

Deeper CUDA integration with TileContext

Version 7 introduces a dedicated TileContext that maps directly onto Nvidia’s cuTile programming model. A TaskGraph can now replace a just-in-time (JIT) generated kernel with a call to a CUDA library such as cuBLAS or CUTLASS. The source describes this as “swap JIT-generated kernels for CUDA library calls, or chain the two together,” giving developers fine-grained control over performance versus portability.

The same hybrid API is available for Apple’s MXL library, meaning a single Java code base can target both Nvidia and Apple GPUs without rewriting kernel logic. This is especially useful for teams that ship desktop-class macOS clients alongside cloud-native Linux services.

Multi-GPU execution and library bindings

When a node hosts more than one accelerator, TornadoVM can split a TaskGraph across them. The runtime inspects the available devices, partitions the workload, and balances the execution automatically. This capability is exposed through the MultiDeviceContext class, which the source says “adds true multi-GPU execution.”

Library integrations have also been broadened. TornadoVM now ships bindings for Quarkus, Lang4j, Apache Flink and Spring AI, all reachable through SDKMan with a single command (sdk install tornadovm). The most concrete example comes from the Flink integration: a developer writes a normal Table API query, adds a configuration flag, and lets TornadoVM decide at runtime whether to offload.

Flink integration flow

1. Write the query using the standard Flink Table API.
2. Enable acceleration by setting tornado.enable=true in the Flink configuration.
3. At runtime, TornadoVM probes the node for a GPU when the query reaches the task manager.
4. The framework examines the intermediate representation of the query; if loops are data-parallel, they are marked for offload.
5. Eligible loops are compiled to OpenCL or CUDA and dispatched to the accelerator, while the rest runs on the CPU.

When offloading makes sense

Not every Java computation benefits from the accelerator. The source highlights data-parallel workloads as the sweet spot: loops whose iterations have no inter-dependency can be executed concurrently on the GPU. Typical candidates include linear algebra operations such as matrix multiplication, fast Fourier transforms for signal analysis, deep neural network inference and hand-optimized sparse-matrix kernels. If a loop contains a reduction with a data dependency, TornadoVM falls back to the CPU, preserving correctness while avoiding unnecessary overhead.

The takeaway

TornadoVM 7.3 replaces JNI with the standard FFM API, removes the JVMCI dependency, and adds a uniform reflection-ASM-Unsafe path that works on any JDK 21+. It deepens CUDA support through TileContext, brings true multi-GPU execution, and expands library bindings such as Flink and Spring AI.

For a Java team, the practical step is to add the TornadoVM flag to the relevant library configuration, let the runtime probe the hardware, and let it offload data-parallel loops automatically. This makes native GPU acceleration a low-friction option for modern Java workloads.