Chasing Java's release train, from 8 to 16. Part 1: JDK 9, the last big banger ...

A lot has been said about JDK's new 6 months release cadence cycle, and arguably, this is one of the most profound decisions being made in the recent history of the language and JVM platform in general. But if you, like me and other ~60% of the Java developers out there, stuck with JDK 8, the new releases mean little to your day to day job. But the language changes, the standard library changes, the JVM changes and so does the tooling around it. More importantly, the ecosystem also changes by aggressively raising the compatibility baselines. Java in 2021 is not the same as it was in 2020, 2017 and all the more in 2014, when Java 8 has seen the light. The soon to be out JDK 17, the next LTS release, would raise the bar even higher.

So the challenge many of us, seasoned Java developers, are facing is how to stay up to date? The risk of your skills becoming outdated is real. Hence this series of the blog posts: to consolidate the changes in the language, JVM, standard library and tooling over each release, primary from the perspective of the developer. At the end, we should be all set to meet the JDK 17 fully armed. To mention upfront, we are not going to talk about incubating or preview features, nor about come-and-go ones (looking at you, jaotc). Whenever it makes sense, the feature would have a corresponding JEP link, but overall there are many changes not covered by dedicated proposals. I will try to include as much as possible but there are chances some useful features may still slip away.

With that, let us open the stage with JDK 8.

JDK 8

Unsurprisingly, JDK 8 does not only receive the security patches and bugfixes but gets some of the new features from the upstream. The two notable ones are:

• Improved Docker container detection and resource configuration usage: since 8u191, the JVM is fully container-aware (see please JDK-8146115). The support is only available on Linux based platforms and enabled by default.

• JFR backport: since 8u262, JFR is supported seamlessly (see please JDK-8223147), no need to pass -XX:+UnlockCommercialFeatures and alike anymore

Although not strictly related, thanks to the JFR backport, the latest JDK Mission Control 8 release comes very handy to everyone. Hopefully, your organization does update to the recent JDK patch releases on the regular basis and these features are already available to you.

JDK 9

The JDK 9 release was just huge (90+ JEPs): not only from the perspective of the bundled features, but also from the impact on the platform and ecosystem. It was 2017 but even today the groundwork established by JDK 9 still continues. Let us take a look at the major features which went into it.

• JEP-261: Module System: this change by itself is worth a book. Fast track to today, the debates are still ongoing if this feature is useful outside of JDK itself. The adoption of Java modules is not that high to be fair, and besides Automatic-Module-Name, not that many projects decided to take the full-fledged modularization path. Along with numerous JVM and standard library changes, the JPMS comes with:

  • New java command line options:
    • --add-exports: to export a package, which makes its public types and members accessible
    • --add-opens: to open a package, which makes all its types and members accessible
    • --patch-module: adds classes to a specific module
    • --add-modules: adds the listed modules and their transitive dependencies to the module graph
    • --add-reads: makes one module read another
    • --list-modules: lists the observable modules
    • --limit-modules: specifies the limit of the universe of observable modules
    • --describe-module: describes a specified module
    • --validate-modules: validates all modules
    • --illegal-access=permit|warn|debug|deny: accessing internal APIs
  • New javac command line options:
    • --add-modules: specifies root modules to resolve in addition to the initial modules
    • --limit-modules: specifies the limit of the universe of observable modules
    • --module: compiles only the specified module and checks time stamps
    • --module-path: specifies where to find application modules
    • --module-source-path: specifies where to find input source files for multiple modules
    • --module-version: specifies the version of modules that are being compiled
    • --add-exports: specifies a package to be considered as exported from its defining module
    • --add-reads: specifies additional modules to be considered as required by a given module
    • --release: compiles against the public, supported and documented API for a specific VM version
  • New jmod tool

• JEP-260: Encapsulate Most Internal APIs: encapsulates most of the JDK's internal APIs by default so that they are inaccessible at compile time, and prepare for a future release in which they will be inaccessible at run time. As we are going to see later on, the hardening efforts kicked off in this JEP are continued in later JDK releases.

• JEP-282: jlink: The Java Linker: a tool that can assemble and optimize a set of modules and their dependencies into a custom run-time image. It is closely related to JEP-220: Modular Run-Time Images and essentially allows to bundle a highly optimized distribution tailored to the needs of your applications. As you may expect, jlink only works with applications which are fully modularized, otherwise you have to do heavy lifting yourself.

• JEP 238: Multi-Release JAR Files: extends the JAR file format to allow multiple, Java-release-specific versions of class files to coexist in a single archive. The goal of this change is to allow third party libraries and frameworks to take advantage of the language or/and API features available in newer releases but still supporting the older ones. The feature has proven to be quite useful in practice but the tooling support around it, Apache Maven in particular, is still falling behind: see please here and here.

• JEP-241: Remove the jhat Tool: jhat was an experimental, unsupported, and out-of-date tool, superior heap visualizers and analyzers have now been available for many years.

• JEP-228: Add More Diagnostic Commands: defines additional diagnostic commands in order to improve the diagnosability of Hotspot VM and the JDK. In scope of this feature, new diagnostic commands have been added to jcmd tool:

  • VM.info
  • VM.class_hierarchy
  • VM.set_flag
  • JVMTI.data_dump

• JEP-224: HTML5 Javadoc: enhances the javadoc tool to generate HTML5 markup. The primary drivers behind this feature are to increase the semantic value of web pages and to make it easier to create accessible web pages.

• JEP-229: Create PKCS12 Keystores by Default: transitions the default keystore type from JKS to PKCS12. The applications may explicitly specify the keystore type they require.

• JEP-244: TLS Application-Layer Protocol Negotiation Extension: extends the javax.net.ssl package to support the TLS Application Layer Protocol Negotiation (ALPN) Extension, which provides the means to negotiate an application protocol for a TLS connection. It is a necessary prerequisite for HTTP/2 support, both for client-side and for server-side.

• JEP-226: UTF-8 Property Resource Bundles: changes the default file encoding for ResourceBundle class to load properties files from ISO-8859-1 to UTF-8, applications no longer need to convert the properties files using the escape mechanism.

• JEP-243: Java-Level JVM Compiler Interface: introduces a Java based JVM compiler interface (JVMCI) enabling a compiler written in Java to be used by the JVM as a dynamic compiler. It paved the path for Graal JIT compiler integration into HotSpot VM.

• JEP-222: jshell: The Java Shell (Read-Eval-Print Loop): a true gem of the JDK 9 release, an interactive tool to evaluate declarations, statements, and expressions of the Java programming language, together with an API so that other applications can leverage this functionality. Just use it, very handsome, luckily we have JShell Maven Plugin to take care of the classpath manipulations.

• JEP-248: Make G1 the Default Garbage Collector: G1 becomes the default garbage collector on 32- and 64-bit server configurations. This change is based on the assumption that limiting latency is often more important than maximizing throughput.

• JEP-271: Unified GC Logging: reimplements GC logging in a manner that is as consistent as is reasonable with the current GC logging format. There are a lot of details baked in, I would highly recommend the series of the articles (part 1, part 2) to learn about GC logging configuration and migration hints.

• JEP-264: Platform Logging API and Service: defines a minimal logging API which platform classes can use to log messages, together with a service interface for consumers of those messages. A library or application can provide an implementation of this service in order to route platform log messages to the logging framework of its choice. If no implementation is provided then a default implementation based upon the java.util.logging API is used.

  This is quite valuable feature since the applications would be able to configure the JDK to use the same logging framework as the application itself. The Log4j2 already supports that but SLF4J is lagging a bit. In any case, the integration is looking quite straightforward.

• JEP-277: Enhanced Deprecation: allows to communicate better the status and intended disposition of APIs in the specification. The @Deprecated annotation acquired two new properties: forRemoval and since.

  @Deprecated(since = "11", forRemoval = true)
  

  With such additions it became possible to express the expectations regarding the API lifecycle in more structural and formal way.

• JEP-285: Spin-Wait Hints: defines an API to allow Java code to hint to the run-time system that a spin loop is being executed. It resulted into adding a new (empty) method to java.lang.Thread: onSpinWait().

  while ( /* condition */ ) {
      Thread.onSpinWait();
  }
  

  If you would like to learn more about how this method is being implemented in the HotSpot VM, please check JDK-8147844 issue.

• JEP-287: SHA-3 Hash Algorithms: implements the SHA-3 cryptographic hash functions, quite timely addition to the Java's security library. The MessageDigest implementation has been enhanced with SHA3-224, SHA3-256, SHA3-384 and SHA3-512 algorithms.

final MessageDigest digest = MessageDigest.getInstance("SHA3-512");
...

• JEP 254: Compact Strings: adopts a more space-efficient internal representation for strings while maintaining performance in most scenarios and preserving full compatibility for all related Java and native interfaces. Practically, the internal representation has changed from a UTF-16 char array to a byte array plus an encoding-flag field, no changes to existing public APIs.

• JEP-266: More Concurrency Updates: adds an interoperable publish-subscribe framework, enhancements to the CompletableFuture API, and various other improvements.

  • Reactive Streams publish/subscribe APIs (and supporting classes):
  • java.util.concurrent.Flow
  • java.util.concurrent.Flow.Publisher
  • java.util.concurrent.Flow.Subscriber
  • java.util.concurrent.Flow.Processor
  • java.util.concurrent.Flow.Subscription
  • java.util.concurrent.SubmissionPublisher
  • The CompletableFuture gained a plethora of new public API methods:
  • CompletableFuture<T> newIncompleteFuture()
  • Executor defaultExecutor() (subclassing enhancement, to allow supporting an alternative default executor)
  • CompletableFuture<T> copy()
  • CompletableFuture<T> minimalCompletionStage()
  • CompletableFuture<T> completeAsync(Supplier supplier, Executor executor)
  • CompletableFuture<T> completeAsync(Supplier supplier)
  • CompletableFuture<T> orTimeout(long timeout, TimeUnit unit)
  • CompletableFuture<T> completeOnTimeout(T value, long timeout, TimeUnit unit)
  • static <U> CompletableFuture<U> failedFuture(Throwable ex)
  • static <U> CompletionStage<U> completedStage(U value)
  • static <U> CompletionStage<U> failedStage(Throwable ex)
  • static Executor delayedExecutor(long delay, TimeUnit unit)
  • static Executor delayedExecutor(long delay, TimeUnit unit, Executor executor)

  Arguably, the inclusion of the delays and timeouts into standard library was the most awaited feature. It eliminated a number of the flaws and pain points that developers had been facing using CompletableFuture APIs for years.

• JEP-143: Improve Contended Locking: improves the overall performance of contended Java object monitors.

• JEP-290: Filter Incoming Serialization Data: allows incoming streams of object-serialization data (ObjectInputStream) to be filtered in order to improve both security and robustness. A process-wide filter could be configured via a system property or a configuration file. The system property, if supplied, supersedes the security property value:

  • System property: jdk.serialFilter
  • Security property: jdk.serialFilter in conf/security/java.properties

• JEP-102: Process API Updates: improves the API for controlling and managing operating-system processes.

  • A new java.lang.ProcessHandle interface has been introduced
  • The java.lang.Process class API has been enriched
    • boolean supportsNormalTermination()
    • long pid()
    • CompletableFuture<Process> onExit()
    • ProcessHandle toHandle()
    • ProcessHandle.Info info()
    • Stream<ProcessHandle> children()
    • Stream<ProcessHandle> descendants()
  • The java.lang.ProcessBuilder class API has been enriched
    • static List<Process> startPipeline(List<ProcessBuilder> builders)

• JEP-280: Indify String Concatenation: changes the static String-concatenation bytecode sequence generated by javac to use invokedynamic calls to JDK library functions. This will enable future optimizations of String concatenation without requiring further changes to the bytecode emitted by javac. This one deserves more attention since it touches upon the most painful code review subject: string concatenation using '+' in Java. Suppose we have a following code snippet.

  public static void main(String[] args) {
      final String s = "folder" + File.separator + "file";
      System.out.println(s);
  }
  

  In JDK 8, when we compile it, the following byte code is going to be generated:

   ...
   public static void main(java.lang.String[]);
      Code:
         0: new           #16                 // class java/lang/StringBuilder
         3: dup
         4: ldc           #18                 // String folder
         6: invokespecial #20                 // Method java/lang/StringBuilder."":(Ljava/lang/String;)V
         9: getstatic     #23                 // Field java/io/File.separator:Ljava/lang/String;
        12: invokevirtual #29                 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
        15: ldc           #33                 // String file
        17: invokevirtual #29                 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
        20: invokevirtual #35                 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
        23: astore_1
        24: getstatic     #39                 // Field java/lang/System.out:Ljava/io/PrintStream;
        27: aload_1
        28: invokevirtual #45                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
        31: return
  

  The StringBuilder has been introduced by the compiler to handle the concatenation efficiently. But on JDK 9 (and onwards), the picture is completely different:

   ...
   public static void main(java.lang.String[]);
      Code:
         0: getstatic     #2                  // Field java/io/File.separator:Ljava/lang/String;
         3: invokedynamic #3,  0              // InvokeDynamic #0:makeConcatWithConstants:(Ljava/lang/String;)Ljava/lang/String;
         8: astore_1
         9: getstatic     #4                  // Field java/lang/System.out:Ljava/io/PrintStream;
        12: aload_1
        13: invokevirtual #5                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
        16: return
  

  If you would like to learn more, besides reading JEP itself, I would highly recommend to glance over this blog post: part 1, part 2.

• JEP-269: Convenience Factory Methods for Collections: defines the library APIs to make it convenient to create instances of collections and maps with small numbers of elements, so as to ease the pain of not having collection literals in the Java programming language. Undoubtedly, the fluent collections factories significantly reduced the amount of boilerplate (as well as tricks) the developers had to deal with.

  • static <E> List<E> List.of(...)
  • static <E> Set<E> Set.of(...)
  • static <K, V> Entry<K, V> Map.of(...)
  • static <K, V> Entry<K, V> Map.ofEntries(...)
  • static <K, V> Entry<K, V> entry(K k, V v) (helper factory method to construct entries)

  Please take a notice that each of the factory methods has many overloads, the returned collections are unmodifiable and are safe for concurrent access by multiple threads. Null elements, keys, and values are not allowed, the resulting objects are serializable.

• JEP-259: Stack-Walking API: defines an efficient standard API for stack walking that allows easy filtering of, and lazy access to, the information in stack traces.

      final List<Class<?>> callers = StackWalker
          .getInstance(Option.RETAIN_CLASS_REFERENCE)
          .walk(frames -> frames
              .map(StackFrame::getDeclaringClass)
              .collect(Collectors.toList()));
  

  Before that, for many years Throwable::getStackTrace() and Thread::getStackTrace() were the only viable options to capture the stack traces in Java programmatically.

• JEP-193: Variable Handles: defines a standard means to invoke the equivalents of various java.util.concurrent.atomic and sun.misc.Unsafe operations upon object fields and array elements, a standard set of fence operations for fine-grained control of memory ordering, and a standard reachability-fence operation to ensure that a referenced object remains strongly reachable. The methods to create VarHandle instances are located in java.lang.invoke.MethodHandles, which we are going to talk about next, but there are a few static methods available as well.

  • static void fullFence()
  • static void acquireFence()
  • static void releaseFence()
  • static void loadLoadFence()
  • static void storeStoreFence()

  For the majority of the Java developers, the occasion to use VarHandles may never pop up, but it is a must-have tool in case you need to deal with memory fencing constraints and fine grained memory control.

• JEP-274: Enhanced Method Handles: enhances the MethodHandle, MethodHandles, and MethodHandles.Lookup classes to ease common use cases and enable better compiler optimizations.

  • New public methods added to MethodHandle:
    • MethodHandle asSpreader(...)
    • MethodHandle asCollector(...)
    • MethodHandle withVarargs(...)
  • New public methods added to MethodHandles.Lookup:
    • Lookup dropLookupMode(...)
    • Class<?> defineClass(...)
    • Class<?> findClass(...)
    • Class<?> accessClass(...)
    • VarHandle findVarHandle(...)
    • VarHandle findStaticVarHandle(...)
    • VarHandle unreflectVarHandle(...)
    • boolean hasPrivateAccess()
  • New static methods added to MethodHandles:
    • static Lookup privateLookupIn(...) (so missed in JDK 8)
    • static MethodHandle arrayConstructor(...)
    • static MethodHandle arrayLength(...)
    • static VarHandle arrayElementVarHandle(...)
    • static VarHandle byteArrayViewVarHandle(...)
    • static VarHandle byteBufferViewVarHandle(...)
    • static MethodHandle varHandleExactInvoker(...)
    • static MethodHandle varHandleInvoker(...)
    • static MethodHandle zero(...)
    • static MethodHandle empty(...)
    • static MethodHandle dropArgumentsToMatch(...)
    • static MethodHandle foldArguments(...)
    • static MethodHandle loop(...)
    • static MethodHandle whileLoop(...)
    • static MethodHandle doWhileLoop(...)
    • static MethodHandle countedLoop(...)
    • static MethodHandle iteratedLoop(...)
    • static MethodHandle tryFinally(...)
  • New methods to java.util.concurrent.atomic.AtomicXxx classes family (AtomicBoolean, AtomicInteger, AtomicLong, ...)
    • compareAndExchange(...)
    • compareAndExchangeAcquire(...)
    • compareAndExchangeRelease(...)
    • getAcquire()
    • getOpaque()
    • getPlain()
    • setOpaque(...)
    • setPlain(...)
    • setRelease(...)
    • weakCompareAndSetAcquire(...)
    • weakCompareAndSetPlain(...)
    • weakCompareAndSetRelease(...)
    • weakCompareAndSetVolatile(...)

  As you may noticed, a number of methods return (or accept) VarHandle.

• JEP-213: Milling Project Coin: introduces five small amendments to the Java Programming Language. Let us take a closer look at those.

  • Allow @SafeVargs on private instance methods

    public class Testable<T> {
        @SafeVarargs
        private void test(T ... tests) {
            // ... implementation here ...
        }
    }
    

  • Allow effectively-final variables to be used as resources in the try-with-resources statement

    final ByteArrayOutputStream out = new ByteArrayOutputStream();
    try (out) {
        // ... implementation here ...
    }
    

  • Allow diamond with anonymous classes if the argument type of the inferred type is denotable

    final PrivilegedAction<String> action = new PrivilegedAction<>() {
        @Override
        public String run() {
            // ... implementation here ...
            return ...;
        }
    };
    

  • Remove underscore from the set of legal identifier names

    public static void main(String[] args) {
        final int _= 0;
    }
    

    Would result in compilation error:

    ... as of release 9, '_' is a keyword, and may not be used as an identifier
    

  • Support for private methods in interfaces

    public interface Testable {
        private void test() {
            // ... implementation here ...
        }
    }
    

• JEP-263: HiDPI Graphics on Windows and Linux: implements HiDPI graphics on Windows and Linux.

• Multiple Stream enhancements:

  • Stream<T> takeWhile(...)
  • Stream<T> dropWhile(...)
  • static <T> Stream<T> ofNullable(...)
  • static <T> Stream<T> iterate(...) (enhanced favor)

• A few Optional enhancements:

  • void ifPresentOrElse(...)
  • Optional<T> or(...)
  • Stream<T> stream()

• A number of Collectors enhancements:

  • static <T,A,R> Collector<T,?,R> filtering(...)
  • static <T,U,A,R> Collector<T,?,R> flatMapping(...)

• Better parallel Files.lines(...) splitting implementation

• A chunk of new methods and overloads was added to Math class

  • static long multiplyExact(...)
  • static long multiplyFull(...)
  • static long multiplyHigh(...)
  • static long floorDiv(...)
  • static int floorMod(...)
  • static double fma(...)

• Lexicographical array comparison using Arrays::compare (great help to Apache Cassandra and Apache HBase), plus a few more additions:

  • static int compareUnsigned(...)
  • static boolean equals(...)
  • static int mismatch(...)

• The ServiceLoader also got some love:

  • Optional<S> findFirst()
  • Stream<ServiceLoader.Provider<S>> stream() (and a new generic interface ServiceLoader.Provider<S>)
  • static <S> ServiceLoader<S> load(...)

• The Scanner class has been modernized as well:

  • Stream<MatchResult> findAll(...)
  • Stream<String> tokens()

• Certainly positive improvements were introduced into Matcher class:

  • Matcher appendReplacement(...)
  • StringBuilder appendTail(...)
  • String replaceAll(...)
  • String replaceFirst(...)
  • Stream<MatchResult> results()

• Increase precision of Clock.systemUTC(): enhances the implementation of the system clocks so that they offer at least the same precision than the underlying clock available on the system. This one may hit many by surprise.

  /* JDK 8 */ Clock.systemUTC().instant() returns '2021-03-31T02:08:45.330Z'
  /* JDK 9 */ Clock.systemUTC().instant() returns '2021-03-31T02:08:45.823907100Z'
  

  The change in precision is explained by the fact that since JDK 9, the Clock will see higher precision, depending on the available clock of the operating system. The developers have the ability to change that using existing Clock::tick method:

  final Clock clock = Clock.tick(Clock.systemUTC(), Duration.ofMillis(1));
  

  Or leverage a new way, added in JDK 9, Clock:tickMillis:

  final Clock clock = Clock.tickMillis(ZoneId.systemDefault());
  

  To learn more about the clock precision quirks, please check out this great article Nanotrusting the Nanotime by Aleksey Shipilëv.

• Enhanced SecureRandom algorithms and new methods:

  • static SecureRandom getInstance(...)
  • SecureRandomParameters getParameters()
  • void nextBytes(...)
  • void reseed()

  For more exhaustive overview please check out this excellent Everything about Java's SecureRandom article.

• More useful methods added to InputStream (and less reasons to look for help beyond the standard library):

  • byte[] readAllBytes()
  • int readNBytes(...)
  • long transferTo(OutputStream out)

• More Objects methods

  • static <T> T requireNonNullElse(...)
  • static <T> T requireNonNullElseGet(...)
  • static int checkIndex(...)
  • static int checkFromToIndex(...)
  • static int checkFromIndexSize(...)

• The Enumeration interface was complemented with asIterator() method

• JDBC 4.3 API baseline which includes many changes across java.sql package

• ConcurrentHashMap::computeIfAbsent locking improvements (see please JDK-8161372)

• jhsdb - the command line debugger / server / UI debugger. Could attach to a Java process or could be launched as a postmortem debugger to analyze the content of a core dump from a crashed Java Virtual Machine (JVM).

The JDK 9 changeset was massive, at the same time - this is the last big bang release. We have covered most interesting pieces of it, at least from the developer's point of view. In the upcoming posts we are going to dissect other releases, one by one, and reveal the hidden gems of each.