Caution advised when using the Java ForkJoinPool or parallel streams with a SecurityManager

,

In Java 7 the fork/join framework was introduced as an additional tool in the concurrency toolbox. It can be used to conveniently solve problems that lend themselves to easy subdivision, in a parallell fashion without having to worry about any concurrency mechanics.

Aside from being a convenient user facing feature, this framework has also become the basis for other JDK implementations that require such a parallellization approach. Features like Arrays.parallelSort() and parallel streams. And there lies the rub.

The fork/join framework uses a ForkJoinPool to take the parallel tasks and execute them on a bunch of threads. By default it will use its own implementation of a ForkJoinPool.ForkJoinWorkerThreadFactory to construct the threads to be used, but it can be configured with a custom factory in its constructor.

In the case of JDK features such as parallel streams there is no API to specify this factory. In fact the JDK will default to using a system wide, static ForkJoinPool that is stored inside of the common field of the ForkJoinPool class. This pool is initialized in the method makeCommonPool of the same class.

This common pool turns out to be constructed in a very specific, very peculiar way. As Doug Lea explains in a post to the concurrency-interest mailing list, there was a concern that these "easy" ways to parallelize certain workloads could put undue strain on certain production systems and it was felt to be necessary, or at least desirable, to allow administrators to limit the amount of implicit parallelism exhibited by the JVM in certain production scenarios.

One result of this policy is that the logic for instantiating the common pool defaults to using a different, very conservative, ForkJoinWorkerThreadFactory implementation called the InnocuousForkJoinWorkerThreadFactory when it detects that a SecurityManager is active. This implementation runs worker threads with no permissions. This allows "innocuous" parallelism where something is calculated for example, but prevents parallel tasks from doing anything that requires permissions such as opening files.

This leads us to the first problem: when running with a security manager you get a different configuration of the common pool and you may not be able to run your parallel code, depending on its required permissions. Luckily there is a workaround for this. The ForkJoinPool documents a system property called java.util.concurrent.ForkJoinPool.common.threadFactory that allows you to specifically set the ThreadFactory implementation to be used. With this you can provide your own implementation, one that runs with normal permissions and Bob is your uncle.

Unless you are running in a WebStart environment. Wait, does anyone still use WebStart? Yes, sadly, we do. Since JDK 7u45 "insecure" properties set in a JNLP file are no longer passed to the application, unless you sign your JNLP file. The ThreadFactory property is not a secure property. Either you sign your JNLP files (which is incredibly cumbersome, and in the case of required customer customization, often not economically feasible) or you wrap that property in a "jnlp." prefixed "secure" property and translate that inside your application into the right property.

Assuming that you did this (and we did) it will sadly still not work. When running in a Java WebStart environment, a user-level classloader is used to load all resources specified in the JNLP file and as a result you can no longer use Class.forName or ClassLoader.getSystemClassLoader to load resources and classes. The ForkJoinPool uses ClassLoader.getSystemClassLoader to try to load the custom ThreadFactory you set using the system property and will thus fail.

The executive summary is that if you are running your application using Java WebStart you must construct your ForkJoinPool instances with a custom ThreadFactory if you want to parallelize any operation that requires security permissions and you should not use parallel streams for such operations. At work we have decided not to use parallel streams at all until we have a usable workaround for the described problem.