There is Always More to Learn

Wanting to change up my list of podcasts (Accidental Tech Podcast, Alternativlos, Back to Work, Core Intuition, CRE: Technik, Kultur, Gesellschaft, Developing Perspective, Fragmented, Hello Internet, Pragmatic, Raumzeit, Robot or Not?, Roderick on the Line, Software Engineering Radio, The Flop House, The Incomparable, The Talk Show With John Gruber, Upgrade, Welcome to Macintosh, You Look Nice Today) a bit, I went on the lookout for something not Apple-related but still developer centric. I stumbled upon the Fragmented podcast and have been continuously surprised at the amount and kind of things I learn from it. Notwithstanding reading the internet every day, this podcast managed to broaden my horizon: introduced new interesting people to read on the internet, discussed developing practices I either wasn't (AutoValue) or only tangentially (RxJava) aware of. It was a good example of the fact that every sizable community will have interesting things to learn that you don't already know about.

But wait, there's more (to consider about the ForkJoinPool and parallel streams)

I wrote earlier about the problems with using a ForkJoinPool explicitly, or implicitly through parallel streams, together with a SecurityManager. There are even more caveats with the way the ForkJoinPool is used by the parallel streams implementation and with the model of concurrency implemented by it.

The first issue with parallel streams is that they use a static, JVM-wide instance of a ForkJoinPool, called the commonPool. Implicitly using this opaque shared pool, you lose a lot of visibility into and control over important pool parameters that may be crucial to attaining high performance. Oleg Shelajev of Zeroturnaround writes about some of the congestion and management issues you can run into.

The second problem is that the ForkJoinPool as it is implemented in the JDK offers a very specific and narrow form of parallel processing. Edward Harned from Coopsoft writes at length about many of the characteristics and pitfalls of the particular form of recursive decomposition as implemented by the Java ForkJoinPool. You need to be very aware of your particular parallelization problem and make a very considered decision about whether this kind of fork/join approach is right for you.

Combine these two aspects together: little to no control over the parameters of the shared JVM-wide commonPool and the very narrow utility of the fork/join parallelization model, and it becomes clear that the utility of parallel streams is very limited and harbours many hidden dangers. It is possible that as things currently stand, it is a little bit too easy to shoot yourself in the proverbial foot.

Caution advised when using the Java ForkJoinPool or parallel streams with a SecurityManager

In Java 7 the fork/join framework was introduced as an additional tool in the concurrency toolbox. It can be used to conveniently solve problems that lend themselves to easy subdivision, in a parallell fashion without having to worry about any concurrency mechanics.

Aside from being a convenient user facing feature, this framework has also become the basis for other JDK implementations that require such a parallellization approach. Features like Arrays.parallelSort() and parallel streams. And there lies the rub.

The fork/join framework uses a ForkJoinPool to take the parallel tasks and execute them on a bunch of threads. By default it will use its own implementation of a ForkJoinPool.ForkJoinWorkerThreadFactory to construct the threads to be used, but it can be configured with a custom factory in its constructor.

In the case of JDK features such as parallel streams there is no API to specify this factory. In fact the JDK will default to using a system wide, static ForkJoinPool that is stored inside of the common field of the ForkJoinPool class. This pool is initialized in the method makeCommonPool of the same class.

This common pool turns out to be constructed in a very specific, very peculiar way. As Doug Lea explains in a post to the concurrency-interest mailing list, there was a concern that these "easy" ways to parallelize certain workloads could put undue strain on certain production systems and it was felt to be necessary, or at least desirable, to allow administrators to limit the amount of implicit parallelism exhibited by the JVM in certain production scenarios.

One result of this policy is that the logic for instantiating the common pool defaults to using a different, very conservative, ForkJoinWorkerThreadFactory implementation called the InnocuousForkJoinWorkerThreadFactory when it detects that a SecurityManager is active. This implementation runs worker threads with no permissions. This allows "innocuous" parallelism where something is calculated for example, but prevents parallel tasks from doing anything that requires permissions such as opening files.

This leads us to the first problem: when running with a security manager you get a different configuration of the common pool and you may not be able to run your parallel code, depending on its required permissions. Luckily there is a workaround for this. The ForkJoinPool documents a system property called java.util.concurrent.ForkJoinPool.common.threadFactory that allows you to specifically set the ThreadFactory implementation to be used. With this you can provide your own implementation, one that runs with normal permissions and Bob is your uncle.

Unless you are running in a WebStart environment. Wait, does anyone still use WebStart? Yes, sadly, we do. Since JDK 7u45 "insecure" properties set in a JNLP file are no longer passed to the application, unless you sign your JNLP file. The ThreadFactory property is not a secure property. Either you sign your JNLP files (which is incredibly cumbersome, and in the case of required customer customization, often not economically feasible) or you wrap that property in a "jnlp." prefixed "secure" property and translate that inside your application into the right property.

Assuming that you did this (and we did) it will sadly still not work. When running in a Java WebStart environment, a user-level classloader is used to load all resources specified in the JNLP file and as a result you can no longer use Class.forName or ClassLoader.getSystemClassLoader to load resources and classes. The ForkJoinPool uses ClassLoader.getSystemClassLoader to try to load the custom ThreadFactory you set using the system property and will thus fail.

The executive summary is that if you are running your application using Java WebStart you must construct your ForkJoinPool instances with a custom ThreadFactory if you want to parallelize any operation that requires security permissions and you should not use parallel streams for such operations. At work we have decided not to use parallel streams at all until we have a usable workaround for the described problem.

Encapsulation does not imply bloated classes

In an interesting discussion of anemic models, DDD and functional domain modeling Debasish Gosh links to an article by Scott Meyer (of Effective C++ fame) from 2000 that convincingly argues against bloated OO modeling. The goal should not be to pack all possible functionality related to the class on that class itself. Instead he argues that having more than the essential methods on a class actually decreases encapsulation since you are increasing the amount of functionality that breaks when internal details to your class change.

To paraphrase: everything that can be implemented using the public API of your class should be outside the class. Obviously design is never black or white like this, and he acknowledges that, but it is a realistic and non-typical take on encapsulation that is worth discussing.

Streaming Audio Problems on Android in Sleep Mode

Some Android Apps will suddenly drop audio streams shortly after going to sleep while on a wifi connection. The SomaFM App for example has this behaviour on my Nexus S under Android 4.1.

The reason is that the Power Saving Polling (PSP) mode on Android was recently changed. Apps should request a "Wifi Lock Mode" WIFI_MODE_FULL_HIGH_PERF if they want streaming to continue uninterrupted during sleep. There are at least two different Android bug reports discussing this behaviour.

A workaround is to install the free WIFI High Performance Widget and enabling it when you want to listen to a stream. It will force aforementioned Wifi Lock Mode during sleep.

"Anyway, it's not just the sci-fi"

“Anyway. It’s not just sci-fi. I’m also depressed about the lack of future in fashion. Every hep shop seems to be full of tweeds and leather and carefully authentic bits of restrained artisinal fashion. I think most of Shoreditch would be wondering around in a leather apron if it could. With pipe and beard and rickets. Every new coffee shop and organic foodery seems to be the same. Wood, brushed metal, bits of knackered toys on shelves. And blackboards. Everywhere there’s blackboards.”