Async Executor

Motivation

Need ability to invoke tasks asynchronously at the bottom of core.async. And fast.

Goals

• Executor interface
  • Need ability to invoke tasks asynchronously
    • Effectively ExecutorService.submit(Runnable)
  • May want ability to shutdown the executor?
• Executor implementation(s)
• Need some implementation to cover JDK 1.6, 1.7, 1.8
• Possible implementations
  • ThreadPoolExecutor
  • ForkJoin 
• Executor library version
  • Provide implementation without external dependency
    • Use what's in the JDK or
    • Repackage jsr166 into our own package (jsr166 is public domain)
  • Allow path for users to use a bleeding edge version
• Configuration 
  • Both ThreadPoolExecutor and ForkJoin have a number of of possible configuration parameters
  • Provide good defaults but also let user change them

Non-goals

• Do not need to provide general access to ForkJoin (fork, join, recursive computation, etc)
• Do not need access to futures for tasks
• Do not need task cancellation

Proposed solution

• Define protocol for async task execution
  • (defprotocol Executor (exec `[e task]))
• Provide several implementations
  • for ThreadPoolExecutor
  • for ForkJoinPool 
    • Repackage jsr166y into our own packages (put in Clojure 1.6)
    • Allow user to use jsr166e instead?
    • Use built-in JDK version
• The general ordering of "bestness" is:
  • JDK 1.6: jsr166y >> Clojure ForkJoinPool >> ThreadPoolExecutor
  • JDK 1.7: jsr166e >> jsr166y >> Clojure ForkJoinPool >> JDK ForkJoinPool
  • JDK 1.8: jsr166e >> JDK ForkJoinPool
• Allow user to override the implementation we chose with their own custom executor that does whatever they want
  • Done via an "install" api which is presumed to be called prior to system use and updates the executor in an atom

Table of what version you'll get depending on your environment based on the ordering rules above:

Background / Reference

Background:

• JSR 166 
  • jsr166x - included in JDK 1.6, does not contain ForkJoin
  • jsr166y - compiled with JDK 1.6, included in JDK 1.7. jsr166y jar has fixes past what is in the JDK 1.6.
  • jsr166e - compiled with JDK 1.7, to be included in JDK 1.8. jsr166e jar has fixes past what is in JDK 1.7. 
• Scala solution
  • Repackage known version of jsr166y as scala.concurrent and include in the Scala language (Akka makes use of it from there)
    • Covers JDK 1.6%2B (good) but does not contain the very latest updates for JDK 1.8 that are in jsr166e (bad)
    • When they have a version of Scala that requires JDK 1.7, they will presumably update to jsr166e
  • Akka has pluggable executor providers and using a custom one allows you to make use of jsr166e if you need it