Skip to end of metadata
Go to start of metadata

Language-level multi-platform support

This is an extension of the discussion surrounding the feature expressions page, and attempts to create a complete map of the problems, the potential solutions and their implications.

Problems

Problem #1

Writing code that targets multiple Clojure platforms currently results in large amounts of almost-but-not-quite-the-same code, resulting in code duplication.

For example:


;; euclidean-vector.clj 
  (extend-protocol EuclideanVector
     clojure.lang.PersistentVector
     ...
     (magnitude [this]
         (Math/sqrt (reduce %2B (map #(Math/pow % 2) this))))
     ...)
 
;; euclidean-vector.cljs 
(extend-protocol EuclideanVector
   cljs.core.PersistentVector
   ...
   (magnitude [this]
       (Math/sqrt (reduce %2B (map #(Math/pow % 2) this))))
   ...)

These code snippets differ in key places, but are otherwise identical in structure and content.

Problem #2

Writing code that targets multiple Clojure platforms often requires awkward or extraneous namespaces and files.

For example, in pure JVM Clojure, one could simply write one file, “euclidean-vector.clj” which contains a protocol definition, and an implementation against Clojure’s vectors 

In a hybrid Clojure/ClojureScript project, on the other hand, some of the implementation code is different, so one would need to split it out like so:


src/shared/euclidean-vector.clj (containing protocol definition & code that can be cross-compiled)
src/jvm/euclidean-vector/impl.clj (containing clojure-specific impl)
src/cljs/euclidean-vector/impl.cljs (containing cljs-specific impl)

Problem #3

(This is very closely related to problem #2, except pertaining to how the build process and tooling are affected rather than the structure of files in a project.)

Currently, compilation sources must be specified at the level of the whole file. If a single source file needs to be included in multiple compilations, it requires third-party tooling; such tools currently are immature and inconsistent in their approach, and utilize “ugly” techniques such as parsing comments or pre-reader source manipulation.

The primary existing tools are:

Problem #4

Currently, there is no way to conditionally compile different code based on the version of Clojure, version of the JVM, version of the platform (Rhino vs V8 in cljs), or availability of add-on features (for example, extended cryptography or JSR166 on older versions of java).

Even though such features could conceivably be included in build tooling (as leiningen plugins, for example) this wouldn’t help in cases where the code was being run in a non-lein environment, such as after being packaged in a jar.

Problem #5

The APIs of many core and contrib libraries were designed with particular platforms in mind. For example, the clojure.string library specifically does NOT duplicate core functionality already found in the java.lang.String API, such as ‘contains’ and ‘substring’. Another example are basic math operations like ‘sqrt’ and ‘log’.  

This means that idiomatic JVM Clojure code, of necessity, makes extensive use of interop forms for basic features. This means in turn, that almost all code intended for multiple platforms will encounter one of Problems 1-3.

Proposals

Simple macro

Put feature data in a global dynamic var.

Write a simple macro (called, for example, 'feature-cond') which has test and value expressions. The test expressions could check the feature data (perhaps with some syntactic sugar or pattern matching), and the value expressions would be emitted only if the feature expression passed.

Example:

(defn sqrt [x]
  (feature-cond
    :js (Math/sqrt x)
    :jvm (java.lang.Math/sqrt x)
    :clr (System.Math/Sqrt x)))

Benefits:

  • Solves problems 1, 2, 3 & 4
  • No modification to the core language or conceptual leaps

Tradeoffs:

  • Would break if used inside existing macros that rewrite or inspect their source, since they would see the 'feature-cond' expression rather than the evaluated platform-specific expression.
  • Can't extend code you don't own
Rewriting macro

Put feature data in a global dynamic var.

Allow users to edit a (possibly global) substitution mapping, possibly with syntactic sugar or pattern matching.

Wrap forms which may contain platform-specific code in a macro which will introspect the code and replace specified symbols with alternative symbols from the substitution map.

Example:

(def-alternative builtins/sqrt
  :js Math/sqrt
  :jvm java.lang.Math/sqrt
  :clr System.Math/Sqrt)
(replacing-alternatives
  (defn sqrt [x]
    (builtins/sqrt x)))

Benefits:

  • No modification to the core language
  • No conceptual leaps

Tradeoffs:

  • Limited to symbol replacement.
Feature expressions

Read-time eval

New macro phase

 Compiler reads metadata

Global Source rewriting

Leave it to tools / do nothing

Considerations

Granularity

At what level does the solution allow users to target code? Currently, one could say that code can bet targeted at the "classpath" level; multiple versions of the same code can't be on the same classpath during compilation. This implies that code targeting different platforms must be in different files which either reside in different directories or have different file extensions.

Other possibilities include:

  • Namespaces
  • Interned things (vars)
  • Top-level forms
  • Arbitrary forms

Integration point in the compilation pipeline

All these solutions hook in and supply different behavior at some point in the compilation process. Whatever else is true, this must happen before the actual compiler, since what is a valid form to one implementation's compiler might be an error in another.

This allows several different integration points:

  • A new preprocessing phase, before reading
  • Read time
  • A new compilation phase, post-read, pre-macroexpansion
  • Macro-expansion time

EDN compatibility?

Does the solution work for reading EDN data as well as source code?

Data representation of features available

Many of the proposals involve testing what features are available/present. This implies that there is a data structure, global or ambient at compilation time, that is inspect-able to determine what the environment is.

What kind of a data is present, how detailed it is, and what the actual structure of the representation is is largely orthogonal to which proposal is selected.

Power

What is possible when detecting a platform/feature? In order of least to most powerful:

  • Simple boolean check
  • Composite boolean checks (and & not operators allowed)
  • Range checks (greater than, less than, equality checks)
  • Arbitrary code execution

Locality

To what degree does the proposal preserve referential transparency? Is it always possible to tell if/when some code might be modified just from looking at that code, or could some other code be modifying it "from a distance"?

Compatibility

Is there any possible way the proposal could cause problems when used with existing code?

Programmatic Visibility

Using the proposal, is it possible to reflect/introspect on the parsed value to see where it came from and verify that it was the result of a feature expression? See Kevin Downey's comment: http://dev.clojure.org/display/design/Feature%2BExpressions?focusedCommentId=5243068#comment-5243068

Labels:
None
Edit Labels