Language-level multi-platform support
This is an extension of the discussion surrounding the feature expressions page, and attempts to create a complete map of the problems, the potential solutions and their implications.
Problems
Problem #1
Writing code that targets multiple Clojure platforms currently results in large amounts of almost-but-not-quite-the-same code, resulting in code duplication.
For example:
;; euclidean-vector.clj(extend-protocol EuclideanVectorclojure.lang.PersistentVector...(magnitude [this](Math/sqrt (reduce %2B (map #(Math/pow % 2) this))))...)
;; euclidean-vector.cljs(extend-protocol EuclideanVectorcljs.core.PersistentVector...(magnitude [this](Math/sqrt (reduce %2B (map #(Math/pow % 2) this))))...)
These code snippets differ in key places, but are otherwise identical in structure and content.
Problem #2
Writing code that targets multiple Clojure platforms often requires awkward or extraneous namespaces and files.
For example, in pure JVM Clojure, one could simply write one file, “euclidean-vector.clj” which contains a protocol definition, and an implementation against Clojure’s vectors
In a hybrid Clojure/ClojureScript project, on the other hand, some of the implementation code is different, so one would need to split it out like so:
src/shared/euclidean-vector.clj (containing protocol definition & code that can be cross-compiled)
src/jvm/euclidean-vector/impl.clj (containing clojure-specific impl)
src/cljs/euclidean-vector/impl.cljs (containing cljs-specific impl)
Problem #3
(This is very closely related to problem #2, except pertaining to how the build process and tooling are affected rather than the structure of files in a project.)
Currently, compilation sources must be specified at the level of the whole file. If a single source file needs to be included in multiple compilations, it requires third-party tooling; such tools currently are immature and inconsistent in their approach, and utilize “ugly” techniques such as parsing comments or pre-reader source manipulation.
The primary existing tools are:
lein-cljsbuild with crossovers (https://github.com/emezeske/lein-cljsbuild/blob/0.3.0/doc/CROSSOVERS.md)
- cljx (https://github.com/lynaghk/cljx)
Problem #4
Currently, there is no way to conditionally compile different code based on the version of Clojure, version of the JVM, version of the platform (Rhino vs V8 in cljs), or availability of add-on features (for example, extended cryptography or JSR166 on older versions of java).
Even though such features could conceivably be included in build tooling (as leiningen plugins, for example) this wouldn’t help in cases where the code was being run in a non-lein environment, such as after being packaged in a jar.
Problem #5
The APIs of many core and contrib libraries were designed with particular platforms in mind. For example, the clojure.string library specifically does NOT duplicate core functionality already found in the java.lang.String API, such as ‘contains’ and ‘substring’. Another example are basic math operations like ‘sqrt’ and ‘log’.
This means that idiomatic JVM Clojure code, of necessity, makes extensive use of interop forms for basic features. This means in turn, that almost all code intended for multiple platforms will encounter one of Problems 1-3.
Proposals
Simple macro
Put feature data in a global dynamic var.
Write a simple macro (called, for example, 'feature-cond') which has test and value expressions. The test expressions could check the feature data (perhaps with some syntactic sugar or pattern matching), and the value expressions would be emitted only if the feature expression passed.
(defn sqrt [x]
(feature-cond
:js (Math/sqrt x)
:jvm (java.lang.Math/sqrt x)
:clr (System.Math/Sqrt x)))
Benefits:
- Solves problems 1, 2, 3 & 4
- No modification to the core language or conceptual leaps
Tradeoffs:
- Would break if used inside existing macros that rewrite or inspect their source, since they would see the 'feature-cond' expression rather than the evaluated platform-specific expression.
- Can't extend code you don't own
Rewriting macro
Put feature data in a global dynamic var.
Allow users to edit a (possibly global) substitution mapping, possibly with syntactic sugar or pattern matching.
Wrap forms which may contain platform-specific code in a macro which will introspect the code and replace specified symbols with alternative symbols from the substitution map.
Example:
(def-alternative builtins/sqrt
:js Math/sqrt
:jvm java.lang.Math/sqrt
:clr System.Math/Sqrt)
(replacing-alternatives
(defn sqrt [x]
(builtins/sqrt x)))
Benefits:
- No modification to the core language
- No conceptual leaps
Tradeoffs:
- Limited to symbol replacement.
Read-time eval
New macro phase
Compiler reads metadata
Global Source rewriting
Leave it to tools / do nothing
Considerations
Granularity
At what level does the solution allow users to target code? Currently, one could say that code can bet targeted at the "classpath" level; multiple versions of the same code can't be on the same classpath during compilation. This implies that code targeting different platforms must be in different files which either reside in different directories or have different file extensions.
Other possibilities include:
- Namespaces
- Interned things (vars)
- Top-level forms
- Arbitrary forms
Integration point in the compilation pipeline
All these solutions hook in and supply different behavior at some point in the compilation process. Whatever else is true, this must happen before the actual compiler, since what is a valid form to one implementation's compiler might be an error in another.
This allows several different integration points:
- A new preprocessing phase, before reading
- Read time
- A new compilation phase, post-read, pre-macroexpansion
- Macro-expansion time
EDN compatibility?
Does the solution work for reading EDN data as well as source code?
Data representation of features available
Many of the proposals involve testing what features are available/present. This implies that there is a data structure, global or ambient at compilation time, that is inspect-able to determine what the environment is.
What kind of a data is present, how detailed it is, and what the actual structure of the representation is is largely orthogonal to which proposal is selected.
Power
What is possible when detecting a platform/feature? In order of least to most powerful:
- Simple boolean check
- Composite boolean checks (and & not operators allowed)
- Range checks (greater than, less than, equality checks)
- Arbitrary code execution
Locality
To what degree does the proposal preserve referential transparency? Is it always possible to tell if/when some code might be modified just from looking at that code, or could some other code be modifying it "from a distance"?
Compatibility
Is there any possible way the proposal could cause problems when used with existing code?
Programmatic Visibility
Using the proposal, is it possible to reflect/introspect on the parsed value to see where it came from and verify that it was the result of a feature expression? See Kevin Downey's comment: http://dev.clojure.org/display/design/Feature%2BExpressions?focusedCommentId=5243068#comment-5243068