Synopsis
The ClojureScript compiler currently directly prints JavaScript source code strings. It is desirable to instead produce a JavaScript Abstract Syntax Tree to simplify code generation, emit source maps, and to enable higher level optimizations.
Problems
- Compiler should be more functionally pure
- Code generation currently emits strings as a side effect
- Enhancing the compiler is difficult because printing forces ordering and limits higher order composition
- It's not safe to interleave multiple passes of analysis, transformations, and code generation
- emit complects Code Generation and Code Printing
- requires simultaneous consideration of JavaScript's abstract structure and particulars of syntax
- code generation is trivially functionally pure; code printing could be, but wouldn't benefit much
- SourceMaps! Strings lack structure to associate mappings
- If you currently are printing "foo(bar)", you might need to assign different source lines to both foo and bar
- Adding source mappings to printing would give very low mapping resolution for the current ad-hoc strings
- Increasing the resolution would yield something that looks very much like an AST
Goals
- Decouple code generation from code printing
- Simplify compiler.clj
- Include source mappings on outputted AST
- Preserve compiler and generated code performance
Approach
- Target the Google Closure AST
- Advantages
- Keeps us out of the JS AST design business
- Existing code printer
- Includes facilities for Closure Compiler type system
- Perf? Can skip roundtrip to JS source and to disk
- Disadvantages
- Closure AST nodes are stateful
- Each Node has an explicit parent reference
- Cloning policy is necessary to reuse subtrees (wasn't an issue in my port so far, see below)
- Increases our dependence on Closure
- Closure AST evolves more quickly than the JS spec
- Recent changes: http://code.google.com/p/closure-compiler/source/list?path=/trunk/src/com/google/javascript/jscomp/parsing/IRFactory.java&start=2295
- Closure AST nodes are stateful
- Alternative: Intermediate JavaScript AST as Clojure data-structures
- Requires an additional translation step
- GWT and Dart use a different AST; See: http://code.google.com/p/google-web-toolkit/source/browse/trunk/dev/core/src/com/google/gwt/dev/js/ClosureJsAstTranslator.java?r=10778
- Closure Compiler Discussion: https://groups.google.com/forum/?fromgroups=#!topic/closure-compiler-discuss/OEwLWkw4Kug
- Advantages
- Thin JS-AST DSL
- Recursively transpiles arguments
- Can be extended to automatically attach source mappings from dynamic bindings
- Does a Hiccup like seq expansion
- Eliminate js* form, replaces it with DSL invokes: (js* "foo({0})" x) becomes (js*/call 'foo x)
Preliminary Progress
- Fork: https://github.com/brandonbloom/clojurescript/tree/js-ast
- Interesting files: compiler.clj and js.clj
- As of Nov 9
- Compiles all the CLJS in the ClojureScript tree
- Bugged in advanced compilation mode
- Does not generate source mappings
- Does not bypass the extra round-trip to JS source and to disk
- Takes 3X as long as naive string printing; 10 vs 30 seconds for (compile-root "src/cljs/cljs") on my machine
- core.clj and core.cljs are fully ported from (js* "..." ...) to (js*/... ...)
Labels: