Skip to end of metadata
Go to start of metadata


UPDATE 06052013 - Herwig

Updated the page with input from David Nolen, the description of the prototype is left intact at the end of the page.

I concur with giving specify an extend-like interface, which can be passed functions.

UPDATE 06052013-2 - Herwig

I pushed a commit to github that makes most of clojurescript runnable again with the refactoring:

There is still an issue with accessing locals from reify, when implementing IFn. Due to the call method hack.

Added replys to comments.

Problem statement

Even with namespacing extend-type is a global modification of a Clojure program. While powerful, this feature comes at some cost to modularity. On dynamic hosts like JavaScript, Clojure can provide an efficient modification of individual instances so that they may participate in a protocol. Such a feature would have the additional benefit of actually being more general than extend-type - that is the global modification (via JavaScript prototypes in the case of ClojureScript) is a special case of instance modification.


Add two new macros: specify and specify!. Both have a syntax similar to clojure.core/extend, but take a target object in place of the type and are macros.

specify! will mutate the target object by assigning protocol methods as keys. This is mostly useful to create new prototypes, but can occasionally be useful for external objects from libraries or eval.

specify will clone the target object before specify! ing the clone.

The function implementing the method (existing-fn) will be called method-fn in this.


Giving specify an interface like extend has following difficulties

  1. Since there is one slot per method, you need to pass a multi-arity fn when implementing a multi-arity method.
    In a naive implementation, that would cost another arity dispatch, which already has been done at the method call site.
    This can be mitigated by having specify reuse the existing infrastructure for arity optimization: properties on the fn object called .cljs$lang$arity$*
    specify can peel off those properties and use them as methods, which leads to the next cost
  2. A redundant fn object when implementing inline (the one that would do the arity dispatch otherwise)
    this could be mitigated by having specify recognize fn forms and emitting the arity-methods directly 
  3. specify will have to dispatch on the type(s) of method-fn
    the type being whether it has .cljs$lang$arity$*
    see Issues > Method arities 


Method arities

Since the arity dispatch is already done at the method call site, we want to directly call the appropriate arity of the method-fn. Since method-fns are lexically bound, we cannot statically know in all cases, how to get the target arity thunk from method-fn.

As the fallback, specify will have to look at the method-fn in every call (of specify!), but this could be mitigated, when the type of a method-fn is known from analysis.

Compatibility with external objects / Name clashes

Since specify may come into contact with external data, there is a risk of name clashes, since under advanced compilation method names are shortened to few characters.

specify should check if a property exists on the target object before assigning and should throw if so. This will also guard against double specification with possibly different method-fns.

For external data, such conflicts can and should be resolved with externs files.

We need to think about this some more - we really don't want any checks for modifying prototypes. We need specify! to work elegantly for both scenarios - David

Ok, that's true. Maybe a warning would be more appropriate but the warning strings would already equal half a reflection layer. In production, people really shouldn't specify the same object more than once. For development and macros that will use specify on the prototype (extend-type), there could be another metadata flag ^:force-specify - Herwig

I'm inclined to say that specify! need not warn at all. If you're going to specify! foreign data you don't control that's a user error. For foreign data I think it's probably far wiser to use reify.

It's a user error, alright, but it's one that might manifest as a heisenbug from compilation to compilation, because the set of clobbered property names will change randomly. It won't show up under normal compilation and not even under advanced compilation with minification disabled (there is an option in gclosure for that, we should expose it). A truly horrifying perspective in my eyes. We should at least warn about it. - Herwig

The docstring of specify! should make it clear that it's not intended for data objects outside the ClojureScript compilation process and suggest reify for those cases. - David

I'm not exactly happy with that, but I think its manageable, given that clojurescript has still some way to go towards 1.0 - Herwig

Object methods and IFn

As per the refactoring from CLJS-414, generating (unmangled) Object methods is possible with (the private version of) specify.

When implementing IFn, extend-type generates a multi-arity fn and uses that as a call method. This hack was carried over into CLJS-414.

specify will generate arity methods, call and apply directly, when implementing IFn.

David, can you comment on how .apply vs .cljs$lang$applyTo are used?

.apply is for regular JavaScript fns and simple ClojureScript fns, .cljs$lang$applyTo is for everything else, .cljs$lang$applyTo gets checked first - David

I will need to unify the code for emitting a function with the code for specifying IFn. That move might even make fn* redundant in the analysis format, since it can then be implemented as (specify! (Function.) IFn ...) - Herwig

I don't see how since you still want the general version that dispatches via argument.length - David

Well, my example presupposed that specifying an Object as IFn would generate .call and .apply, as requested, which would supposedly need to dispatch on argument.lenght and do all the other things fn is doing, won't it? - Herwig

I don't see why you need to generate .apply instead of cljs$lang$applyTo. We do probably need to generate .call, this is mostly needed for compilation mode other than advanced - David

1. pass a function-like object into jquery 2. jquery calls .apply on it. ... We don't need to generate .apply for every specify IFn; a single .apply method that calls .call or .cljs$lang$applyTo can be assigned to all implementors of IFn. Maybe just Function.prototype.apply - Herwig

It's an entirely unrealistic expection to think you can pass instances of IFn to external JavaScript library and think they will work. The point of adding a .call property is about a uniform ClojureScript calling convention. JS libs will not generally invoke functions via .call or .apply. Adding .apply serves no purpose for ClojureScript that clj$lang$applyTo doesn't solve. - David

Right, adding .call doesn't automatically make an object callable. Thanks for catching that!

But we do want cljs$core$IFn$_invoke$arity$variadic and cljs$lang$maxFixedArity, right? - Herwig

Update 290513: I've implemented generating .cljs$core$IFn$_invoke$arity$ methods and .cljs$lang$applyTo along with .-cljs$lang$maxFixedArity and a wrapper for .call. Surprisingly, this seems to slow down IFn invokations slightly, at least on my machine, despite the generated code looking pretty optimal: and - Herwig

The slowdown issue has been resolved by type hinting. An area I want to explore is the possibility of using a single .call method for all fns (containing a switch and pulling the arity off this). - Herwig

Method names

Should the method names be written as symbols instead of keywords? extend has keywords, but it's a function.

specify might become a function on clojurescript if and when cljs gets reflection, which it probably won't.

Still, other dynamic hosts, that have reflection built in might profit, if we stick to keywords now.

there's also the added benefit of specifying maps of fns at the macro level and merging them there. This one is probably worth asking the community about once the proposal is more fully baked - David

Protocol method calling convention

Right now there is a special case, where IFn$_invoke is called without the otherwise usual first this parameter. I will explore the possibility to remove the redundant first parameters from other protocol methods aswell. Specify will need to introduce the binding via this-as or a wrapper function in the case of external implementations. - Herwig



I'm willing to implement the optimization for cost 1: to pull arity methods from the method-fn, if available. That would leave the overhead of the type dispatch(es) when invoking specify %2B a couple of redundant fn objects per specify call site.

I don't see why we need type dispatches, redundant fns for inline specifications do not add any more cost than what we're currently paying - which is nonexistent as far as I can tell from benchmarking - David

By type dispatch I mean checking for arity-methods on method-fn / other type specific optimisations. Currently extend-type just generates the arity-methods. A full fn would also generate an arity dispatch and apply that will never be used by specify. - Herwig

Right we should probably avoid generating the full fn - David

My point is that we don't generate the full fn, the user does:

- Herwig

Yes, I think we should special case this since in this case that the fn arities map back to the protocol, there's no need for the wrapper. - David


Both could be thought of as confined costs, but I would like to make sure that they won't be show-stoppers, if nobody gets around to figure out how the specify macro could interact with the analyzer in such a way that they are removed. 

analyzer/resolve-existing-var is all you need, this will contain all the known information about the fn, arities, max-arity, whether it's variadic etc - David

Does this also work for lexically bound names? What about expressions? Can I analyze them from a macro? - Herwig

It does work for lexically bound names though I'm not sure why you need that for specify. Expression themselves don't generally have any information. I don't see why you couldn't analyze from a macro, you're passed the environment so I think you should be able to call analyzer from the macro yes. - David


UPDATE 06052013-3 - Herwig

Most major decisions seem to be ironed out. I'll start with the ground work for:

  1. Implementing protocol methods with the arity-methods of a regular fn
  2. Emitting arity-dispatch for .call outside of cljs.compiler/emit for use in specify!

There is still time to chime in on the design; it might take a week or two before I can allocate another time block like this to start pushing patches

Proposal Description of prototype implementation as part of refactoring: CLJS-414

The operation to extend an instance will be called specify in the next proposal and have an interface similar to deftype. See discussion at CLJS-398


UPDATE 06052013

The refactoring done as part of this implementation will be rebased to current master with names specify and specify* set to private, until the design for specify is fully fleshed out.

Outline of the implementation

extend-type now expands to specify on the .prototype property.

specify expands to specify* of protocols and method-arity-maps with fn expressions.

Since the whole protocol/type stack in clojurescript is based on modifying the prototype anyway, the patches from CLJS-414 basically just pull out the redundancies and label them.

Kernel operation: specify*

specify* is loosely based on clojure.core/extend, in that it takes maps of implementing closures.


specify* is needed, because currently CLJS doesn't lift lambdas, so new function objects are allocated every time specify is evaluated. This is fine for specifying prototypes or configuration-object-like protocol instances. It is not so fine for specifying 1000 objects to take part in IEquiv and IHash.

Differences from extend

  • It is a macro
    • bootstrapping: takes a PersistentMap parameter
    • gclosure minification %2B no reflection layer: clojurescript doesn't know how to access name mangled protocol fields
  • It takes method names as symbols
    • by the time we have an optional reflection layer there might be a cljs.reflect/specify with same interface as extend
  • Takes arities in addition to protocol and method name: (specify* o P {-m {1 m-impl-1, 3 m-impl-3}})
    • OK for a low-level user visible operation??
    • we allow users to implement only parts of a protocol anyway
    • use single-arity fast path
    • no need to unpack single-arity impls from multi-arity fns, which breaks down if user passes javascript fn
    • abstracts protocol name mangling: essentially a setter generator

extend-type refactored: specify

  • Takes fn bodies like extend-type and instantiates them
  • extend-type now just expands to a specify on the .prototype property
  • 'Object' pseudo protocol lives here




  • specify* can't-be used with IFn in the current patchset to CLJS-414 as of writing 
  • specify expects to see each method only once, should error when user tries to do
  • Is specify* too low level?
    • Are users that just want the methods reused better served by defspecifyer with deftype syntax?
    • Might just be a temporary solution until we get lambda lifting anyway
    • Are there other use cases?
  1. May 04, 2013

    Why not `specify` and `specify!` one which clones its target and one which does not. `specify` does mean that we'll likely want a cloning protocol.

    I see now why `specify*` takes the map form, since the user can provide an existing function. But I don't see why `specify*` needs the arity information. If it's a real fn form you can look at the arglist. If it's an existing ClojureScript fn you can use the analyzer to figure this out. If it's a JavaScript fn it's a cost you should be OK w/ paying.

    As far as the justification, wanting specify to take previously defined fns, makes sense: performance numbers:

    However it might be worth putting a little more thought into how we could allow people to do specify implementations inline w/o taking or minimizing the performance hit. We can't use the same trick we do with `reify` and perhaps in the end anything we come up with may be too ugly, but probably worth outlining here that we at least considered it thoroughly.

  2. May 04, 2013

    Here's benchmark illustrating an approach that caches the fns on the namespace and add locals as a property to avoid the issues around creating the closure around the implementations once:

    V8 fares well but not so for the other browsers.

  3. May 05, 2013

    My actual train of thought for the arities was:

    • One fn per arity is always generated by the compiler
    • protocol calls are already arity-dispatched, so we would want to call the generated thunks
    • at this point the full fn just acts as a container for its arity methods
    • we can save the allocation of this container by passing the arities directly

    I guess you could say, that saving the allocation of a single object is not worth dictating the design of even the lowest level public functionality. To which I would say: I don't know

    A more practical issue is the added complexity of pulling the arities off an fn

    Do you think it's worthwile to go forward with such a design?

    Thanks for the benchmark!

    1. May 05, 2013

      I see no good justification for explicit arities, it will be more work to implement yes. But specify! and specify are both something that people will want to use - we need to present a sensible interface.

      Related, specify! and specify should both support Object methods and IFn. In the case of IFn - adding call and apply methods was always a bit of a hack. It would be better to add the specific cljs$core$IFn$_invoke$arity$N properties as well as clj$lang$applyTo. Doing this will make using collections as fns considerably more efficient which is an optimization I've been meaning to look into for some time.

      I also think we should follow the extend syntax very closely.

  4. May 05, 2013

    One thing that you haven't brought up is how this will interact with advanced compilation? Closure won't know anything about the objects coming via JSONP or whatever so likely hood of clobbering an existing property name isn't small.

  5. May 06, 2013

    Again, thanks for the input. I'd have completely missed the advanced compilation issue.

    I've updated the page with issue descriptions and a new proposal.

    I had noted the IFn hack, but didn't want to change anything while refactoring and before your input in case it was on purpose.

    I had used symbols instead of keywords in the original proposal to signify that specify is a macro. Do you think it's still better to follow extend's syntax in that?

  6. May 29, 2013

    I updated my specify branch:

    cljs$core$IFn$_invoke, cljs$lang$applyTo aswell as a .call wrapper are now generated directly, when specifying an IFn.

    Even though the generated code is optimized pretty well by closure, a quick benchmark of using collections as functions shows a slight slowdown compared to current master.

    I'm not sure why that happens; I've created a gist for review, with the generated advanced optimized code, pretty printed: