Skip to end of metadata
Go to start of metadata

Problem Statement

It is easy to change the values of compile-time vars like *warn-on-reflection* and *unchecked-math* from a particular point in a file until the end of the file with set!.  You can also change the value of one of these vars for one top-level form by setting it to a desired value before the top-level form, and then setting its value back to the original after that top-level form.

However, there are cases where it would be desirable to change the value of such a var for an individual expression.

  1. For example, you may want *unchecked-math* to be true for part of a function's definition, either a few individual subexpressions, or a larger part of it.
  2. Another is that you want *warn-on-reflection* to be true for most of your library, but there are a few instances of reflection that you are aware of and either they cannot be eliminated, or they are rare enough in practice that they are not a performance problem.  You want to mark the expressions using reflection so that they do not cause a compile-time warning, but you want such warnings enabled everywhere else in that same function in case future changes introduce a new use of reflection.

Future plan that I consider the biggest bang-for-the-buck for this idea: In Clojure core and all modular contrib libraries, make it the default to build with reflection warnings enabled.  After type-hinting the ones away that can be easily and safely done so, wrap the rest of the expressions causing the warnings in (known-reflection ...) so they no longer cause warnings, but are clearly marked in the source code as causing reflection.  From that point forward, every time a developer adds code that causes a new reflection warning, they will find out on the very next build attempt, and it will likely get fixed or marked with (known-reflection ...) quite soon afterwards.  Such changes in Clojure core and contrib would be done in separate commits after this change was in.

Proposal A: new special form compile-time-let

This is likely the less desirable way to do it.  You can skip it and read Proposal B if you want to get to the good stuff.

Introduce a new special form in Clojure called compile-time-let that takes a vector of bindings, and makes those bindings active in the textual scope of the compile-time-let expression during compile time.  For example:

;; I know it is easy to eliminate reflection in this simple example.
;; Imagine a case where it was impossible or undesirable to eliminate reflection.

(defn recip [x]
(if (ratio? x)
(compile-time-let [*warn-on-reflection* false]
(/ (.denominator x) (.numerator x)))
(/ 1 x)))

The following macro known-reflection could also be added to shorten disabling *warn-on-reflection*, if that became a common case:

(defmacro known-reflection [& body]
`(compile-time-let [~'*warn-on-reflection* false]

With such a macro the definition of recip above could be rewritten:

(defn recip [x]
(if (ratio? x)
(known-reflection (/ (.denominator x) (.numerator x)))
(/ 1 x)))

A proof-of-concept implementation of compile-time-let for Clojure on the JVM has been added as an attachment patch1.txt to this page.  Select "Attachments" under the Tools menu near the top right of the page to see it.

If you want to change *warn-on-reflection* or *unchecked-math* for an entire file, the current method of adding a line like (set! *warn-on-reflection* true) near the top of the file continues to work, and is the recommended method to do so.

Proposal B: metadata annotation on code

This suggested syntax is from Aaron Cohen:

(defn recip [x]
(if (ratio? x)
^{:warn-on-reflection false} (/ (.denominator x) (.numerator x))
(/ 1 x)))
^:warn-on-reflection ^:unchecked-math (%2B 1 2)

The macro known-reflection from proposal A above can be implemented as follows with this idea.  There may be shorter ways, too, but this one is tested to work.

(defmacro known-reflection [& body]
^{:warn-on-reflection false}
(do ~@body)))

In case you are curious, this implementation of known-reflection does not work:

(defmacro known-reflection [& body]
^{:warn-on-reflection false}
`(do ~@body))

Earlier I believed that for my patch metadata-patch1.txt to work, it would require first applying a patch to fix CLJ-865 (select "Attachments" under the Tools menu near the top right of this page to find metadata-patch1.txt).  Further testing has shown that my patch seems to work even without fixing CLJ-865.  I think fixing CLJ-865 is a good idea anyway, but it isn't a prerequisite for this feature as implemented by the current patch.

If you want to change *warn-on-reflection* or *unchecked-math* for an entire file, the current method of adding a line like (set! *warn-on-reflection* true) near the top of the file continues to work, and is the recommended method to do so.


Proposal B is better in several ways: it takes advantage of metadata annotating Clojure code that already exists in the compiler, and extends its use a little bit.  It creates no new special forms.  Source code with the new metadata annotations would compile without errors using older versions of Clojure, although the older Clojure compiler would silently give unintended compile-time behavior.

With Proposal A, new code that used the compile-time-let special form would not compile with older versions of Clojure, unless augmented with a definition of compile-time-let that would not give the desired behavior, but would simply be a "no op" that allows the code to compile.

Open questions

Note: The following open questions could be addressed by a later patch.  The existing patches that are restricted to modifying *unchecked-math* and *warn-on-reflection* are useful without the enhancement below.

Question: Should expression-level modification of *compiler-options* also be allowed, e.g. by implementing the proper behavior for keywords :elide-meta and :disable-locals-clearing?

If so, should it be done with a key :compiler-options whose value is a map, like this?

^{:warn-on-reflection false :compiler-options {:elide-meta [:doc :inline] :disable-locals-clearing true}}
( ... Clojure code here ... )

Or would it be preferable to do it with no :compiler-options keyword at all, like this?

^{:warn-on-reflection false :elide-meta [:doc :inline] :disable-locals-clearing true}
( .... Clojure code here ... )

Regardless of the choice above, I believe the proper behavior would be to pushThreadBindings() a new value of the var *compiler-options* whenever metadata like the above is seen, and the new value of *compiler-options* should be the result of merging its current value with the new key/value pairs.  For example, if *compiler-options* was currently equal to {:elide-meta [:doc]}, then within the following expression:

^{:disable-locals-clearing true}
( ... Clojure code here ... )

the value of *compiler-options* would be {:elide-meta [:doc] :disable-locals-clearing true}.


Discussions on the clojure-dev group:

Thread with subject "Proposal for *warn-on-reflection* to be true by default" begun on Feb 25, 2012:

Thread with subject "Arbitrary subexpressions with custom compiler flags" begun on Mar 20, 2012:

  1. May 28, 2012

    It seems like avoiding additional special forms is a good idea, especially considering all the work going on with alterate implementations. Why do you propose a new special form over the metadata proposal?

  2. May 28, 2012

    This may be exactly the wrong answer, but for one, I was able to figure out how to implement it relatively quickly.  Comments in the thread proposing metadata for this purpose, especially Stuart Sierra's statement "This would be a pretty substantial change to the current compiler, I think" make me wonder whether anyone will exert the effort required to make it happen.

    1. Jun 06, 2012

      Since my earlier reply to Brandon's question, I've learned how to implement the metadata proposal, and have added some examples under "Proposal B" and a patch implementing it.

      1. Jun 07, 2012

        Patch link?

        1. Jun 07, 2012

          It is mentioned in the text in the Proposal B section.  Near the top right of this page, click on the Tools menu, then menu item Attachments.  The patch is called metadata-patch1.txt.

  3. Jul 23, 2012

    Is reflection the primary use case motivating this? I ask because there might be ways to make the problem go away entirely. E.g., have the compiler walk all classes visible to a package and automatically eliminate reflection when there is no ambiguity.

    1. Jul 23, 2012

      It was what motivated me personally.  The current patches allow you to do per-expression changes to *unchecked-math*, too, which might be useful to some people, although I haven't solicited or heard any feedback about how useful people would find that.  Although the attached patch doesn't implement it, the idea could be used to do per-subexpression assignments for the other options now in *compiler-options*, i.e. :elide-meta and :disable-locals-clearing, and such a patch should be able to automatically work for any new keys that might be added to *compiler-options* in the future.

      In your scenario where the compiler automatically eliminated unambiguous reflection, would the behavior be to always warn about cases of ambiguous reflection, to always be silent about it, or to follow the current value of *warn-on-reflection*?  Maybe there would be far fewer cases of ambiguous reflection, but for whatever remains, we are in the same situation as we are today.

    2. Jul 23, 2012

      Out of curiosity, how have you and Rich been able to notice the new uses of reflection in the June and July 2012 commits to Clojure core?  By enabling *warn-on-reflection* by hand in the sections of code with changes?  Enabling it globally in the changed files or in all of Clojure core and doing a diff on the before- and after-patch compiler output?  Looking for them by eye and brain manually?

      1. Jul 27, 2012

        I have used all of those except the diffing. Most are caught by code review, and a few in interactive REPL sessions. 

        1. Jul 27, 2012

          A small comment, then I'll try not to ask about it again for at least a couple of months Smile


          (1) we did something like this enhancement in Clojure

          (2) we then eliminated all reflection warnings in Clojure core, either by type hints, or marking them with (known-reflection ...).  There were around 30 to 40 of them last I checked around Feb 2012, and many can easily be type-hinted away.


          (3) we enabled *warn-on-reflection* by default in all Clojure core builds, e.g. by uncommenting this line in build.xml:

          <!-- <sysproperty key="clojure.compile.warn-on-reflection" value="true"/> -->

          then the prescreening process would automatically detect any proposed patch that introduced new reflection warnings, because it automatically detects any compile time warnings and marks such patches as not ready yet.  The only possible uses of reflection that could be introduced by a patch would be called out explicitly in the source code with (known-reflection ...) or ^{:warn-on-reflection false}.

  4. Jul 27, 2012

    I don't have a strong enough feel on this to push for getting it into 1.5, but I am thinking about it.

  5. Apr 12, 2013

    While interesting, I don't consider this a priority, and it's a big feature to support a small benefit (warn-on-reflection true by default). There may be other ways to achieve that, and it might be practically untenable anyway due to the effect on existing codebases.

    If we get build modes we might be able to do this there

    1. Apr 12, 2013

      By "build modes" do you mean the same thing as "build profiles"?

      1. Apr 12, 2013