Skip to end of metadata
Go to start of metadata

We need to make it easier for people to contribute to the Clojure project

Clojure's approach to contributed libraries has some limitations that are based upon legacy aspects of the free hosting services the project has utilized thus far.

Two new things open the door to a better system:

  • Github now has the organization notion, and Clojure is now a github organization
  • We now have self-hosting hardware for things like Confluence and Jira

The problem and how to solve it

  • People are resistant to participate in clojure.contrib
    1. Lack of modularity
    2. Shared repo
    3. CA
    4. Non-github workflows (no pulls)
    5. No ego stroke like http://github.com/me-me-me/my-lib
  • We can address 1 & 2 by
    • Allowing contrib libs as individual repos under the Clojure github organization
    • These will still be Clojure project destinations according to the CA
    • Project owners and helpers will get commit rights
    • Much more independence on releases etc.
    • For many, it will simply mean using a Clojure repo instead of a personal one
      • As well as adopting Clojure's stewardship policy
      • And build/test guidelines?
      • Having things start and grow there will greatly facilitate moving things into Clojure core libs or including in Clojure releases
    • Curation will remain an issue
      • We want to accept people's early half-baked ideas
        • But may never become fully baked
      • Consumers will need some help sorting out the good stuff
        • Not really different from separate libs on github in this area
        • Use same metrics for evaluation
          • Community adoption
          • Personal recommendations
          • Frequency of updates/fixes/releases etc
        • But still must combat contrib === stdlib presumption
    • In addition, we can host a Jira instance for issue/feature/bug tracking and patch reception
      • should be able to make sub-projects for each lib?
        • This versus Assembla where bug tracking is tied to larger project
  • Fixing 3 is a non-starter
    • It is not broken
    • Helping people understand that is an education issue
      • We need our own treatise/FAQ as the Sun one is now gone
  • Number 4 is off the table for the moment
    • Pulls are not that great
      • Difficult for people to evaluate as part of screening/approval process
      • Can't see what you're getting before you get it
      • Leaves a dependency trail on external repos
    • Presumed incompatible with the CA
    • So, patches remain
  • Number 5
    • I wonder how many people will admit to this aspect of Github

Moving forward

While we will eventually want to break up contrib, I think we should experiment with some new libs first. I think cemerick's nrepl or the not-yet-contrib match library might be good candidates. I've spoken to cemerick and he's on board.

Ready To Do

, possibly add separate page for owners)

Decide on naming convention for contrib repos

  • Stand up a Jira instance.
  • FAQ for CA (Rich?)
    • I've attached a copy of Sun's CA FAQ to this page
  • FAQ for patches vs pulls (Rich?)
  • Guidelines for lib owners (review and update Guidelines for Clojure Contrib committers, possibly add separate page for owners)
    • build/test guidelines: you must use maven or leiningen. When in doubt, meet at the pom.
    • Intra-contrib dep policy: allowed, use maven/leiningen
    • External dep policy: allowed, use maven/leiningen.
      • not so fast, we'll at least want to restrict to certain licenses
    • Note that internal deps (and especially external deps), while permitted, greatly reduce the ability of the lib to ever be promoted into Clojure.

Naming convention for contrib repos

  • github.com/clojure/prefix.name  = clojure.prefix.name package
  • prefix can be data/tools/math/core/java/io

Decisions Needed

Issues

Labels:
None
Edit Labels

11 Comments

Hide/Show Comments

  1. Sep 28, 2010

    Rob Sanheim

    What about creating a clojure-contrib org, and then naming things like: github.com/clojure-contrib/rockets, implying ppackage clojure.contrib.rockets? 

    1. Sep 28, 2010

      Rich Hickey

      It's possible, but less desirable as the point here is to reduce the number of top level projects by taking advantage of the new nesting options. In particular, each top-level org needs its own membership management, whereas creating groups within an org is easy. Getting down to 2 membership lists (Clojure github org and confluence/jira) would be a huge plus vs our current 6-8.

  2. Sep 30, 2010

    Rob Sanheim

    One comment regarding auth: I'm pushing for Crowd to link up JIRA and confluence because Contegix already has a lot of experience with this solution and I know they can get it setup pretty easily.  I think anything else would require custom work and maybe even time and work from someone on our side.  

    Current status on this is that I'm emailing back and forth with Atlassian to get the open source licenses setup -- hope to have those soon.

  3. Oct 02, 2010

    Chouser

    • You must accept contributions only via patches submitted to the tracking system (i.e. no pulls)

    This holds for anyone that has a CA but is not a committer, right?  Anyone with commit privs may commit changes directly to the repos where they have that priv, assuming the change doesn't require the kind of discussion that happens on tickets?  That is roughly the current contrib policy I think, I just want to confirm it.

    1. Oct 02, 2010

      Chas Emerick

      Related to that, can project owners grant commit privs to other CAified contributors?  That would go a long way towards alleviating patch-related pains.

      1. Oct 08, 2010

        Rich Hickey

        No, that would be transferring overall Clojure stewardship. Not all people with CAs are entrusted with stewardship of the project (as individual library owners are to an extent). However, if you have a frequent contributors/partners that I trust, it will be no problem for me to add them as committers on your project at your request.

    2. Oct 08, 2010

      Rich Hickey

      Yes. Committers can go straight in with edits of their own authorship.

  4. Oct 02, 2010

    Chas Emerick

    Some miscellaneous thoughts:

    1. What happens to a contrib library's JIRA, wiki, repo, etc. when it's brought into core: merged into core's resources in whole or simply linked in?  The latter would be far preferable IMO.  Even after a lib is promoted, its former leads are almost certainly still the best people to continue its maintenance as well as work on "next generation" versions that wouldn't have the backwards compatibility guarantees/restrictions that being in core implies.  This would imply something like a multimodule build process for core (something Rich may retch at).
    2. How does Leiningen fare within the scope of an integrated build process?  Perhaps it's time for a lein-maven-plugin, or maybe contrib's hudson instance's default parent POM could define a lein profile that bound execs of lein's clean, test, and jar targets to the appropriate maven phases.  Another option would be to push hard for lein to emit useful pom.xml files.
    3. Who gets to start a new project under the contrib umbrella, who makes that decision, and by what criteria?
    4. I've heard rumors of a Nexus instance going up somewhere (at least in part as a potential replacement for the clojars repo), perhaps adjacent to the JIRA / conf / fisheye installs?  If so, perhaps having separate nexus repos for "blessed" libraries is another good way to indicate a minimum degree of quality, etc. (1.x.x+ versions being another route that I could have sworn was mentioned on this page earlier....?)
      1. On a related note, getting clojure and contrib artifacts (at least "blessed" ones) sync'ed into maven central automagically would be fabulous.  That may or may not imply releasing to Sonatype's OSS nexus (in addition to Clojure/core's nexus)?

    Excellent new direction :-D

    1. Oct 08, 2010

      Rich Hickey

      1) We're still thinking about that. I think we'd all like it to be mostly a build/deploy configuration change and no other. There are several different scenarios, only some of which require a build dependency (e.g. if the Clojure compiler started using the unification lib). Many others will just be the act of 'blessing' a library, and possibly shipping it, while not changing how it is built or where it lives.

      2)  Dunno about this. Good ideas from maven experts welcome.

      3) I decide. The criteria is roughly what it always was - the library has to be something (potentially) generally useful for inclusion in or with Clojure. The community can certainly voice their opinions about what would be usefully included, but special-purpose things (like bridges to particular dbs) are probably not in scope. With the new setup and some ability to inform the community how contrib works, it should be possible to accept libraries in their infancy and let people try things, as well as get more hands involved earlier. If the library has been developed externally to some point, it has to be the work of a sole author with a CA. I have to trust the principle author and all committers with stewardship.

      4. Again, expert help welcome. I don't think Nexus is on the agenda yet, possibly due to none of us knowing much about it :)

  5. Oct 02, 2010

    Chas Emerick

    I cringe just a little at prefab functional categorizations such as the proposed naming conventions: would an R-Tree implementation go under the 'data' or 'math' prefix, etc.  Perhaps github and/or JIRA have project-level tags that can be used to indicate functional affiliation?

    1. Oct 08, 2010

      Rich Hickey

      Well, there are already 60 contrib libs in as many or more namespaces. Having all that be flat, with the obvious contention for things like 'graph' is not workable IMO. R-Trees are data, and if you guess wrong, you just look in the next likely place.