We need to make it easier for people to contribute to the Clojure project
Clojure's approach to contributed libraries has some limitations that are based upon legacy aspects of the free hosting services the project has utilized thus far.
Two new things open the door to a better system:
- Github now has the organization notion, and Clojure is now a github organization
- We now have self-hosting hardware for things like Confluence and Jira
The problem and how to solve it
- People are resistant to participate in clojure.contrib
- Lack of modularity
- Shared repo
- CA
- Non-github workflows (no pulls)
- No ego stroke like http://github.com/me-me-me/my-lib
- We can address 1 & 2 by
- Allowing contrib libs as individual repos under the Clojure github organization
- These will still be Clojure project destinations according to the CA
- Project owners and helpers will get commit rights
- Much more independence on releases etc.
- For many, it will simply mean using a Clojure repo instead of a personal one
- As well as adopting Clojure's stewardship policy
- And build/test guidelines?
- Having things start and grow there will greatly facilitate moving things into Clojure core libs or including in Clojure releases
- Curation will remain an issue
- We want to accept people's early half-baked ideas
- But may never become fully baked
- Consumers will need some help sorting out the good stuff
- Not really different from separate libs on github in this area
- Use same metrics for evaluation
- Community adoption
- Personal recommendations
- Frequency of updates/fixes/releases etc
- But still must combat contrib === stdlib presumption
- We want to accept people's early half-baked ideas
- In addition, we can host a Jira instance for issue/feature/bug tracking and patch reception
- should be able to make sub-projects for each lib?
- This versus Assembla where bug tracking is tied to larger project
- should be able to make sub-projects for each lib?
- Fixing 3 is a non-starter
- It is not broken
- Helping people understand that is an education issue
- We need our own treatise/FAQ as the Sun one is now gone
- Number 4 is off the table for the moment
- Pulls are not that great
- Difficult for people to evaluate as part of screening/approval process
- Can't see what you're getting before you get it
- Leaves a dependency trail on external repos
- Presumed incompatible with the CA
- So, patches remain
- Pulls are not that great
- Number 5
- I wonder how many people will admit to this aspect of Github
Moving forward
While we will eventually want to break up contrib, I think we should experiment with some new libs first. I think cemerick's nrepl or the not-yet-contrib match library might be good candidates. I've spoken to cemerick and he's on board.
Ready To Do
, possibly add separate page for owners)
Decide on naming convention for contrib repos
- Stand up a Jira instance.
- Will we need separate Jira/Confluence user ids? I hope not
- Might need to use LDAP
- http://www.opends.org/
- doesn't do much for Jira
- Or beg for Open source free license for Crowd
- or suck it up, people have Jira/Confluence/FishEye logins - ugh
- or:
- Might need to use LDAP
- Investigate sub-project support.
- Will we need separate Jira/Confluence user ids? I hope not
- FAQ for CA (Rich?)
- I've attached a copy of Sun's CA FAQ to this page
- FAQ for patches vs pulls (Rich?)
- Guidelines for lib owners (review and update Guidelines for Clojure Contrib committers, possibly add separate page for owners)
- build/test guidelines: you must use maven or leiningen. When in doubt, meet at the pom.
- Intra-contrib dep policy: allowed, use maven/leiningen
- External dep policy: allowed, use maven/leiningen.
- not so fast, we'll at least want to restrict to certain licenses
- Note that internal deps (and especially external deps), while permitted, greatly reduce the ability of the lib to ever be promoted into Clojure.
Naming convention for contrib repos
- github.com/clojure/prefix.name = clojure.prefix.name package
- prefix can be data/tools/math/core/java/io
Decisions Needed
- Test lib candidates
- nrepl (http://github.com/cemerick/nREPL)
- Unifycle (http://fogus.me/fun/unifycle/)
- finger trees
- match (http://github.com/dcolthorp/matchure)
- After test candidates, Plan for moving existing contrib libs (talk to Sierra)
- What's missing from this plan?
- Are there stories we can tell in time to promote the conj?
Issues
- What is the Jira/git story?
- Use FishEye (not the outdated git plugin for Jira)
- http://www.atlassian.com/software/fisheye/
- We can use FishEye for free too as open source project
Labels:
11 Comments
Hide/Show CommentsSep 28, 2010
Rob Sanheim
What about creating a clojure-contrib org, and then naming things like: github.com/clojure-contrib/rockets, implying ppackage clojure.contrib.rockets?
Sep 28, 2010
Rich Hickey
It's possible, but less desirable as the point here is to reduce the number of top level projects by taking advantage of the new nesting options. In particular, each top-level org needs its own membership management, whereas creating groups within an org is easy. Getting down to 2 membership lists (Clojure github org and confluence/jira) would be a huge plus vs our current 6-8.
Sep 30, 2010
Rob Sanheim
One comment regarding auth: I'm pushing for Crowd to link up JIRA and confluence because Contegix already has a lot of experience with this solution and I know they can get it setup pretty easily. I think anything else would require custom work and maybe even time and work from someone on our side.
Current status on this is that I'm emailing back and forth with Atlassian to get the open source licenses setup -- hope to have those soon.
Oct 02, 2010
Chouser
This holds for anyone that has a CA but is not a committer, right? Anyone with commit privs may commit changes directly to the repos where they have that priv, assuming the change doesn't require the kind of discussion that happens on tickets? That is roughly the current contrib policy I think, I just want to confirm it.
Oct 02, 2010
Chas Emerick
Related to that, can project owners grant commit privs to other CAified contributors? That would go a long way towards alleviating patch-related pains.
Oct 08, 2010
Rich Hickey
No, that would be transferring overall Clojure stewardship. Not all people with CAs are entrusted with stewardship of the project (as individual library owners are to an extent). However, if you have a frequent contributors/partners that I trust, it will be no problem for me to add them as committers on your project at your request.
Oct 08, 2010
Rich Hickey
Yes. Committers can go straight in with edits of their own authorship.
Oct 02, 2010
Chas Emerick
Some miscellaneous thoughts:
Excellent new direction :-D
Oct 08, 2010
Rich Hickey
1) We're still thinking about that. I think we'd all like it to be mostly a build/deploy configuration change and no other. There are several different scenarios, only some of which require a build dependency (e.g. if the Clojure compiler started using the unification lib). Many others will just be the act of 'blessing' a library, and possibly shipping it, while not changing how it is built or where it lives.
2) Dunno about this. Good ideas from maven experts welcome.
3) I decide. The criteria is roughly what it always was - the library has to be something (potentially) generally useful for inclusion in or with Clojure. The community can certainly voice their opinions about what would be usefully included, but special-purpose things (like bridges to particular dbs) are probably not in scope. With the new setup and some ability to inform the community how contrib works, it should be possible to accept libraries in their infancy and let people try things, as well as get more hands involved earlier. If the library has been developed externally to some point, it has to be the work of a sole author with a CA. I have to trust the principle author and all committers with stewardship.
4. Again, expert help welcome. I don't think Nexus is on the agenda yet, possibly due to none of us knowing much about it :)
Oct 02, 2010
Chas Emerick
I cringe just a little at prefab functional categorizations such as the proposed naming conventions: would an R-Tree implementation go under the 'data' or 'math' prefix, etc. Perhaps github and/or JIRA have project-level tags that can be used to indicate functional affiliation?
Oct 08, 2010
Rich Hickey
Well, there are already 60 contrib libs in as many or more namespaces. Having all that be flat, with the obvious contention for things like 'graph' is not workable IMO. R-Trees are data, and if you guess wrong, you just look in the next likely place.