Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] git / nightly builds
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-09-29 08:20:31


On Sep 28, 2013, at 4:47 PM, Paul Hargrove <phhargrove_at_[hidden]> wrote:

> I am watching with intense interest, as GASNet will be moving to git on Nov 1.
> Could you give me a pointer to where I can get copies all the scripts that you guys use for nightly tarballs, commit emails and such?
> I'll want to have look after you have all the wrinkles ironed out to your satisfaction.

FWIW, I think we're generally reviewing these scripts mainly because they seem to have become too complicated. Moving to adapt them to use git was probably the catalyst for the review; not really the reason.

But the 3 scripts are located in the hwloc master branch (I just described them in my prior mail):

  config/distscript.csh
  contrib/nightly/make_snapshot_tarball
  contrib/dist/make_dist_tarball

> FWIW: a viewpoint from somebody who has yet to actually try to implement his ideas:
>
> We have PLANNED to script our nightlies to be named with a date stamp and 6 or 8 chars of the SHA1.
> The format would be something like PROJECT-BRANCH-20130928-12abcdef.tar.gz
> Where BRANCH=v<major>.<minor> for the UPCOMING release in most scenarios (but could be a named feature branch like "oshmem")
> That way directory listings would sort by branch and date (simple for mere humans) while the SHA1 would allow fetching the corresponding version from git. The full SHA1 would, of course, also be in a file in the tarball (actual filename TBD).

FWIW: I talked to Dave Goodell about this quite a bit before going with "git describe". I think I mentioned that our prior tarballs used the SVN r number, which therefore made it quite easy to order the tarballs; the git hash obviously doesn't provide this ordering.

"git describe" provides a convenient mechanism, IMHO, and does all the work for you. It tells you exactly how many commits you are beyond the last tag on that branch. hwloc's tags conveniently imply exactly which branch you're on, so it all worked out (i.e., each release series has its own branch -- the 1.7.x releases are on the v1.7 branch, the 1.6.x releases are on the v1.6 branch, and so on).

The only branch that didn't have any tags at all was master, so I just created a "dev" tag on master, and that was sufficient.

Two other minor points contributed to my decision:

1. Dave indicated that both MPICH and other open source projects use the "git describe" scheme for their nightly tarballs
2. Using "git describe", I didn't have to muck with the date, branch, etc.

> I don't think we consider "git describe" to be a useful naming mechanism for nightlies, though for other snapshots it might be useful.
> For RCs, we pan to tag something like "1.7.3RC" at the start of the RC cycle to get "git describe" to give names containing "1.7.3RC-<N>-<hash>" which make some sense at a glance, even though N is incremented with every commit and may grow much higher than a contentional RC number would. Again, "1.7.3" in this example is the UPCOMING release rather than the previous one.
>
> For a developer's "make dist" from a developer's clone, however, I think we agree "git describe" is as good as anything else readily available.
>
> So, in short: I think our plan aligns with yours on scenarios #1 ("make dist" by Jane Developer) and #3 (official releases), but we wanted something more people-friendly for #2 (nightly tarballs).

Fair enough.

That being said, when we tell users to get a nightly tarball (e.g., to get a bug fix), my experience is that they don't know/care about the nightly tarball numbering scheme: they always just get the most recent version.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/