Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] git / nightly builds
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-09-28 07:30:31


On Sep 28, 2013, at 4:41 AM, Brice Goglin <Brice.Goglin_at_[hidden]> wrote:

> I am having lots of problems when switching the regression testing stuff
> (jenkins) to git, mostly because of make dist. Actually it worked 2 days
> ago, no it breaks because hwloc-unknown.tar.* remain after make distcheck.

This means the versions junk still isn't right yet. :-\

> 1) There's something I don't understand in the dist scripts.
> config/distscript.csh is only called the top-level Makefile.am, with 4th
> argument = HWLOC_SVN_R, which is never set. So we always fallback to
> git-describe. When building from a tarball, you get "unknown". That
> seems to break make distcheck. We need to pass something in that
> argument, but I don't see what.

Good catch; sorry about that. HWLOC_SVN_R no longer exists (as you noted). I just removed that 4th argument to distscript.csh. Now, distscript (on master and v1.7) only edits VERSION if snapshot=1 and snapshot_version is empty (i.e., if you do "make dist" in a git clone instead of running contrib/nightly/make_nightly_snapshot, which will edit VERSION before running "make distcheck").

> 2) VPATH dist support is more fragile than before (I always build under
> $srcdir/build). In the past, we could do a VPATH make dist by just
> symlinking srcdir/doc/doxygen-doc to builddir/doc/doxygen-doc. This now
> generates "unknown" tarballs because we check for .git existence
> explicitly. I fixed my own case by checking for ../.git as well but
> that's not satisfying.

Mmm. I preserved your ../.git check in https://github.com/open-mpi/hwloc/commit/c2b7f3d3c713feadb1d2c5a7ccd747cb6673d249.

I don't think I knew/realized you were doign VPATH dist builds by doing the sym link.

> It looks like we can fix this by checking for $srcdir/.git instead. If
> we want VPATH support where $builddir isn't a child of $srcdir, we'll
> have to set GIT_DIR=$srcdir/.git before calling git-describe too.
>
> I think this is becoming way too complicated. Nobody won't be able to
> maintain that code in a couple years. Worse, what if you leave Cisco and
> stop working on hwloc one day? In my other projects, I would just
> override the VERSION makefile variable on the make command-line to
> change the tarball name (you won't get that VERSION in lstopo --version,
> but you would still see 1.8a1 from configure). We should rethink what we
> actually need here.

Yes, these are good points. I agree: the system is too complicated right now. :-\

Let's go through the use cases of what we want:

1. "make dist" in a developer's git clone. This should make a "hwloc-<git describe>.tar.*.

2. make a nightly snapshot tarball. The more I think about this, the more I think it's the same as #1, right?

3. make a release tarball, "hwloc-major.minor.releasegreek.tar.*".

Are these the three (or two, if #2 is the same as #1) use cases we want? If so, I can see about making the code simpler -- e.g., I didn't know about overriding the VERSION Makefile macro trick...

> For instance if we can simpify things by not
> building 1.8-final when we build 1.8-rcX, that'd be fine with me.

I think that part is actually fairly simple; it just runs "make dist", strips out the greek value from VERSION, and runs "make dist" again.

> Random other questions:
> * where do you configure commit emails? can we drop/change the
> open-mpi/hwloc subject prefix? I removed the hwloc-svn prefix from
> mailman to avoid double-prefixing for now
> * can we get commit diffs in email, with some truncation limit to 50kB
> or so?

Yeah, I'm a bit disappointed by the github email service hook (the config is here: https://github.com/open-mpi/hwloc/settings/hooks; scroll down to "Email"). There's *very* little configuration available:

- the address to send to
- an email address secret
- what address to send from

There's no option for diffs (!), and no option to customize the mail/subject. :-\

Do you have a favorite git commit emailing script? We can probably use the generic github "WebHook URLs" hook (at the top of the list) and host the diff-emailing script at IU (or anywhere, actually).

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/