Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [hwloc-devel] tarball growing
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2010-09-28 06:40:14


Eh. Other than not liking it, is there a *problem* with the tarball getting larger?

We could also make 2 tarballs if you really care - one with the tests and one without.

Sent from my PDA. No type good.

On Sep 28, 2010, at 2:28 AM, "Brice Goglin" <Brice.Goglin_at_[hidden]> wrote:

> The bz2 tarball of hwloc 1.0.2 was 2.1MB. hwloc 1.1 will be at least
> 2.7MB. I know that bandwidth is free, but I am still not confortable
> with the size increasing that much.
>
> Obviously, the problem comes from tarballs under tests/linux:
>
> 605774 28 sept. 08:12 tests/linux/256ppc-8n8s4t.tar.gz
> 311293 28 sept. 08:12 tests/linux/256ppc-8n8s4t-nocache.tar.gz
> 232021 28 sept. 08:12 tests/linux/256ppc-8n8s4t-nosys.tar.gz
> 81856 28 sept. 08:12 tests/linux/48amd64-4d2n6c-sparse.tar.gz
> 72906 28 sept. 08:12 tests/linux/96em64t-4n4d3ca2co.tar.gz
> 72839 28 sept. 08:12 tests/linux/128ia64-17n4s2c.tar.gz
> 53033 28 sept. 08:12 tests/linux/64fake-4n2s2ca2c2t.tar.gz
> 41148 28 sept. 08:12 tests/linux/256ia64-64n2s2c.tar.gz
> 29786 28 sept. 08:12 tests/linux/8em64t-4c2t.tar.gz
> 23947 28 sept. 08:12 tests/linux/256ia64-64n2s2c.tar.gz.output
> 20391 28 sept. 08:12 tests/linux/256ppc-8n8s4t-nocache.tar.gz.output
> 20391 28 sept. 08:12 tests/linux/256ppc-8n8s4t.tar.gz.output
>
>
>
> Ideas:
> 1) Remove useless tarballs: Some early machines are very simple and
> likely covered by larger machines added later to the test suite. But
> those early tarballs are small anyway, there's not much improvement to
> expect there.
> 2) The *extracted* size of the tarballs looks mostly related to
> directory sizes (multiple of 4kB), I don't know if we can hope an
> improvement inside the tarball for these, I assume that tar already
> compresses them properly...
> 3) Reduce duplication between variants: Have the test script remove some
> parts of the extracted tarball directly instead of keeping multiple
> variants of the same machines with/without some files. This one should
> help, I'll work on it tonight.
> 4) Remove some useless files from the tarballs. Assuming we know what
> we'll need in the future... there's likely something to remove under the
> topology, cache, and device-tree directories.
> 5) Stop distributing tarballs. having them in SVN is useful. I am not
> sure about this one, maybe wait for some users to actually complain
> about the tarball size :)
> 6) Try to generate some of our huge tarballs manually. Any intern to do
> this? :)
>
> Any other idea?
>
> Brice
>
> _______________________________________________
> hwloc-devel mailing list
> hwloc-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel