Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] Lustre hints via environment variables/runtime parameters
From: Eric Chamberland (Eric.Chamberland_at_[hidden])
Date: 2012-12-01 08:47:09


Hi,

I am using openmpi 1.6.3 with lustre. I can change the stip count via
"striping_unit" but if I try to change the stripe size via
"striping_factor", all my options are ignored and fall back on the
default values.

Here is what I do:

=============================
setenv ROMIO_HINTS /home/ericc/romio-hints

cat $ROMIO_HINTS
striping_unit 16
striping_factor 1048576

rm temp ; mpirun -n 3 idx2 ; lfs getstripe temp

temp
lmm_stripe_count: 1
lmm_stripe_size: 65536
lmm_stripe_offset: 28
         obdidx objid objid group
             28 23877295 0x16c56af 0

=============================

If I remove the "striping_factor 1048576" entry in my hint file, I get this:

=============================
cat $ROMIO_HINTS
striping_unit 16
#striping_factor 1048576

rm temp ; mpirun -n 3 idx2 ; lfs getstripe temp

temp
lmm_stripe_count: 36
lmm_stripe_size: 65536
lmm_stripe_offset: 21
         obdidx objid objid group
             21 27078098 0x19d2dd2 0
              5 26516786 0x1949d32 0
             18 26272707 0x190e3c3 0
              2 22198271 0x152b7ff 0
             14 24302770 0x172d4b2 0
             16 20970007 0x13ffa17 0
             28 23877307 0x16c56bb 0
              6 25726827 0x1888f6b 0
             31 23623835 0x168789b 0
             23 24231103 0x171bcbf 0
             34 23963185 0x16da631 0
              3 23462711 0x1660337 0
             13 27515658 0x1a3db0a 0
             26 23502238 0x1669d9e 0
              7 26708491 0x1978a0b 0
             32 21946148 0x14edf24 0
             17 26912937 0x19aa8a9 0
              4 24586434 0x17728c2 0
             27 23277776 0x16330d0 0
              9 23634614 0x168a2b6 0
             11 25769779 0x1893733 0
             33 24915767 0x17c2f37 0
             29 20790315 0x13d3c2b 0
              8 25647332 0x18758e4 0
             20 26938873 0x19b0df9 0
             19 26182463 0x18f833f 0
             12 25346469 0x182c1a5 0
             15 25681819 0x187df9b 0
             24 23898261 0x16ca895 0
             10 26554081 0x1952ee1 0
             25 23512409 0x166c559 0
              0 28428909 0x1b1ca6d 0
             30 23953009 0x16d7e71 0
             22 24117691 0x17001bb 0
             35 20972494 0x14003ce 0
              1 25492821 0x184fd55 0
=============================

And if I don't put anything in the file, I get this:

=============================
cat $ROMIO_HINTS
#striping_unit 16
#striping_factor 1048576

rm temp ; mpirun -n 3 idx2 ; lfs getstripe temp

temp
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 18
         obdidx objid objid group
             18 26272802 0x190e422 0

=============================

which are the default values of our lustre.

Any idea?

Thanks,

Eric