Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] one more finding in openmpi-1.7.5a1
From: tmishima_at_[hidden]
Date: 2014-02-14 00:40:18


Thanks. I'm not familiar with mindist mapper. But obviously
checking for ORTE_MAPPING_BYDIST is missing. In addition,
ORTE_MAPPING_PPR is missing again by my mistake.

Please try this patch.

if OPAL_HAVE_HWLOC
            } else if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_CORE);
            } else if (ORTE_MAPPING_BYL1CACHE == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L1CACHE);
            } else if (ORTE_MAPPING_BYL2CACHE == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L2CACHE);
            } else if (ORTE_MAPPING_BYL3CACHE == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L3CACHE);
            } else if (ORTE_MAPPING_BYSOCKET == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SOCKET);
            } else if (ORTE_MAPPING_BYNUMA == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NUMA);
            } else if (ORTE_MAPPING_BYBOARD == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_BOARD);
            } else if (ORTE_MAPPING_BYHWTHREAD == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_HWTHREAD);
            } else if (ORTE_MAPPING_PPR == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
            } else if (ORTE_MAPPING_BYDIST == ORTE_GET_MAPPING_POLICY
(mapping)) {
                ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
#endif

Tetsuya Mishima

> Hi,
> after this patch we get this in jenkins:
>
> 07:03:15 [vegas12.mtr.labs.mlnx:01646] [[26922,0],0] ORTE_ERROR_LOG: Not
implemented in file rmaps_mindist_module.c at line 39107:03:15
[vegas12.mtr.labs.mlnx:01646] [[26922,0],0] ORTE_ERROR_LOG: Not
> implemented in file base/rmaps_base_map_job.c at line 285
>
>
>
>
>
> On Fri, Feb 14, 2014 at 6:35 AM, <tmishima_at_[hidden]>wrote:
>
>
> Sorry, one more shot - byslot was dropped!
>
>     if (NULL == spec) {
>         /* check for map-by object directives - we set the
>          * ranking to match if one was given
>          */
>         if (ORTE_MAPPING_GIVEN & ORTE_GET_MAPPING_DIRECTIVE(mapping)) {
>             if (ORTE_MAPPING_BYSLOT == ORTE_GET_MAPPING_POLICY(mapping))
{
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
>             } else if (ORTE_MAPPING_BYNODE == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NODE);
> #if OPAL_HAVE_HWLOC
>             } else if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_CORE);
>             } else if (ORTE_MAPPING_BYL1CACHE == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L1CACHE);
>             } else if (ORTE_MAPPING_BYL2CACHE == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L2CACHE);
>             } else if (ORTE_MAPPING_BYL3CACHE == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L3CACHE);
>             } else if (ORTE_MAPPING_BYSOCKET == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SOCKET);
>             } else if (ORTE_MAPPING_BYNUMA == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NUMA);
>             } else if (ORTE_MAPPING_BYBOARD == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_BOARD);
>             } else if (ORTE_MAPPING_BYHWTHREAD == ORTE_GET_MAPPING_POLICY
> (mapping)) {
>                 ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_HWTHREAD);
> #endif
>
> Tetusya Mishima
>
> > I've found it. Please add 2 lines(770, 771) in rmaps_base_frame.c:
> >
> > 747      if (NULL == spec) {
> > 748          /* check for map-by object directives - we set the
> > 749           * ranking to match if one was given
> > 750           */
> > 751          if (ORTE_MAPPING_GIVEN & ORTE_GET_MAPPING_DIRECTIVE
> > (mapping)) {
> > 752              if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
> > (mapping)) {
> > 753                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_CORE);
> > 754              } else if (ORTE_MAPPING_BYNODE ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 755                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NODE);
> > 756              } else if (ORTE_MAPPING_BYL1CACHE ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 757                  ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_L1CACHE);
> > 758              } else if (ORTE_MAPPING_BYL2CACHE ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 759                  ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_L2CACHE);
> > 760              } else if (ORTE_MAPPING_BYL3CACHE ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 761                  ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_L3CACHE);
> > 762              } else if (ORTE_MAPPING_BYSOCKET ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 763                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SOCKET);
> > 764              } else if (ORTE_MAPPING_BYNUMA ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 765                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NUMA);
> > 766              } else if (ORTE_MAPPING_BYBOARD ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 767                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_BOARD);
> > 768              } else if (ORTE_MAPPING_BYHWTHREAD ==
> > ORTE_GET_MAPPING_POLICY(mapping)) {
> > 769                  ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_HWTHREAD);
> > 770              } else if (ORTE_MAPPING_PPR == ORTE_GET_MAPPING_POLICY
> > (mapping)) {
> > 771                  ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
> > 772              }
> >
> > Tetsuya Mishima
> >
> > > You are welcome, Ralph.
> > >
> > > But, after fixing it, I'm facing another problem whin I use ppr
option:
> > > [mishima_at_manage openmpi-1.7.4]$ mpirun -np 2 -map-by ppr:1:socket
> > -bind-to
> > > socket -report-bindings ~/mis/openmpi/demos/m
> > > yprog
> > > [manage.cluster:28057] [[25570,0],0] ORTE_ERROR_LOG: Not implemented
in
> > > file rmaps_ppr.c at line 389
> > > [manage.cluster:28057] [[25570,0],0] ORTE_ERROR_LOG: Not implemented
in
> > > file base/rmaps_base_map_job.c at line 285
> > >
> > > I confirmed it worked when it reverted back.
> > > I'm a little bit confused. Could you take a look?
> > >
> > > Tetsuya Mishima
> > >
> > > > Thanks - these used to be bitmaps, but changed when we started
> getting
> > so
> > > many options. Sadly, they are very rarely used, so bugs like this can
> go
> > > unnoticed for long times. Appreciate you taking such
> > > > a close look at them.
> > > >
> > > > Ralph
> > > >
> > > > On Feb 13, 2014, at 4:55 PM, tmishima_at_[hidden] wrote:
> > > >
> > > > >
> > > > > Hi Ralph,
> > > > >
> > > > > I would report one more finding in openmpi-1.7.5a1.
> > > > >
> > > > > Because ORTE_MAPPING_BY...s are not a bit field expression,
> > > > > at orte_rmaps_base_set_ranking_policy in rmaps_base_frame.c
> > > > > you should not use "&" to compare them:
> > > > >
> > > > >   747      if (NULL == spec) {
> > > > >   748          /* check for map-by object directives - we set the
> > > > >   749           * ranking to match if one was given
> > > > >   750           */
> > > > >   751          if (ORTE_MAPPING_GIVEN &
ORTE_GET_MAPPING_DIRECTIVE
> > > > > (mapping)) {
> > > > >   752              if (ORTE_MAPPING_BYCORE ==
> ORTE_GET_MAPPING_POLICY
> > > > > (mapping)) {
> > > > >   753                  ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_CORE);
> > > > >   754              } else if (ORTE_MAPPING_BYNODE ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   755                  ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_NODE);
> > > > >   756              } else if (ORTE_MAPPING_BYL1CACHE ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   757                  ORTE_SET_RANKING_POLICY(tmp,
> > > ORTE_RANK_BY_L1CACHE);
> > > > >   758              } else if (ORTE_MAPPING_BYL2CACHE ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   759                  ORTE_SET_RANKING_POLICY(tmp,
> > > ORTE_RANK_BY_L2CACHE);
> > > > >   760              } else if (ORTE_MAPPING_BYL3CACHE ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   761                  ORTE_SET_RANKING_POLICY(tmp,
> > > ORTE_RANK_BY_L3CACHE);
> > > > >   762              } else if (ORTE_MAPPING_BYSOCKET ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   763                  ORTE_SET_RANKING_POLICY(tmp,
> > > ORTE_RANK_BY_SOCKET);
> > > > >   764              } else if (ORTE_MAPPING_BYNUMA ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   765                  ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_NUMA);
> > > > >   766              } else if (ORTE_MAPPING_BYBOARD ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   767                  ORTE_SET_RANKING_POLICY(tmp,
> > > ORTE_RANK_BY_BOARD);
> > > > >   768              } else if (ORTE_MAPPING_BYHWTHREAD ==
> > > > > ORTE_GET_MAPPING_POLICY(mapping)) {
> > > > >   769                  ORTE_SET_RANKING_POLICY(tmp,
> > > > > ORTE_RANK_BY_HWTHREAD);
> > > > >   770              }
> > > > >
> > > > > Regards,
> > > > > Tetsuya Mishima
> > > > >
> > > > > _______________________________________________
> > > > > users mailing list
> > > > > users_at_[hidden]
> > > > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > > >
> > > > _______________________________________________
> > > > users mailing list
> > > > users_at_[hidden]
> > > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
>
http://www.open-mpi.org/mailman/listinfo.cgi/users_______________________________________________

> users mailing list
> users_at_[hidden]http://www.open-mpi.org/mailman/listinfo.cgi/users