Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] plpa
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-02-25 10:13:14


This confusion is probably all my fault for taking so long to release
PLPA and get it into OMPI. Sorry about this, guys. :-(

Here's where we are:

- PLPA v1.1 has been released (http://www.open-mpi.org/projects/plpa/)
- I updated the /vendor/plpa branch this morning
- Rather than try to sync up with Sharon for his paffinity/linux
patch, I made a new branch from trunk r17581: /tmp-public/plpa-v1.1.
I brought in the new PLPA v1.1 on that branch using the SVN 3rd party
import methodology.

 From your branch, I'm not sure if we did effectively the same things
(I see that you brought in plpa v1.1, etc.). So I think we can:

1. use my branch and bring in the paffinity and paffinity/linux patch,
then merge that to the trunk
2. use your branch and do something similar (although I'm not sure you
used the SVN 3rd party import stuff or not...?)

Which do you want to do?

On Feb 25, 2008, at 9:52 AM, Jeff Squyres wrote:

> Lenny --
>
> Is this the patch that Sharon was working on? I literally just
> created a new branch for bringing in plpa v1.1. Should I do it on
> your rank_file branch instead?
>
>
> On Feb 25, 2008, at 9:46 AM, lennyve_at_[hidden] wrote:
>
>> Author: lennyve
>> Date: 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> New Revision: 17584
>> URL: https://svn.open-mpi.org/trac/ompi/changeset/17584
>>
>> Log:
>> Added patched files due to PLPA.1.1 API
>>
>> Added:
>> tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
>> tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
>> tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
>> tmp-public/rank_file/opal/mca/paffinity/linux/paffinity_linux.h
>> tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_component.c
>> tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_module.c
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
>> 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> @@ -0,0 +1,53 @@
>> +#
>> +# Copyright (c) 2004-2005 The Trustees of Indiana University and
>> Indiana
>> +# University Research and Technology
>> +# Corporation. All rights reserved.
>> +# Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> +# of Tennessee Research Foundation. All
>> rights
>> +# reserved.
>> +# Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> +# University of Stuttgart. All rights
>> reserved.
>> +# Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> +# All rights reserved.
>> +# Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
>> +# $COPYRIGHT$
>> +#
>> +# Additional copyrights may follow
>> +#
>> +# $HEADER$
>> +#
>> +
>> +SUBDIRS = plpa
>> +
>> +# To find plpa_bottom.h
>> +AM_CPPFLAGS = -I$(top_srcdir)/opal/mca/paffinity/linux/plpa/src/
>> libplpa
>> +
>> +sources = \
>> + paffinity_linux.h \
>> + paffinity_linux_component.c \
>> + paffinity_linux_module.c
>> +
>> +# Make the output library in this directory, and name it either
>> +# mca_<type>_<name>.la (for DSO builds) or libmca_<type>_<name>.la
>> +# (for static builds).
>> +
>> +if OMPI_BUILD_paffinity_linux_DSO
>> +component_noinst =
>> +component_install = mca_paffinity_linux.la
>> +else
>> +component_noinst = libmca_paffinity_linux.la
>> +component_install =
>> +endif
>> +
>> +mcacomponentdir = $(pkglibdir)
>> +mcacomponent_LTLIBRARIES = $(component_install)
>> +mca_paffinity_linux_la_SOURCES = $(sources)
>> +mca_paffinity_linux_la_LDFLAGS = -module -avoid-version
>> +mca_paffinity_linux_la_LIBADD = \
>> + $(top_ompi_builddir)/opal/mca/paffinity/linux/plpa/src/
>> libplpa/libplpa_included.la
>> +
>> +noinst_LTLIBRARIES = $(component_noinst)
>> +libmca_paffinity_linux_la_SOURCES =$(sources)
>> +libmca_paffinity_linux_la_LDFLAGS = -module -avoid-version
>> +libmca_paffinity_linux_la_LIBADD = \
>> + $(top_ompi_builddir)/opal/mca/paffinity/linux/plpa/src/
>> libplpa/libplpa_included.la
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
>> 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> @@ -0,0 +1,34 @@
>> +# -*- shell-script -*-
>> +#
>> +# Copyright (c) 2004-2005 The Trustees of Indiana University and
>> Indiana
>> +# University Research and Technology
>> +# Corporation. All rights reserved.
>> +# Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> +# of Tennessee Research Foundation. All
>> rights
>> +# reserved.
>> +# Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> +# University of Stuttgart. All rights
>> reserved.
>> +# Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> +# All rights reserved.
>> +# Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
>> +#
>> +# $COPYRIGHT$
>> +#
>> +# Additional copyrights may follow
>> +#
>> +# $HEADER$
>> +#
>> +
>> +# MCA_paffinity_linux_CONFIG([action-if-found], [action-if-not-
>> found])
>> +# -----------------------------------------------------------
>> +m4_include(opal/mca/paffinity/linux/plpa/config/plpa.m4)
>> +
>> +AC_DEFUN([MCA_paffinity_linux_POST_CONFIG],[
>> + PLPA_DO_AM_CONDITIONALS
>> +])dnl
>> +
>> +AC_DEFUN([MCA_paffinity_linux_CONFIG],[
>> + PLPA_SET_SYMBOL_PREFIX([opal_paffinity_linux_plpa_])
>> + PLPA_INCLUDED([opal/mca/paffinity/linux/plpa])
>> + PLPA_INIT([$1],[$2])
>> +])dnl
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
>> 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> @@ -0,0 +1,22 @@
>> +# -*- shell-script -*-
>> +#
>> +# Copyright (c) 2004-2005 The Trustees of Indiana University and
>> Indiana
>> +# University Research and Technology
>> +# Corporation. All rights reserved.
>> +# Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> +# of Tennessee Research Foundation. All
>> rights
>> +# reserved.
>> +# Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> +# University of Stuttgart. All rights
>> reserved.
>> +# Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> +# All rights reserved.
>> +# Copyright (c) 2007 Los Alamos National Security, LLC. All
>> rights
>> +# reserved.
>> +# $COPYRIGHT$
>> +#
>> +# Additional copyrights may follow
>> +#
>> +# $HEADER$
>> +#
>> +
>> +PARAM_CONFIG_FILES="Makefile"
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux.h
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/paffinity_linux.h
>> 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> @@ -0,0 +1,69 @@
>> +/*
>> + * Copyright (c) 2004-2005 The Trustees of Indiana University and
>> Indiana
>> + * University Research and Technology
>> + * Corporation. All rights reserved.
>> + * Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> + * of Tennessee Research Foundation. All
>> rights
>> + * reserved.
>> + * Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> + * University of Stuttgart. All rights
>> reserved.
>> + * Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> + * All rights reserved.
>> + * Copyright (c) 2006-2007 Cisco Systems, Inc. All rights reserved.
>> + *
>> + * $COPYRIGHT$
>> + *
>> + * Additional copyrights may follow
>> + *
>> + * $HEADER$
>> + */
>> +
>> +/**
>> + * @file
>> + *
>> + * Processor affinity for Linux.
>> + *
>> + * Linux sucks. There are at least 3 different ways that
>> + * sched_setaffinity is implemented.
>> + *
>> + * Fortunately we have an independent project called Portable Linux
>> + * Processor Affinity (PLPA) which allows us to do processor
>> affinity
>> + * without knowing which flavor of afffinity is installed on the
>> + * system a priori - PLPA does a few probes behind the scenes and
>> + * utilizes the correct syntax to the correct system call to set
>> + * or get processor affinity for us.
>> + *
>> + */
>> +
>> +
>> +#ifndef MCA_PAFFINITY_LINUX_EXPORT_H
>> +#define MCA_PAFFINITY_LINUX_EXPORT_H
>> +
>> +#include "opal_config.h"
>> +
>> +#include "opal/mca/mca.h"
>> +#include "opal/mca/paffinity/paffinity.h"
>> +#include "opal/mca/paffinity/linux/plpa/src/libplpa/plpa.h"
>> +
>> +
>> +#if defined(c_plusplus) || defined(__cplusplus)
>> +extern "C" {
>> +#endif
>> +
>> + /**
>> + * Globally exported variable
>> + */
>> + OPAL_DECLSPEC extern const opal_paffinity_base_component_1_1_0_t
>> + mca_paffinity_linux_component;
>> +
>> +
>> + /**
>> + * paffinity query API function
>> + */
>> + const opal_paffinity_base_module_1_1_0_t *
>> + opal_paffinity_linux_component_query(int *query);
>> +
>> +#if defined(c_plusplus) || defined(__cplusplus)
>> +}
>> +#endif
>> +#endif /* MCA_PAFFINITY_LINUX_EXPORT_H */
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_component.c
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_component.c 2008-02-25 09:46:28 EST (Mon, 25 Feb
>> 2008)
>> @@ -0,0 +1,91 @@
>> +/*
>> + * Copyright (c) 2004-2007 The Trustees of Indiana University and
>> Indiana
>> + * University Research and Technology
>> + * Corporation. All rights reserved.
>> + * Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> + * of Tennessee Research Foundation. All
>> rights
>> + * reserved.
>> + * Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> + * University of Stuttgart. All rights
>> reserved.
>> + * Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> + * All rights reserved.
>> + * Copyright (c) 2007 Cisco, Inc. All rights reserved.
>> + * $COPYRIGHT$
>> + *
>> + * Additional copyrights may follow
>> + *
>> + * $HEADER$
>> + *
>> + * These symbols are in a file by themselves to provide nice linker
>> + * semantics. Since linkers generally pull in symbols by object
>> + * files, keeping these symbols as the only symbols in this file
>> + * prevents utility programs such as "ompi_info" from having to
>> import
>> + * entire components just to query their version and parameters.
>> + */
>> +
>> +#include "opal_config.h"
>> +
>> +#include "opal/constants.h"
>> +#include "opal/mca/paffinity/paffinity.h"
>> +#include "paffinity_linux.h"
>> +
>> +/*
>> + * Public string showing the paffinity ompi_linux component version
>> number
>> + */
>> +const char *opal_paffinity_linux_component_version_string =
>> + "OPAL linux paffinity MCA component version " OPAL_VERSION;
>> +
>> +/*
>> + * Local function
>> + */
>> +static int linux_open(void);
>> +
>> +/*
>> + * Instantiate the public struct with all of our public information
>> + * and pointers to our public functions in it
>> + */
>> +
>> +const opal_paffinity_base_component_1_1_0_t
>> mca_paffinity_linux_component = {
>> +
>> + /* First, the mca_component_t struct containing meta information
>> + about the component itself */
>> +
>> + {
>> + /* Indicate that we are a paffinity v1.1.0 component (which
>> also
>> + implies a specific MCA version) */
>> +
>> + OPAL_PAFFINITY_BASE_VERSION_1_1_0,
>> +
>> + /* Component name and version */
>> +
>> + "linux",
>> + OPAL_MAJOR_VERSION,
>> + OPAL_MINOR_VERSION,
>> + OPAL_RELEASE_VERSION,
>> +
>> + /* Component open and close functions */
>> +
>> + linux_open,
>> + NULL
>> + },
>> + {
>> +
>> + /* The component is checkpoint ready */
>> + MCA_BASE_METADATA_PARAM_CHECKPOINT
>> + },
>> +
>> + /* Query function */
>> +
>> + opal_paffinity_linux_component_query
>> +};
>> +
>> +
>> +static int linux_open(void)
>> +{
>> +
>> mca_base_param_reg_int
>> (&mca_paffinity_linux_component.paffinityc_version,
>> + "priority",
>> + "Priority of the linux paffinity
>> component",
>> + false, false, 10, NULL);
>> +
>> + return OPAL_SUCCESS;
>> +}
>>
>> Added: tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_module.c
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =====================================================================
>> --- (empty file)
>> +++ tmp-public/rank_file/opal/mca/paffinity/linux/
>> paffinity_linux_module.c 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
>> @@ -0,0 +1,185 @@
>> +/*
>> + * Copyright (c) 2004-2005 The Trustees of Indiana University and
>> Indiana
>> + * University Research and Technology
>> + * Corporation. All rights reserved.
>> + * Copyright (c) 2004-2005 The University of Tennessee and The
>> University
>> + * of Tennessee Research Foundation. All
>> rights
>> + * reserved.
>> + * Copyright (c) 2004-2005 High Performance Computing Center
>> Stuttgart,
>> + * University of Stuttgart. All rights
>> reserved.
>> + * Copyright (c) 2004-2005 The Regents of the University of
>> California.
>> + * All rights reserved.
>> + * Copyright (c) 2006-2007 Cisco Systems, Inc. All rights reserved.
>> + *
>> + * $COPYRIGHT$
>> + *
>> + * Additional copyrights may follow
>> + *
>> + * $HEADER$
>> + */
>> +
>> +#include "opal_config.h"
>> +
>> +/* This component will only be compiled on Linux, where we are
>> + guaranteed to have <unistd.h> and friends */
>> +#include <stdio.h>
>> +
>> +#include <unistd.h>
>> +#include <stdlib.h>
>> +#include <string.h>
>> +#include <errno.h>
>> +
>> +#include "opal/constants.h"
>> +#include "opal/mca/base/mca_base_param.h"
>> +#include "opal/mca/paffinity/paffinity.h"
>> +#include "opal/mca/paffinity/base/base.h"
>> +#include "paffinity_linux.h"
>> +#include "plpa/src/libplpa/plpa.h"
>> +
>> +
>> +/*
>> + * Local functions
>> + */
>> +static int linux_module_init(void);
>> +static int linux_module_set(opal_paffinity_base_cpu_set_t cpumask);
>> +static int linux_module_get(opal_paffinity_base_cpu_set_t *cpumask);
>> +static int linux_module_map_to_processor_id(int socket, int core,
>> int *processor_id);
>> +static int linux_module_map_to_socket_core(int processor_id, int
>> *socket, int *core);
>> +static int linux_module_get_processor_info(int *num_processors, int
>> *max_processor_id);
>> +static int linux_module_get_socket_info(int *num_sockets, int
>> *max_socket_num);
>> +static int linux_module_get_core_info(int socket, int *num_cores,
>> int *max_core_num);
>> +
>> +/*
>> + * Linux paffinity module
>> + */
>> +static const opal_paffinity_base_module_1_1_0_t module = {
>> +
>> + /* Initialization function */
>> +
>> + linux_module_init,
>> +
>> + /* Module function pointers */
>> +
>> + linux_module_set,
>> + linux_module_get,
>> + linux_module_map_to_processor_id,
>> + linux_module_map_to_socket_core,
>> + linux_module_get_processor_info,
>> + linux_module_get_socket_info,
>> + linux_module_get_core_info,
>> + NULL
>> +};
>> +
>> +
>> +const opal_paffinity_base_module_1_1_0_t *
>> +opal_paffinity_linux_component_query(int *query)
>> +{
>> + int param;
>> +
>> + param = mca_base_param_find("paffinity", "linux", "priority");
>> + mca_base_param_lookup_int(param, query);
>> +
>> + return &module;
>> +}
>> +
>> +
>> +static int linux_module_init(void)
>> +{
>> + /* Nothing to do */
>> +
>> + return OPAL_SUCCESS;
>> +}
>> +
>> +
>> +
>> +/
>> ************************************************************************
>> + See the note in paffinity_linux.h -- there are at least 3
>> different
>> + ways that Linux's sched_setaffinity()/sched_getaffinity() are
>> + implemented. Thankfully there is the Portable Linux Processor
>> + Affinity project which determines the flavor of affinity at
>> runtime
>> + and takes care of of the problem.
>> +
>> + Using get/set affinity functions from plpa - configured with an
>> + opal prefix.
>> +
>> + User needs to set a mask with the bit number of the cpu set. We
>> provide
>> + macros to do this.
>> +
>> +
>> ************************************************************************/
>> +
>> +static int linux_module_set(opal_paffinity_base_cpu_set_t mask)
>> +{
>> +
>> + opal_paffinity_linux_plpa_cpu_set_t plpa_mask;
>> + unsigned int i;
>> +
>> + if (sizeof(mask) > sizeof(plpa_mask)) {
>> + return OPAL_ERR_BAD_PARAM;
>> + } else {
>> + PLPA_CPU_ZERO(&plpa_mask);
>> + for (i = 0; i < sizeof(plpa_mask) ; i++) {
>> + if (PLPA_CPU_ISSET(i,&mask)) {
>> + PLPA_CPU_SET(i,&plpa_mask);
>> + }
>> + }
>> + }
>> +
>> + if (0 != opal_paffinity_linux_plpa_sched_setaffinity(getpid(),
>> +
>> sizeof(plpa_mask),
>> +
>> &plpa_mask)) {
>> + return OPAL_ERR_IN_ERRNO;
>> + }
>> + return OPAL_SUCCESS;
>> +}
>> +
>> +
>> +static int linux_module_get(opal_paffinity_base_cpu_set_t *mask)
>> +{
>> + opal_paffinity_linux_plpa_cpu_set_t plpa_mask;
>> + unsigned int i;
>> +
>> + if (NULL == mask) {
>> + return OPAL_ERR_BAD_PARAM;
>> + }
>> +
>> + if (sizeof(*mask) > sizeof(plpa_mask)) {
>> + return OPAL_ERR_BAD_PARAM; /* look up in header file */
>> + }
>> +
>> + if (0 != opal_paffinity_linux_plpa_sched_getaffinity(getpid(),
>> sizeof(plpa_mask), &plpa_mask)) {
>> + return OPAL_ERR_IN_ERRNO;
>> + }
>> + for (i = 0; i < sizeof(mask); i++) {
>> + if (PLPA_CPU_ISSET(i,&plpa_mask)) {
>> + PLPA_CPU_SET(i,mask);
>> + }
>> + }
>> +
>> + return OPAL_SUCCESS;
>> +}
>> +
>> +static int linux_module_map_to_processor_id(int socket, int core,
>> int *processor_id)
>> +{
>> + return opal_paffinity_linux_plpa_map_to_processor_id(socket,
>> core, processor_id);
>> +}
>> +
>> +static int linux_module_map_to_socket_core(int processor_id, int
>> *socket, int *core)
>> +{
>> + return
>> opal_paffinity_linux_plpa_map_to_socket_core(processor_id, socket,
>> core);
>> +}
>> +
>> +static int linux_module_get_processor_info(int *num_processors, int
>> *max_processor_id)
>> +{
>> + return
>> opal_paffinity_linux_plpa_get_processor_info(num_processors,
>> max_processor_id);
>> +}
>> +
>> +static int linux_module_get_socket_info(int *num_sockets, int
>> *max_socket_num)
>> +{
>> + return opal_paffinity_linux_plpa_get_socket_info(num_sockets,
>> max_socket_num);
>> +}
>> +
>> +static int linux_module_get_core_info(int socket, int *num_cores,
>> int *max_core_num)
>> +{
>> + return opal_paffinity_linux_plpa_get_core_info(socket,
>> num_cores, max_core_num);
>> +}
>> +
>> _______________________________________________
>> svn-full mailing list
>> svn-full_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/svn-full
>
>
> --
> Jeff Squyres
> Cisco Systems
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
Cisco Systems