Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] 2 GB limitation of MPI_File_write_all
From: Rayson Ho (raysonlogin_at_[hidden])
Date: 2012-10-20 10:12:13


Hi Eric,

Sounds like it's also related to this problem reported by Scinet back in July:

http://www.open-mpi.org/community/lists/users/2012/07/19762.php

And I think I found the issue, but I still have not followed up with
the ROMIO guys yet. And I was not sure if Scinet was waiting for the
fix or not - next time I visit U of Toronto, I will see if I can visit
the Scinet office and meet with the Scinet guys!

http://www.open-mpi.org/community/lists/users/2012/08/19907.php

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/

On Fri, Oct 19, 2012 at 4:45 PM, Gus Correa <gus_at_[hidden]> wrote:
> Hi Eric
>
> Have you tried to create a user-defined MPI type
> (say MPI_Type_Contiguous or MPI_Type_Vector) and pass them
> to the MPI function calls, instead of MPI_LONGs?
> Then you could use the new type and the new number
> (i.e., an integer number smaller than "size", and
> smaller than the maximum integer 2,147,483,647 )
> in the MPI function calls (e.g., MPI_File_write_all).
> Maybe the "invalid argument" error message relates to this.
> If I remember right, the 'number of elements' in MPI calls
> is a positive integer (int, 32 bits).
>
> See these threads about this workaround:
>
> http://www.open-mpi.org/community/lists/users/2009/02/8100.php
> http://www.open-mpi.org/community/lists/users/2010/11/14816.php
>
> Also, not MPI but C.
> I wonder if you need to declare "size" as 'long int',
> or maybe 'long long int', to represent/hold correctly
> the large value that you want
> (360,000,000,000 > 2,147,483,647).
>
> I hope this helps,
> Gus Correa
>
>
> On 10/19/2012 02:31 PM, Eric Chamberland wrote:
>>
>> Hi,
>>
>> I get this error when trying to write 360 000 000 000 MPI_LONG:
>>
>> with Openmpi-1.4.5:
>> ERROR Returned by MPI_File_write_all: 35
>> ERROR_string Returned by MPI_File_write_all: MPI_ERR_IO: input/output
>> error
>>
>> with Openmpi-1.6.2:
>> ERROR Returned by MPI_File_write_all: 13
>> ERROR_string Returned by MPI_File_write_all: MPI_ERR_ARG: invalid
>> argument of some other kind
>>
>> First, the error in 1.6.2 seems to be less usefull to understand what
>> happened for the user...
>>
>> Second, am I wrong to try to write that much MPI_LONG? Is this
>> limitation documented or to be fixed?
>>
>> Thanks,
>>
>> Eric
>>
>> =====================================================
>> Here is the code:
>>
>> #include <stdio.h>
>> #include "mpi.h"
>>
>> int main (int argc, char *argv[])
>> {
>> MPI_Datatype filetype;
>> MPI_File fh;
>> long *local_array;
>> MPI_Status status;
>>
>> MPI_Init( &argc, &argv );
>>
>> int nb_proc = 0;
>> MPI_Comm_size( MPI_COMM_WORLD, &nb_proc );
>> if (nb_proc != 1) {
>> printf( "Test code for 1 process!\n" );
>> MPI_Abort( MPI_COMM_WORLD, 1 );
>> }
>> int size=90000000*4;
>> local_array = new long[size];
>>
>> MPI_File_open(MPI_COMM_WORLD, "2.6Gb",
>> MPI_MODE_CREATE | MPI_MODE_WRONLY,
>> MPI_INFO_NULL, &fh);
>>
>> int ierr = MPI_File_write_all(fh, local_array, size, MPI_LONG, &status);
>> if (ierr != MPI_SUCCESS) {
>> printf("ERROR Returned by MPI_File_write_all: %d\n",ierr);
>> char* lCharPtr = new char[MPI_MAX_ERROR_STRING];
>> int lLongueur = 0;
>> MPI_Error_string(ierr,lCharPtr, &lLongueur);
>> printf("ERROR_string Returned by MPI_File_write_all: %s\n",lCharPtr);
>> MPI_Abort( MPI_COMM_WORLD, 1 );
>> }
>>
>> MPI_File_close(&fh);
>>
>> MPI_Finalize();
>> return 0;
>> }
>>
>> ~
>>
>> ~
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users