Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Type_struct for structs with dynamic arrays
From: Jitendra Kumar (jkumar_at_[hidden])
Date: 2008-08-20 14:07:47


George,
Yes that's what I understood after struggling with it over a week. I
need to send such messages frequently so creating and destroying the
data type each time may be expensive. What would be the best alternative
for sending such malloced data ? Though I can always pack the data in a
long array and unpack at the opposite end as I know the structure of the
data at each node. Anything more efficient and elegant will be better.

Thanks for the help.
Jitendra

George Bosilca wrote:
> Jitendra,
>
> You will not be able to reuse the same data-type. All internal arrays
> are malloced and their position in memory will be completely random
> from call to call (or use to use of the data-type). Therefore, even if
> your structure looks the same, as you use non static arrays in your
> structure, you will have to recreate the data-type for every new
> structure (of type individual).
>
> george.
>
> On Aug 19, 2008, at 5:12 PM, Jitendra Kumar wrote:
>
>> George,
>> Thanks for your reply. However, I am still not able to resolve the
>> issue. I have been looking at one of your old post
>> http://www.open-mpi.org/community/lists/users/2005/08/0123.php
>> (I have tried to explain the issues below with snippets of codes and on
>> screen debugging messages)
>>
>> I am trying not to use MPI_BOTTOM because I wish to use the same
>> declared data type again and again for sending any struct of that type.
>> So I specified the addresses as follows.
>> 521 MPI_Get_address(&parentPop[0].indiv, &origin);
>> 522 printf("origin %ld \n", origin);
>> 523 MPI_Get_address(&parentPop[0].indiv[0].intvar[0],
>> &disp[0]);
>> 524 printf("disp 0 %ld \n", disp[0]);
>> 525 MPI_Get_address(&parentPop[0].indiv[0].realvar[0],
>> &disp[1]);
>> 526 printf("disp 1 %ld\n", disp[1]);
>> 527 MPI_Get_address(&parentPop[0].indiv[0].binvar[0],
>> &disp[2]);
>> 528 printf("disp 2 %ld\n", disp[2]);
>> 529 MPI_Get_address(&parentPop[0].indiv[0].gene[0][0],
>> &disp[3]);
>> 530 printf("disp 3 %ld\n", disp[3]);
>> 531 MPI_Get_address(&parentPop[0].indiv[0].obj[0], &disp[4]);
>> 532 printf("disp 4 %ld %ld\n", disp[4],
>> &parentPop[0].indiv[0].obj[0]);
>> 533 MPI_Get_address(&parentPop[0].indiv[0].constr[0],
>> &disp[5]);
>> 534 printf("disp 5 %ld\n", disp[5]);
>> 535 MPI_Get_address(&parentPop[0].indiv[0].constr_violation,
>> &disp[6]);
>> 536 printf("disp 6 %ld\n", disp[6]);
>> 537 MPI_Get_address(&parentPop[0].indiv[0].crowd_dist,
>> &disp[7]);
>> 538 printf("disp 7 %ld\n", disp[7]);
>> 539 MPI_Get_address(&parentPop[0].indiv[0].rank, &disp[8]);
>> 540 printf("disp 8 %ld\n", disp[8]);
>> 541
>> MPI_Get_address(&parentPop[0].indiv[0].strategyParameter[0], &disp[9]);
>> 542 printf("disp 9 %ld\n", disp[9]);
>> 543 // base = disp[0];
>> 544 base = origin;
>>
>> The values of displacements looks like this:
>> 3: origin 6088720
>> 3: disp 0 6089648
>> 3: disp 1 6089680
>> 3: disp 2 6089712
>> 3: disp 3 6089776
>> 3: disp 4 6089808
>> 3: disp 5 6089840
>> 3: disp 6 6090672
>> 3: disp 7 6090680
>> 3: disp 8 6090688
>> 3: disp 9 6089872
>>
>> after subtracting origin from the displacement to get relative
>> displacements.
>> 569 for(i=0; i<10; i++)
>> 570 {
>> 571 disp[i] -= base;
>> 572 fprintf(stdout, "Rank %d: disp[%d] %d\n", rank, i,
>> disp[i]);
>> 573 }
>>
>> 3: Rank 3: disp[0] 928
>> 3: Rank 3: disp[1] 960
>> 3: Rank 3: disp[2] 992
>> 3: Rank 3: disp[3] 1056
>> 3: Rank 3: disp[4] 1088
>> 3: Rank 3: disp[5] 1120
>> 3: Rank 3: disp[6] 1952
>> 3: Rank 3: disp[7] 1960
>> 3: Rank 3: disp[8] 1968
>> 3: Rank 3: disp[9] 1152
>>
>> 574 MPI_Type_create_struct(10, blockcounts, disp, types,
>> &Individual);
>> 575
>> 576 /* Check that the datatype has correct extent */
>> 577 MPI_Type_extent(Individual, &extent);
>> 578 if(extent != sizeof(individual))
>> 579 {
>> 580 printf("Rank %d: adjusting the extent (%d) of the
>> data type %d\n", rank, extent, sizeof(individual));
>> 581 MPI_Datatype indold = Individual;
>> 582 MPI_Type_create_resized(indold, 0,
>> sizeof(individual), &Individual);
>> 583 MPI_Type_free(&indold);
>> 584 }
>> 585 MPI_Type_commit(&Individual);
>> 586 /* MPI derived datatype for indivdual declared */
>>
>> After creating the datatype, for test I tried to send and receive a
>> struct to itself and save it to a variable of the same struct type. But
>> the value of the variables in the receiving struct remains unchanged.
>> Though the MPI_Get_elements shows the correct number of elements in the
>> received datatype.
>>
>> 588 int count;
>> 589 individual recvind;
>> 590 allocateMemoryInd(&parentPop[0], &recvind);
>> 591 initializeInd(&parentPop[0], &recvind);
>> 592
>> 593 fprintf(stdout, "Rank %d: before receive recvind.nobj %f
>> recvind.numreal %f\n", rank, recvind.obj[0], recvind.realvar[0]);
>> 594 MPI_Sendrecv(&parentPop[0].indiv[0], 1, Individual,
>> rank, rank, &recvind, 1, Individual, rank, rank, MPI_COMM_WORLD,
>> &status);
>> 595 count = 0;
>> 596 MPI_Get_elements(&status, Individual, &count);
>> 597 fprintf(stdout, "Rank %d: after receive recvind.nobj %f
>> (expected value %f) recvind.numreal %f (expected value %f) count %d\n",
>> rank, recvind.obj[0], parentPo p[0].indiv[0].obj[0],
>> recvind.realvar[0], parentPop[0].indiv[0].realvar[0], count);
>>
>> 3: Rank 3: adjusting the extent (1048) of the data type 80
>> 3: Rank 3: before receive recvind.nobj 0.000000 recvind.numreal
>> -26.938538
>> 3: Rank 3: after receive recvind.nobj 0.000000 (expected value 0.000000)
>> recvind.numreal -26.938538 (expected value -126.740479) count 16
>>
>> I highly appreciate your help resolving the issue.
>>
>> Thanks,
>> Jitendra
>>> Message: 1
>>> Date: Sun, 17 Aug 2008 23:30:54 +0200
>>> From: George Bosilca <bosilca_at_[hidden]>
>>> Subject: Re: [OMPI users] MPI_Type_struct for structs with dynamic
>>> arrays
>>> To: Open MPI Users <users_at_[hidden]>
>>> Message-ID: <B9DD8054-97D1-47BA-A88C-27F0C5529622_at_[hidden]>
>>> Content-Type: text/plain; charset="us-ascii"; Format="flowed";
>>> DelSp="yes"
>>>
>>> Jitendra,
>>>
>>> There is a problem with the addresses you provide to MPI_Type_struct.
>>> For all arrays instead of giving the pointer to the array, you provide
>>> a pointer to the pointer in the individual struct.
>>>
>>> Try the following
>>> MPI_Get_address(&parentPop[0].indiv[0].intvar[0], &disp[0]);
>>> instead of
>>> MPI_Get_address(&parentPop[0].indiv[0].intvar, &disp[0]);
>>>
>>> Please note the [0] after the array name. Please do the same to all
>>> arrays and I think the MPI_Type_struct will do the rest.
>>>
>>> Btw, you dont have to substract the disp[0] from all addresses.
>>> Instead you can use MPI_BOTTOM and all your addresses can be absolute.
>>>
>>> george.
>>>
>>> On Aug 11, 2008, at 1:07 AM, Jitendra Kumar wrote:
>>>
>>>
>>>> Hi,
>>>> I am trying to use MPI derived datatype doutines for sending a struct
>>>> which contains dynamically allocated arrays. I tried implementing it
>>>> using MPI_Type_struct. It doesn't throws any error but messages being
>>>> received (of the declared datatype) aren't correct. Some memory
>>>> corruption seems to be going on as the value of 'rank' at receive end
>>>> are changed to 0 right after the receive . Below are the snippets of
>>>> my
>>>> struct and implementation of the derived datatype.
>>>> I am not sure where things are going wrong. I would highly appreciate
>>>> any pointers or suggestions. Is there any better alternative way
>>>> instead
>>>> of MPI_Type_struct considering that frequent communication of these
>>>> structs are needed?
>>>>
>>>> Thanks,
>>>> Jitendra
>>>>
>>>> The struct looks like this:
>>>> 51 typedef struct
>>>> 52 {
>>>> 53 int *intvar;
>>>> 54 double *realvar;
>>>> 55 double *binvar;
>>>> 56 int **gene;
>>>> 57
>>>> 58 double *obj;
>>>> 59 double *constr;
>>>> 60 double constr_violation;
>>>> 61 double crowd_dist;
>>>> 62 int rank;
>>>> 63 double *strategyParameter;
>>>> 64 }
>>>> 65 individual;
>>>>
>>>> Implementation of the derived datatype:
>>>>
>>>> 483 blockcounts[0] = parentPop[0].numInteger;
>>>> 484 blockcounts[1] = parentPop[0].numReal;
>>>> 485 blockcounts[2] = parentPop[0].numBinary;
>>>> 486 sum = 0;
>>>> 487 for(i=0; i<parentPop[0].numBinary; i++)
>>>> 488 {
>>>> 489 sum = sum + parentPop[0].nbits[i];
>>>> 490 }
>>>> 491
>>>> 492 blockcounts[3] = sum;
>>>> 493 blockcounts[4] = parentPop[0].nobj;
>>>> 494 blockcounts[5] = parentPop[0].ncon;
>>>> 495 blockcounts[6] = 1;
>>>> 496 blockcounts[7] = 1;
>>>> 497 blockcounts[8] = 1;
>>>> 498 blockcounts[9] = parentPop[0].numInteger +
>>>> parentPop[0].numReal;
>>>>
>>>> 506 types[0] = MPI_INT;
>>>> 507 types[1] = MPI_DOUBLE;
>>>> 508 types[2] = MPI_DOUBLE;
>>>> 509 types[3] = MPI_INT;
>>>> 510 types[4] = MPI_DOUBLE;
>>>> 511 types[5] = MPI_DOUBLE;
>>>> 512 types[6] = MPI_DOUBLE;
>>>> 513 types[7] = MPI_DOUBLE;
>>>> 514 types[8] = MPI_INT;
>>>> 515 types[9] = MPI_DOUBLE;
>>>> 516
>>>> 517 MPI_Get_address(&parentPop[0].indiv[0].intvar,
>>>> &disp[0]);
>>>> 518 printf("parentpop.indiv0 %ld disp %ld (%ld)\n",
>>>> &parentPop[0].indiv[0], disp, disp[0]);
>>>> 519 MPI_Get_address(&parentPop[0].indiv[0].realvar,
>>>> &disp[1]);
>>>> 520 printf("disp 1 %ld\n", disp[1]);
>>>> 521 MPI_Get_address(&parentPop[0].indiv[0].binvar,
>>>> &disp[2]);
>>>> 522 printf("disp 2 %ld\n", disp[2]);
>>>> 523 MPI_Get_address(&parentPop[0].indiv[0].gene, &disp[3]);
>>>> 524 MPI_Get_address(&parentPop[0].indiv[0].obj, &disp[4]);
>>>> 525 MPI_Get_address(&parentPop[0].indiv[0].constr,
>>>> &disp[5]);
>>>> 526
>>>> MPI_Get_address(&parentPop[0].indiv[0].constr_violation,
>>>> &disp[6]);
>>>> 527 MPI_Get_address(&parentPop[0].indiv[0].crowd_dist,
>>>> &disp[7]);
>>>> 528 MPI_Get_address(&parentPop[0].indiv[0].rank, &disp[8]);
>>>> 529
>>>> MPI_Get_address(&parentPop[0].indiv[0].strategyParameter, &disp[9]);
>>>> 530 base = disp[0];
>>>> 531 for(i=0; i<10; i++)
>>>> 532 {
>>>> 533 disp[i] -= base;
>>>> 534 }
>>>> 535
>>>> 536 MPI_Type_create_struct(10, blockcounts, disp, types,
>>>> &Individual);
>>>> 537
>>>> 538 /* Check that the datatype has correct extent */
>>>> 539 MPI_Type_extent(Individual, &extent);
>>>> 540 if(extent != sizeof(individual))
>>>> 541 {
>>>> 542 MPI_Datatype indold = Individual;
>>>> 543 MPI_Type_create_resized(indold, 0,
>>>> sizeof(individual), &Individual);
>>>> 544 MPI_Type_free(&indold);
>>>> 545 }
>>>> 546 MPI_Type_commit(&Individual);
>>>>
>>>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users