Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] [Open MPI] #3351: JAVA scatter error
From: Siegmar Gross (Siegmar.Gross_at_[hidden])
Date: 2012-12-19 09:22:02


Hi

I shortend this email so that you get earlier to my comments.

> > In my opinion Datatype.Vector must set the size of the
> > base datatype as extent of the vector and not the true extent, because
> > MPI-Java doesn't provide a function to resize a datatype.
>
> No, I think Datatype.Vector is doing the Right Thing in that it acts
> just like MPI_Type_vector. We do want these to be *bindings*, after
> all -- meaning that they should be pretty much a 1:1 mapping to the
> C bindings.
>
> I think the real shortcoming is that there is no Datatype.Resized
> function. That can be fixed.

Are you sure? That would at least solve one problem.

> > We should forget
> > ObjectScatterMain.java for the moment and concentrate on
> > ObjectBroadcastMain.java, which I have sent three days ago to the list,
> > because it has the same problem.
> >
> > 1) ColumnSendRecvMain.java
> >
> > I create a 2D-matrix with (Java books would use "double[][] matrix"
> > which is the same in my opinion, but I like C notation)
> >
> > double matrix[][] = new double[P][Q];
>
> I noticed that if I used [][] in my version of the Scatter program,
> I got random results. But if I used [] and did my own offset
> indexing, it worked.

I think if you want a 2D-matrix you should use a Java matrix and not
a special one with your own offset indexing. In my opinion that is
something a C programmer can/would do (I'm a C programmer myself with
a little Java knowledge), but the benefit of Java is that the programmer
should not know about addresses, memory layouts and similar things. Now
I sound like my colleagues who always claim that my Java programs look
more like C programs than Java programs :-(. I know nothing about the
memory layout of a Java matrix or if the layout is stable during the
lifetime of the object, but I think that the Java interface should deal
with all these things if that is possible. I suppose that Open MPI will
not succeed in the Java world if it requires "special" matrices and a
special offset indexing. Perhaps some members of this list have very
good Java knowledge or even know the exact layout of Java matrices so
that Datatype.Vector can build a Java column vector from a Java matrix
which even contains valid values.

> If double[][] is a fundamentally different type (and storage format)
> than double[], what is MPI to do? How can it tell the difference?
>
> > It is easy to see that process 1 doesn't get column 0. Your
> > suggestion to allocate enough memory for a matrix (without defining
> > a matrix) and doing all index computations yourself is in my opinion
> > not applicable for a "normal" Java programmer (it's even hard for
> > most C programmers :-) ). Hopefully you have an idea how to solve
> > this problem so that all processes receive correct column values.
>
> I'm afraid I don't, other than defining your own class which
> allocates memory contiguously, but overrides [] and [][]
> (I'm *assuming* you can do that in Java...?).

Does anybody else in this list know how it can be done?

> > 2) ObjectBroadcastMain.java
> >
> > As I said above, it is my understanding, that I can send a Java object
> > when I use MPI.OBJECT and that the MPI implementation must perform all
> > necessary tasks.
>
> Remember: there is no standard for MPI and Java. So there is no
> "must". :-)

I know and I'm grateful that you try nevertheless to offer a Java
interface. Hopefully you will not misunderstand my "must". It wasn't
complaining, but trying to express that a "normal" Java user would
expect that he can implement an MPI program without special knowledge
about data layouts.

> This is one research implementation that was created. We can update
> it and try to make it better, but we're somewhat crafting the rules
> as we go along here.
>
> (BTW, if we continue detailed discussions about implementation,
> this conversation should probably move to the devel list...)
>
> > Your interface for derived datatypes provides only
> > methods for discontiguous data and no method to create an MPI.OBJECT,
> > so that I have no idea what I would have to do to create one. The
> > object must be serializable so that you get the same values in a
> > heterogeneous environment.
> >
> > tyr java 146 mpiexec -np 2 java ObjectBroadcastMain
> > Exception in thread "main" java.lang.ClassCastException:
> > MyData cannot be cast to [Ljava.lang.Object;
> > at mpi.Comm.Object_Serialize(Comm.java:207)
> > at mpi.Comm.Send(Comm.java:292)
> > at mpi.Intracomm.Bcast(Intracomm.java:202)
> > at ObjectBroadcastMain.main(ObjectBroadcastMain.java:44)
> > ...
>
> After rooting around in the code a bit, I think I understand this
> stack trace a bit better now..
>
> The code line in question is in the Object_Serialize method, where
> it calls:
>
> Object buf_els [] = (Object[])buf;
>
> So it's trying to cast an (Object) to an (Object[]). Apparently,
> this works for intrinsic Java types (e.g., int). But it doesn't
> seem to work with your own class.
>
> Again, here's my disclaimer that I'm not a Java guy... :-) But does
> this mean you need to define an operator[] method on your class, and
> that would allow this casting to work? (not that I'm sure what this
> method would need to *do*, but this is a first step...)

Alternatively the buffer parameter type could be changed from "Object"
to "Object[]". Then everybody would know that an array is expected
(even for a single value). However, I don't know if that has a
consequence which breaks other things. Is a different parameter type
possible or out-of any question?

Kind regards

Siegmar