Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] [Open MPI] #3351: JAVA scatter error
From: Siegmar Gross (Siegmar.Gross_at_[hidden])
Date: 2012-12-19 09:22:02


Hi

I shortend this email so that you get earlier to my comments.

> > In my opinion Datatype.Vector must set the size of the
> > base datatype as extent of the vector and not the true extent, because
> > MPI-Java doesn't provide a function to resize a datatype.
>
> No, I think Datatype.Vector is doing the Right Thing in that it acts
> just like MPI_Type_vector. We do want these to be *bindings*, after
> all -- meaning that they should be pretty much a 1:1 mapping to the
> C bindings.
>
> I think the real shortcoming is that there is no Datatype.Resized
> function. That can be fixed.

Are you sure? That would at least solve one problem.

> > We should forget
> > ObjectScatterMain.java for the moment and concentrate on
> > ObjectBroadcastMain.java, which I have sent three days ago to the list,
> > because it has the same problem.
> >
> > 1) ColumnSendRecvMain.java
> >
> > I create a 2D-matrix with (Java books would use "double[][] matrix"
> > which is the same in my opinion, but I like C notation)
> >
> > double matrix[][] = new double[P][Q];
>
> I noticed that if I used [][] in my version of the Scatter program,
> I got random results. But if I used [] and did my own offset
> indexing, it worked.

I think if you want a 2D-matrix you should use a Java matrix and not
a special one with your own offset indexing. In my opinion that is
something a C programmer can/would do (I'm a C programmer myself with
a little Java knowledge), but the benefit of Java is that the programmer
should not know about addresses, memory layouts and similar things. Now
I sound like my colleagues who always claim that my Java programs look
more like C programs than Java programs :-(. I know nothing about the
memory layout of a Java matrix or if the layout is stable during the
lifetime of the object, but I think that the Java interface should deal
with all these things if that is possible. I suppose that Open MPI will
not succeed in the Java world if it requires "special" matrices and a
special offset indexing. Perhaps some members of this list have very
good Java knowledge or even know the exact layout of Java matrices so
that Datatype.Vector can build a Java column vector from a Java matrix
which even contains valid values.

> If double[][] is a fundamentally different type (and storage format)
> than double[], what is MPI to do? How can it tell the difference?
>
> > It is easy to see that process 1 doesn't get column 0. Your
> > suggestion to allocate enough memory for a matrix (without defining
> > a matrix) and doing all index computations yourself is in my opinion
> > not applicable for a "normal" Java programmer (it's even hard for
> > most C programmers :-) ). Hopefully you have an idea how to solve
> > this problem so that all processes receive correct column values.
>
> I'm afraid I don't, other than defining your own class which
> allocates memory contiguously, but overrides [] and [][]
> (I'm *assuming* you can do that in Java...?).

Does anybody else in this list know how it can be done?

> > 2) ObjectBroadcastMain.java
> >
> > As I said above, it is my understanding, that I can send a Java object
> > when I use MPI.OBJECT and that the MPI implementation must perform all
> > necessary tasks.
>
> Remember: there is no standard for MPI and Java. So there is no
> "must". :-)

I know and I'm grateful that you try nevertheless to offer a Java
interface. Hopefully you will not misunderstand my "must". It wasn't
complaining, but trying to express that a "normal" Java user would
expect that he can implement an MPI program without special knowledge
about data layouts.

> This is one research implementation that was created. We can update
> it and try to make it better, but we're somewhat crafting the rules
> as we go along here.
>
> (BTW, if we continue detailed discussions about implementation,
> this conversation should probably move to the devel list...)
>
> > Your interface for derived datatypes provides only
> > methods for discontiguous data and no method to create an MPI.OBJECT,
> > so that I have no idea what I would have to do to create one. The
> > object must be serializable so that you get the same values in a
> > heterogeneous environment.
> >
> > tyr java 146 mpiexec -np 2 java ObjectBroadcastMain
> > Exception in thread "main" java.lang.ClassCastException:
> > MyData cannot be cast to [Ljava.lang.Object;
> > at mpi.Comm.Object_Serialize(Comm.java:207)
> > at mpi.Comm.Send(Comm.java:292)
> > at mpi.Intracomm.Bcast(Intracomm.java:202)
> > at ObjectBroadcastMain.main(ObjectBroadcastMain.java:44)
> > ...
>
> After rooting around in the code a bit, I think I understand this
> stack trace a bit better now..
>
> The code line in question is in the Object_Serialize method, where
> it calls:
>
> Object buf_els [] = (Object[])buf;
>
> So it's trying to cast an (Object) to an (Object[]). Apparently,
> this works for intrinsic Java types (e.g., int). But it doesn't
> seem to work with your own class.
>
> Again, here's my disclaimer that I'm not a Java guy... :-) But does
> this mean you need to define an operator[] method on your class, and
> that would allow this casting to work? (not that I'm sure what this
> method would need to *do*, but this is a first step...)

Alternatively the buffer parameter type could be changed from "Object"
to "Object[]". Then everybody would know that an array is expected
(even for a single value). However, I don't know if that has a
consequence which breaks other things. Is a different parameter type
possible or out-of any question?

Kind regards

Siegmar