Hi Jeff, Ralph, and all,
Thank you for your reply.
RIKEN and Fujitsu will work toword 10Pflops with Open MPI continuously.
Here we can explain some parts of our MPI:
As page 13 of Koh Hotta's presentation shows, we extended OMPI
# Sorry, this figure is somewhat broken. Arrows point incorrect layers.
First, we created a new BTL component, 'tofu BTL'. It's not so special
one but dedicated to our Tofu interconnect. But its latency was not
enough for us.
So we created a new framework, 'LLP', and its component, 'tofu LLP'.
It bypasses request object creation in PML and BML/BTL, and sends
a message immediately if possible.
Also, we modified tuned COLL to implement interconnect-and-topology-
specific bcast/allgather/alltoall/allreduce algorithm. These algorithm
implementations also bypass PML/BML/BTL to eliminate protocol and software
To achieve above, we created 'tofu COMMON', like sm (ompi/mca/common/sm/).
Is there interesting one?
Though our BTL and COLL are quite interconnect-specific, LLP may be
contributed in the future.
MPI development team,
> I echo what Ralph said -- congratulations!
> Let us know when you'll be ready to contribute back what you can.
> On Jun 27, 2011, at 9:58 PM, Takahiro Kawashima wrote:
> > Dear Open MPI community,
> > I'm a member of MPI library development team in Fujitsu. Shinji
> > Sumimoto, whose name appears in Jeff's blog, is one of our bosses.
> > As Rayson and Jeff noted, K computer, world's most powerful HPC system
> > developed by RIKEN and Fujitsu, utilizes Open MPI as a base of its MPI
> > library. We, Fujitsu, are pleased to announce that, and also have special
> > thanks to Open MPI community.
> > We are sorry to be late announce!
> > Our MPI library is based on Open MPI 1.4 series, and has a new point-
> > to-point component (BTL) and new topology-aware collective communication
> > algorithms (COLL). Also, it is adapted to our runtime environment (ESS,
> > PLM, GRPCOMM etc).
> > K computer connects 68,544 nodes by our custom interconnect.
> > Its runtime environment is our proprietary one. So we don't use orted.
> > We cannot tell start-up time yet because of disclosure restriction, sorry.
> > We are surprised by the extensibility of Open MPI, and have proved that
> > Open MPI is scalable to 68,000 processes level! We feel pleasure to
> > utilize such a great open-source software.
> > We cannot tell detail of our technology yet because of our contract
> > with RIKEN AICS, however, we will plan to feedback of our improvements
> > and bug fixes. We can contribute some bug fixes soon, however, for
> > contribution of our improvements will be next year with Open MPI
> > agreement.
> > Best regards,
> > MPI development team,
> > Fujitsu
> >> I got more information:
> >> http://blogs.cisco.com/performance/open-mpi-powers-8-petaflops/
> >> Short version: yes, Open MPI is used on K and was used to power the 8PF runs.
> >> w00t!
> >> On Jun 24, 2011, at 7:16 PM, Jeff Squyres wrote:
> >>> w00t!
> >>> OMPI powers 8 petaflops!
> >>> (at least I'm guessing that -- does anyone know if that's true?)
> >>> On Jun 24, 2011, at 7:03 PM, Rayson Ho wrote:
> >>>> Interesting... page 11:
> >>>> http://www.fujitsu.com/downloads/TC/sc10/programming-on-k-computer.pdf
> >>>> Open MPI based:
> >>>> * Open Standard, Open Source, Multi-Platform including PC Cluster.
> >>>> * Adding extension to Open MPI for "Tofu" interconnect
> >>>> Rayson
> > _______________________________________________
> > devel mailing list
> > devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Jeff Squyres
> For corporate legal information go to: