Date: From detar@physics.utah.edu  Tue Jun 26 00:35:02 2001
Date: Mon, 25 Jun 2001 22:31:36 -0600 (MDT)
Subject: Re: Message Passing API

Hi Chip,

Here are some preliminary thoughts about the proposed messaging API in
advance of our conference call.

1. For the reduction operation, we need more than just addition of floats.
   For example, the MILC code currently has, in addition
   sum vector of doubles
   sum complex and double complex and vectors thereof
     (this can be subsumed under vectors of floats and doubles,
      if the complex structure is sensible)
   global exclusive 32-bit logical "or" (used for checksums of files)
   global max of floats and doubles

   I would suggest making at least the addition of vectors of floats
   and doubles standard as well as the max of floats and doubles, and
   then provide for a general reduction operation that calls an
   arbitrary user-supplied binary function to provide flexibility.
     
2. The hyper-surface transmission is interesting, but may assume too
   much about layouts and actions.  What if we wanted to transmit not
   just the outer hypersurface, but the next inside hypersurface,
   because the action involves next-neighbor couplings?  I can see the
   value in having a strided copy, however, as we have discussed
   before.

Some random thoughts:

A hardware issue that you probably thought of.  With asynchronous
multiple message passing, it is important that messages arrive in the
same order they are sent.

As for somewhat higher level functionality, as you have noticed, it is
certainly good to be able to send a message repetitively (from the
same addresses) with minimal reinitialization.  And it is good at
program start up to do all the necessary bookkeeping for message
passing for a small list of needed destination patterns - e.g. bit
reverse and butterfly map for the FFT as well as nearest neighbor, so
that they can be invoked later with only minimal further
initialization.

Regards,
Carleton