Subject: MILESTONES Bob Sugar, I am forwarding the responses I got from the Software Committee (so far) in full because they clarify the issue of single versus double precision. There are good arguments (see Bob M's note) that single precision may be gradually phased out, BUT at present there are many examples (see Carleton and Robert E) where single precision gives an advantage in terms of flops/$. It is pretty clear to me (as was suggested in yesterday's call) that we should set milestones in both double and single precision terms. I would like to have some double precision tests (and maybe single precision test?) on both QCDOC and clusters so we have this "apples to apples" comparision. I will talk with Claudio today to get his input on how to modify Section 5 and to get feedback from the Program Committee if he feels this is desirable. Rich ---------------------------------------------------------------------------- >From rdm@physics.columbia.edu Tue Aug 13 19:10:04 2002 Date: Tue, 13 Aug 2002 19:10:07 -0400 (EDT) From: rdm@physics.columbia.edu To: brower@lns.mit.edu, cmendes@cs.uiuc.edu, detar@physics.utah.edu, djholm@fnal.gov, edwards@jlab.org, watson@jlab.org Cc: rdm@physics.columbia.edu Subject: performance criteria Dear Software Committee Members, I wanted to respond to Bob Sugar's message, which Rich forwarded to us. I haven't looked over old notes to see precisely what we discussed in our earlier discussions, but wanted to give my responses. 1) Double Precision: From everything we know, we at CU and the RBC have done well with the single precision QCDSP machine and a sustained speed of 200-300 Gflops. We have substantial evidence that, with care, single precision is reliable for the quark masses, etc we can currently run. We do global sums in double precision. When studying zero modes and topology, where we may use a very small quark mass to check the chiral limit, single precision work requires testing against double precision to establish accuracy. I, for one, do not think this approach scales to sustained 5-10 Teraflops machines. It will impose limitations on the quark masses and volumes we can study. The human overhead of continually checking that results are not influenced by the use of single precision is a substantial hinderance to the physics goals of our collaboration. Reversibility questions in HMC trajectories become more problematic for large volumes and the larger number of steps required by small quark masses. Additionally, single precision performance numbers may look quite strange to any outside reviewer. I strongly support only double precision benchmarks and this is also what I thought we decided during our discussion. At this time, I haven't checked for concrete records to support my recollection. 2) Regarding the range of lattice sites used in the tests, I think it is sensible to use a range, but limit it. I consider lattices in the 4^4 to 8^4 range acceptable for the performance criteria. QCDOC will run 2^4 quite well, but this is a range where clusters will not do well. Beyond 8^4, we don't have the CPU power available to actually generate a long evolution of full QCD light quark lattices, so performance numbers in this range seem unrelated to our physics goals. I prefer to have the performance criteria mention the specific volume range that should be tested, since we do have concrete physics goals. Best, Bob ================================================================================= >From detar@dirac.physics.utah.edu Tue Aug 13 19:57:33 2002 Date: Tue, 13 Aug 2002 17:53:41 -0600 (MDT) From: Carleton DeTar To: brower@lns.mit.edu CC: cmendes@cs.uiuc.edu, djholm@fnal.gov, edwards@jlab.org, watson@jlab.org, rdm@phys.columbia.edu, brower@ctpup.mit.edu, avp@lns.mit.edu, simone@fnal.gov, osborn@physics.utah.edu, petrov@physics.columbia.edu, mdp@fnal.gov, dgr@jlab.org, detar@physics.utah.edu In-reply-to: <200208131748.g7DHmTr27603@ctpup.lns.mit.edu> (message from Richard Brower on Tue, 13 Aug 2002 13:48:29 -0400) Subject: Re: [brower@bucrf20.bu.edu: [sugar@sakar.physics.ucsb.edu: Software and Test Sections of the Proposal]] Hi Rich, Performance measures should be based on the precision we expect to require. On this the jury is still out. It is possible that improved actions will allow us to work with not-so-infinitesimal lattice spacings, so with lattice volumes and values of m_quark*a that do not require double precision in every calculation. Double precision is expensive, not so much because of processor demand, but mainly because of the extra burden it places on memory and communications bandwidth -- this goes for the QCDOC as well as clusters. So if spot checking gives confidence in the adequacy of single precision, we may not need to do wholesale double precision calculations. It is certainly possible that for most of the physics we want to do, our accuracy will be limited more by statistics than by precision. Is there some reason why we can't quote both single and double precision benchmarks? Regards, Carleton ================================================================================= >From edwards@jlab.org Tue Aug 13 20:43:18 2002 MIME-Version: 1.0 Date: Tue, 13 Aug 2002 20:43:21 -0400 From: Robert Edwards Organization: Jefferson Lab X-Accept-Language: en To: Richard Brower CC: cmendes@cs.uiuc.edu, detar@physics.utah.edu, djholm@fnal.gov, watson@jlab.org, rdm@phys.columbia.edu, brower@ctpup.mit.edu, avp@lns.mit.edu, simone@fnal.gov, osborn@physics.utah.edu, petrov@physics.columbia.edu, mdp@fnal.gov, dgr@jlab.org Subject: Re: [brower@bucrf20.bu.edu: [sugar@sakar.physics.ucsb.edu: Software and Test Sections of the Proposal]] Richard, I agree with Carleton's/Bob's sentiments that precision requirements are problem specific - just quote both numbers and let user's decide what precision they demand. However, it is *NOT* true that single and double precision are the same performance on Intel's - the floating point pipe on Intel's can issue in principle 4 single prec. floating ops and 2 in double precision. That is why on the graphs on your software poster (or the SciDAC poster) you'll see linear algebra in double prec. is half that of single prec. and the same is basically true for the Wilson dslash. There is more details (plots) in http://www.jlab.org/~edwards/qcdapi/dslash_p4.pdf Robert