Software Coordinating Committee Conference Call March 17, 2005 1:00 PM EST Recorder: C. DeTar Present: DeTar, Osborn, Brower, Edwards, Holmgren, Morton, Zhang, Renner, Efstratios Efstathiadis (Stratos), Eric Blum, Watson, Petrov, Pochinsky, Simone, Steve Gottlieb, Jung Absent: Mawhinney, Maresca, ** action items ===================================================================== 1.) Progress on Asqtad Level 3 tests/interface to MILC? (Dru, Chulwoo and Carleton) Renner: I have plugged in the Level 3. Not converging.and hangs. Even with just Chulwoo's test code. I found a memory leak and Chulwoo fixed it. Jung: The other inverters (odd dimension, single precision) are almost done. Jung: I am discovering that we need the most rapidly varying dimension (time) to be even. This is necessary in order to do strided gathers with uniform blocks. DeTar: One of our dimensions is always even. Perhaps we can rotate the dimensions logically to achieve this. Gottlieb: If it is easiest to get code ready and working with this restriction, you should do it. We need something running soon. Then remove the restriction later as needed. DeTar: We will also need to work on optimizing the fermion force term. 2.) Progress for Chroma on QCDOC (James and Robert) Edwards: The code has been tested and run on 1000 nodes. DWF Dslash has been optimized. We are using Bagel generated code. Brower: James should get the HMC code running. 3.) What are the actual disk storage requirement for LHPC Structure function at the USQCD? (Future discussion) 4.) ILDG file format DeTar: Dirk Pleiter wants a response from us concerning their proposal for the ILDG file format. [see attached message]. The format seems to be compatible with ours. Edwards: Can QIO read an ILDG file? DeTar: We would have to modify it. I'll need to look into it. Brower: Then we should reply that we need another week to think about it. File splitting/reassembly Petrov: We have two modes of splitting: one to /qhost where the paths are the same and one to the multiple RAID boxes. Do we want one executable with a flag or two executables? DeTar: I think two would be fine. Kerberos at BNL Gottlieb: When will we have a Kerberos client installed on the BNL machine? We need it to be able to move files between FNAL and BNL. Blum: That won't happen at BNL for now. We are working on other solutions. You can use ftp. Gottlieb: Not if the other site refuses ftp. Watson: Rich, this issue needs to be escalated to a higher level. Will you do this? ** Brower: Yes ===================================================================== Committee conference concluded at 2:20 PM EST. Next plenary call March 24 at 1:00 PM EST ===================================================================== Date: Fri, 18 Feb 2005 14:56:21 +0900 Posted: Fri, 18 Feb 2005 14:55:57 +0900 From: Tomoteru Yoshie Subject: (qcdml 433) Re: data format To: qcdml@rccp.tsukuba.ac.jp, ildg-middleware@forge.nesc.ac.uk In-Reply-To: <20050208125831.GA27566@ifh.de> (message from Dirk Pleiter on Tue, 8 Feb 2005 21:58:48 +0900) X-ML-Name: qcdml X-Mail-Count: 433 X-MLServer: fml [fml 4.0.3 release (20011202/4.0.3)]; post only (anyone can post) X-ML-Info: If you have a question, please contact qcdml-admin@rccp.tsukuba.ac.jp; X-Authentication-Warning: localhost.localdomain: yoshie set sender to yoshie@rccp.tsukuba.ac.jp using -f X-Scanned-By: MIMEDefang 2.43 Dear Dirk, I agree to all of your summary/proposals on ILDG file format. For confirmation, I understand you propose that one file includes at least three lime messages, and one message has only one record with type "ildgFormat", one message has only one record with type "ildgDataLFN", one message has only one record with type "ildgBinary". (order of appearance of messages is not fixed). Is this correct? (In lime, one message may have many records, I think.) Let us wait for responses from UKQCD and USQCD. with best regards, Tomoteru > Date: Tue, 8 Feb 2005 21:58:48 +0900 > Posted: Tue, 8 Feb 2005 13:58:31 +0100 > From: Dirk Pleiter > > Dear all, > > My impression was that at the last ILDG meeting we were close to an > agreement on the data format (and we definitely committed ourselves to > agree until end of February, i.e. during the next 3 weeks). Re-calling > the discussions in November/December, I think that there are still a > few details to be fixed before people can really start to use the new > ILDG format. > > Let me try to summarise on what seems to be agreed while adding a few > questions on open issues: > > 1) Gauge configurations are stored in a set of records packed in a LIME > file. > > 2) Three records are foreseen which contain > - an ildgFormat XML document > - a dataLFN > - the binary data > > --> question: Do we foresee a fixed order? > [I propose no] > > --> question: Will non-ILDG records be allowed? > [I think yes] > > --> question: What are the type strings? > [my proposal: "ildgFormat", "ildgDataLFN", "ildgBinary"] > > 3) The ildgFormat XML document contains a minimal set of non mutable > data which is needed to read the binary file, i.e. information on > a) endianess (big or little) > b) precision (number of bits per floating point number) > c) lattice size > > Here the sample document from Chips ILDG5 presentation: > > > > 1.0 > big > 32 > 20 20 20 64 > > > 4) The binary data is stored using the 3x3 NERSC layout leaving only > two free parameters: 32/64 bits, bit/little endian. > > Comment concerning ildgDataLFN: > - I prefer the order of the records either not to fixed or otherwise > to put the ildgDataLFN record to the end. This would be helpful for > implementing an automatic mechanism for updating or appending this > record during submission without re-generating the full LIME file. > Advantage for the user: the dataLFN does not have to be known when > a configuration is generated. Alternative suggestion by Tomoteru: > generate "empty" record with a payload of fixed size, say 1024 bytes. > > If we could agree on this data format issues, then I would volunteer to > start preparing a short document. > > Dirk > > +------------------------------------------------------------------------+ > | NIC/DESY Zeuthen (33762) 77-381 (phone, office) | > | Platanenallee 6 (160) 944 366 21 (mobile) | > | D-15738 Zeuthen, Germany (33762) 77-216 (fax, office) | > +------------------------------------------------------------------------+ > Caminante, no hay camino, se hace camino al andar. (A.Machado) >