Newsgroups: comp.parallel.pvm
From: eve_li@btgmax.zko.dec.com (Eve Li)
Subject: Re: pvm message passing timing
Organization: Digital Equipment Corporation
Date: 12 Aug 1995 17:24:42 GMT
Message-ID: <40io4q$9ti@nntpd.lkg.dec.com>

Benno Overeinder (bjo@fwi.uva.nl) wrote:
: >       receiving processor.
: It all depends on what you want to measure.  I can imagine that
: if you are interested in the performance of some piece of code
: you might be interested in the cpu time.  But then, you are
: not measuring communication times or even idle times of tasks
: waiting for message (load imbalances) which is a fairly important
: thing in parallel processing.  What remains is indeed gettimeofday.

: In shared environments, such as networks of workstations, it difficult 
: (or may I say impossible?) to get rid of measurement disturbations by
: influences of other processes.   If you have accepted this, the only 
: thing that remains is to do some decent stastics to make sure you have
: some confidence in the results, viz., your measurements are not a 
: result of some random process called `the system'.

Good Point.
Since networks of workstations has radom distrubations, it will be ideal
to use some kind of experiment design that will get rid of such
disturbations for the whole experiment. Has anyone  done measurement of 
parallel systems in such an environment? Is there some better way to 
compare two or more  such  systems besides taking samples alternatively
and taking the avaerage (see below)?  


Here is what a method that takes samples for two parallel systems 
alternatively and taking the average: 

assumptions: 
the only difference of two parallel systems are the algorithm running on it.
The hardwares are the same. 

Parallel system  1 runs algorithm A on a networks of workstaions (thus has
disturbations).  
Parallel system  2  runs algorithm B on the same networks of workstations. 

In order to compare two system, take measures as follows:
measure system 1 (using gettimeofday),    
measure system2  ( """")

measue system 1 again
measure system 2 again 

....

after , say 10 measures, take the average of the measures of system 1  and
system 2 , then compare them. 


Is there any flaw of the above system? Of course, one obvious flaw is
that the spike noise  can hurt the measure . But as long as the noise
is persistent during a measure of two systems, it will be balanced out.
Very interesting problem, isn't it?

eve 
: Benno
: -- 
:  Benno Overeinder			 Computer Systems Department	     
:  voice: (+31) 20 525 7536		 University of Amsterdam	    
:  fax:   (+31) 20 525 7490		 Kruislaan 403, 1098 SJ  Amsterdam
:  e-mail: bjo@fwi.uva.nl		 	 The Netherlands

