Newsgroups: comp.parallel.mpi
From: ah@dcs.ed.ac.uk (Anna Hondroudakis)
Subject: Questions for parallel program developers (2nd attempt)
Organization: Department of Computer Science, University of Edinburgh
Date: Sun, 22 Jan 1995 15:37:58 GMT
Message-ID: <D2tCrA.LDy@dcs.ed.ac.uk>



A couple of months ago I posted some questionnaires to this newsgroup. Some of
you have replied to me and I thank you a lot. I am reposting the questionnaires
hoping that more people could help me with these. After this attempt, I will
send  a compilation of the answers to all those who have expressed an interest
in them.


-------------------------------------------------------------------------------
My name is Anna Hondroudakis and I am a PhD student at the department of 
Computer Science of Edinburgh University.

You would be of great help to my research if you answered some questions that
have been posted as three separate postings. The questions
are addressed to three types of people: People who are parallel program 
developers, people who are tool developers and people who are project 
leaders. In this way I try to capture the practices concerning the tools for 
parallel program development from a number of different aspects.  
Please, answer one or more than one set of questions included in this and in
the rest of the postings and send it(them) to my email address:
ah@dcs.ed.ac.uk. If you are interested in my findings, I will send you a 
summary when a sufficient number of answers are received. 

Thank you very much,

Anna Hondroudakis


QUESTIONS FOR PARALLEL PROGRAM DEVELOPERS (1 of 3)

My research is concerned with parallel program performance evaluation and 
visualization tools, or more commonly tuning tools. I am specifically
interested in seeing from a Human Factors point of view the shortcomings
of existing tools and in identifying the needs of the users regarding these 
tools.  I need to see what people do with the tuning tools, e.g, if they use 
them and how and what their observations regarding the tools' usability are.

All the above are relative to "tuning in the small", which is concerned with a 
single tuning unit; The run of the program reveals areas for possible
performance improvements, some parameters and/or the program is changed and 
the program is run again to see whether the performance is actually improved. 

Similarly, I think that emphasis should be given to "tuning in the large" which
is the repetition of the tuning task over time until the tuner is satisfied
with the performance of the parallel program, or the repetition of tuning
every time some parameters of the parallel system change. For example, tuning
has to be resumed when the program has to be ported to a new version of the
programming library or even to a new machine. I am trying to capture this 
process of tuning in the large, see how it is done, what the needs are 
regarding human resources and documentation.


-------------------------------->8 cut here 8<---------------------------------

                                QUESTIONNAIRE


Name: 
-----

Email:
-----

       
PERSONAL DETAILS
  

1)  Could you give some details about the organization you work for? 
(Skip this question if you have already answered it in a previous set of 
questions)
  

      Name 	




      Number of employees


      Any other information, machine resources etc. 


 

2) How many years of experience do you have as a parallel developer?


3) What is your educational background?
 

4) Have you had any training in tuning?

a) Yes b) No

5) Is there any guide in your organization on how to perform tuning? 
a) Yes b) No

6) In how many projects have you been involved with tuning?




7) What makes tuning hard?

a) The difficulty in finding and eliminating the cause of the bad performance
b) The difficulty in keeping track of the changes done
c) Other :



SPECIFIC TUNING EXPERIENCE


Describe a recent tuning experience in terms of :


8) Could you describe your application in terms of :

Project Aims:



Application field:



Machine resources, number of processors used:



Development tools (languages etc):



Data or program decomposition issues:



Lines of code:


Number of people involved:



9) At what stage in the project development cycle was tuning performed? I.e:

a) at the final stage
b) interleaved with other stages
c) other

     


10) What was the performance problem that the application program presented ? 



IF YOU USED A TOOL::


11) If you used a performance evaluation and visualization tool, how is it 
called? 



12) How did the tool help you find the cause of the performance problem?



13) What aspects of the program behaviour could you see by using the tool?

  

14) What else would you like from the tool that it wasn't provided ? 



IF YOU DIDN'T USE A TOOL 

15) If you didn't use a tool which was the reason for not using one?
(Check all that apply)

Non availability in our environment                           [ ]
Couldn't afford the time to learn to use it                   [ ]
We tried to use a tool but it wasn't good enough              [ ] (please name
the tool ) name of tool = 


It was too expensive to obtain                                [ ]
We didn't know whether a suitable tool existed                [ ]
Performance issues weren't significant                        [ ]
Any other reason:



IN ANY CASE:




16) How did you solve the performance problem?




17) Did  you follow a specific "problem solving" strategy? Could you 
say that there is a strategy? How do you go about the task? 


18) How many times did you have to repeat tuning before the program presented 
the desired performance?




19) How long did tuning take altogether?  


20) How much faster did the optimized program run than the initial 
implementation?



21) In which context did your tuning experience happen? E.g. did you have
any time or resource constrains?  


  

22) Was tuning an identified stage of the project? That is was time intended 
to be spent in tuning? 
a) Yes
b) No



23) How many people were involved in this case?
a) 1
b) 2-3
c) Other:




GENERALLY ABOUT TUNING


 
-------------------------------------------------------------------------------
We think of the tuning task as a repetitive process consisting of tuning units 
(see question 18 ). These questions address the documentation needs in tuning.
-------------------------------------------------------------------------------

24) How long does tuning typically take you? 

a) less than a day   b) several days        c) several weeks    d) other
 

25) How many parameters do you usually have to take into account
when you tune your program? 

a) 1-3   b)5 - 10      c)other
 

26) Which are these parameters? E.g.

	
a) number of processors  b) problem size   c) machine topology  
d) mapping of processes on processors  e) communication patterns
f) data distribution  g) other:  
  


27) How many different versions of the program do you keep in order to 
make comparisons and decide what is the best version of the program?  
a) 1--3   b) 5- 10   c) other:
 

28) Do you document your tuning effort? If yes how? 

a) document the source code   b) separate on-line file  c) paper notes d)other


 
29) What sort of things do you normally write down? 

a) values of parameters   b) performance data   c) program changes  d) graphs
e) other
 


30) What are the reasons for documenting your tuning efforts? 
a) as a reminder of the progress made    b) to keep track of the changes
c) to share tuning notes with other people in a cooperative working
environment d) other reasons







 



 

 



-- 
===============================================================================
Anna Hondroudakis         Dept. of Computer Science   Edinburgh University
ah@dcs.ed.ac.uk           Tel.  +44 (0)31 650 5115    Fax. +44 (0)31 667 7209
===============================================================================

