Newsgroups: comp.parallel
From: Edwin Vollebregt <edwin@dutind2.twi.tudelft.nl>
Subject: Re: Shared Memory vs. Message Passing (was SMP vs MPP)
Keywords: Shared memory, Message passing
Organization: Delft University of Technology
Date: Thu, 19 Jan 1995 18:03:30 GMT
Message-ID: <D2nst9.5B0@dutiws.twi.tudelft.nl>


Hello,

It has been noted before in this discussion that shared and distributed 
memory are not specific to the hardware (SMP or MPP, whatever they stand
for), but are a way of viewing the parallel computer.  
  ``Macros can turn a system of one type into one of the other type''.  
How about comparing the shared memory paradigm with the message passing
paradigm ?

One source for confusion is the physically and logically sh.mem.

Shared memory == 
   x  one global address space
   x  all processors can access (read/write) all addresses.

Physically shared memory ==
   example :  traditional vector supers have it, albeit divided into
	memory banks.
   x  flat access time. For all processors the time to access any item
	is the same.
   x  can be implemented for instance using one or more buses, a 
	switching network or a crossbar network.
   x  Not scalable to many processors. Can be explained roughly by saying
	that it kind of requires a full interconnection between memory
	banks and processors.

Virtually / Logically shared memory ==
   x  has characteristics of shared memory (see above), but is not
	physically shared. Implies a less than full interconnection.
   x  --> non-flat access time.

Logically distributed memory ==
   x  Each process/processor has a local address space, and cannot access
	the address spaces of other processes.
   x  Message passing required for exchanging of data.
   x  If communication is slower than local memory access, then you have a
	non-flat access time.


My PERSONAL Conclusions.  Shared memory is easier to program, easier to get
something working in. Also easier to get an old sequential program working in.
However, it is at least as much work to get it as fast as in a message
passing implementation. An advantage is that communication is done
automatically, and some stuff can be done by hardware (cache coherency can
be done in hardware). In message passing you have more control. For
instance you may be able to tell where and when data is needed, and send 
it long before that. Another example is that you can group many data items
in one long message, which is less time consuming than many short messages.

A mixture of the two paradigms, such as in the Cray CRAFT model where you
can use message passing and global variables in the same program, is IMHO
the optimal solution.

Thanx for the time,
	Edwin
-- 
 __________________________________________________________________________
|                            |                                 |           |
| Ir. Edwin A.H. Vollebregt  |  Parallel Algorithms Group      |    0---0  |
|                            |  Section of Applied Mathematics |   /|  /|  |
| phone +31(0)15-785805      |  Delft University of Technology |  0---0 |  |
| fax   +31(0)15-787209      |  P.O. Box 5031                  |  | 0-|-0  |
| edwin@pa.twi.tudelft.nl    |  2600 GA Delft                  |  |/  |/   |
|                            |     The Netherlands             |  0---0    |
|____________________________|_________________________________|___________|



