Newsgroups: comp.parallel
From: jaynes@sequent.com (Steven Jaynes)
Subject: Re: Shared Memory vs. Message Passing (was SMP vs MPP)
Organization: Sequent Computer Systems, Inc.
Date: Fri, 27 Jan 1995 13:49:40 GMT
Message-ID: <1995Jan26.184947.3975@sequent.com>

Edwin,

I tend to agress with your observations.  I would state it even
stronger concerning the difficulty of porting or conversions of large
codes to distributed memory systems.  There, you have the double
problem of code porting + memory management, if the data or code is
of sufficent size to preclude running on one "node".

Although perhaps trite, my observations is that in parallel processing, as
in real estate, the real problem is location, location, location. 

When working with pure SMP systems )Sequent), what you identify as logical or
virtual shared systems (KSR), and distributed memory systems (Intel),
 performance tuning all boils down to locality of data.  This of
course even applies to single processor machines with cache based
designs.  How much delay to "the data you don't have", is the bottom line.

I think many people fail to recognize that there are at least three
separate but linked tasks of:

1) Porting
2) Parallization
3) Tuning for performance & robustness (this means it doesn't crash or
yield wrong answers.)

Like tuning any complex system (including cars) one of the above will
likely affect one or more of the others.  That's why people port codes,
not computers. 

Off soap box.


 Edwin Vollebregt (edwin@dutind2.twi.tudelft.nl) wrote:

: Hello,

: It has been noted before in this discussion that shared and distributed 
: memory are not specific to the hardware (SMP or MPP, whatever they stand
: for), but are a way of viewing the parallel computer.  
:   ``Macros can turn a system of one type into one of the other type''.  
: How about comparing the shared memory paradigm with the message passing
: paradigm ?

: One source for confusion is the physically and logically sh.mem.

: Shared memory == 
:    x  one global address space
:    x  all processors can access (read/write) all addresses.

: Physically shared memory ==
:    example :  traditional vector supers have it, albeit divided into
: 	memory banks.
:    x  flat access time. For all processors the time to access any item
: 	is the same.
:    x  can be implemented for instance using one or more buses, a 
: 	switching network or a crossbar network.
:    x  Not scalable to many processors. Can be explained roughly by saying
: 	that it kind of requires a full interconnection between memory
: 	banks and processors.

: Virtually / Logically shared memory ==
:    x  has characteristics of shared memory (see above), but is not
: 	physically shared. Implies a less than full interconnection.
:    x  --> non-flat access time.

: Logically distributed memory ==
:    x  Each process/processor has a local address space, and cannot access
: 	the address spaces of other processes.
:    x  Message passing required for exchanging of data.
:    x  If communication is slower than local memory access, then you have a
: 	non-flat access time.


: My PERSONAL Conclusions.  Shared memory is easier to program, easier to get
: something working in. Also easier to get an old sequential program working in.
: However, it is at least as much work to get it as fast as in a message
: passing implementation. An advantage is that communication is done
: automatically, and some stuff can be done by hardware (cache coherency can
: be done in hardware). In message passing you have more control. For
: instance you may be able to tell where and when data is needed, and send 
: it long before that. Another example is that you can group many data items
: in one long message, which is less time consuming than many short messages.

: A mixture of the two paradigms, such as in the Cray CRAFT model where you
: can use message passing and global variables in the same program, is IMHO
: the optimal solution.

: Thanx for the time,
: 	Edwin
: -- 
:  __________________________________________________________________________
: |                            |                                 |           |
: | Ir. Edwin A.H. Vollebregt  |  Parallel Algorithms Group      |    0---0  |
: |                            |  Section of Applied Mathematics |   /|  /|  |
: | phone +31(0)15-785805      |  Delft University of Technology |  0---0 |  |
: | fax   +31(0)15-787209      |  P.O. Box 5031                  |  | 0-|-0  |
: | edwin@pa.twi.tudelft.nl    |  2600 GA Delft                  |  |/  |/   |
: |                            |     The Netherlands             |  0---0    |
: |____________________________|_________________________________|___________|


-- 
Steve Jaynes, Sequent Computer Systems, Inc., Beaverton, OR. "What I write
is my own mistake.  Don't blame my employer!" (Facsimile of a disclaimer)



