Newsgroups: comp.parallel.pvm
From: papadopo@cs.utk.edu (Philip Papadopoulos)
Subject: Re: PVM Reduce on the Paragon?
Organization: CS Department, University of Tennessee, Knoxville
Date: 13 Sep 1995 10:03:17 -0400
Message-ID: <436ob5INNj7m@duncan.cs.utk.edu>

In article <433nlb$25vg@info4.rus.uni-stuttgart.de> lepper@ivd.uni-stuttgart.de writes:
>Hi,
>
>I noticed that a call to PVMFREDUCE is awfully slow on a Paragon using
>PVM Version 3.3.8.
You should make the group static with pvm_staticgroup. Also make sure
that the group is the same size as the partition that PVM is managing.
If you do both of these things, then the group code will use the
the native reduction routine (eg. gisum, gssum, gdsum, g<x>high, g<x>low,
g<x>prod, <x>=[i,s,d]), if it exists.

>Does anybody know whether this call uses "PvmDataDefault" as default and
>therefore packs the data ?
Yep. sure does, unless the native routine is used. 
>
>I have no other explanation for this behaviour, because a simple "all send
>their values to one who numerically performs the global operation and sends
>the result back to all other" with PVMFPSEND is noticeable faster.

It is probably a useful optimization to use psend on single mpps. It
takes too much time to figure out if multiple machines are of the same
architecture type so that psend (which sends raw data) would be 
compatible.

psend on the PGON is noticebly faster as the message size grows. 
Why is this? THe PGON is really slow at bcopying local memory to
local memory. The provided bcopy routine in the c library peaks
at about 65-70 MB/sec.  For Comparison, a transfer of a large
message from 1 processor to a different processor can attain 
speeds of 130Mb/sec (for the most recent NIC (network interface chip)).
So copying local memory is about twice as slow as getting a message
from your neighbor (counter-intuitive, yes?).  Packing a PVM
message relies on bcopy on both sender  and receiver (you 
gotta put that non-unit stride data into unit-strided memory 
for sending/receiving). SO a PVM-packed message on the
PGON is roughly 2 bcopies + send, or about 5 times slower than
psend. 
>
>Any comments on this?
The PGON isn't an easy machine to work with!
>
>With best regards
Philip Papadopoulos




