Newsgroups: comp.parallel.pvm
From: jim@ilo.dec.com (Jim Lawton)
Subject: Re: pvm 3.3.8 and ALPHAMP
Organization: Digital Equipment Corporation
Date: 9 Aug 1995 09:05:20 GMT
Message-ID: <409tog$5tv@mrnews.mro.dec.com>


Stefan,

> The last line says the same thing about no space on a device, probably TMP.

In both cases, PVM is complaining about no space, not in /tmp, but in the 
shared-memory space. This is either because there are lots of orphan 
shared-memory segments lying around from a previous run of PVM or because you 
have set the PVMBUFSIZE parameter to a value larger than the max size allowed
for a shared-memory segment in the kernel (which can be changed by modifying 
the kernel config file and rebuilding the kernel).

You can use /usr/ucb/ipcs to show all the currently outsanding System V IPC
resources (semaphores, message queues and shared-memory segments). Then,
/usr/ucb/ipcrm can be used to remove those segments/semaphores that are left
over from a previous run of PVM (perhaps one that crashed).

Here is a simple shell script that I use to do this (be careful if you 
run it as root, as it may delete other semaphores/segments that weren't 
allocated by PVM).

----------------------------ipcclean------------------------------
#!/bin/sh
if [ $USER = "root" ]
then
	setenv USER " "
fi
echo "Clearing message queues"
for i in `ipcs|grep $USER|awk '$1 == "q" {print $2}'`
do
	echo $i
	ipcrm -q $i
done
echo "Clearing shared memory"
for i in `ipcs|grep $USER|awk '$1 == "m" {print $2}'`
do
	echo $i
	ipcrm -m $i
done
echo "Clearing semaphores"
for i in `ipcs|grep $USER|awk '$1 == "s" {print $2}'`
do
	echo $i
	ipcrm -s $i
done
#last line
------------------------ipcclean------------------------

Now, it is possible that, after you do this, you may still be running
out of shared-memory segments. If so, then you can try setting the PVMBUFSIZE
environment variable (if you haven't done so already), or rebuilding PVM with a 
new SHMBUFSIZE (see ~/pvm3/Readme.mp for more info). If that still doesn't get
you enough shared-memory, you may need to rebuild the kernel. 

The System Administration manual (AA-PS2RB-TE is the one I have) page 5-14
describes the constants you need to set in the kernel config file (this is 
pretty standard Unix stuff). The sys admin manual, like all the manuals, is
on your OSF/1 CDROM in the DOCUMENTATION directory.

You can see what the kernel values are set to by doing 

	# /sbin/sysconfig -q ipc

This will return something like:

msg-max = 8192
msg-mnb = 16384
msg-mni = 64
msg-tql = 40
shm-max = 4194304              <<<--- max shared-memory segment size
shm-min = 1
shm-mni = 128
shm-seg = 32
sem-mni = 16
sem-msl = 25
sem-opm = 10
sem-ume = 10
sem-vmx = 32767
sem-aem = 16384
num-of-sems = 60
max-kernel-ports = 22487
port-hash-max-num = 1124350
port-reserved-max-num = 22487
set-max-num = 1029

There is also some other information in ~/pvm3/Readme.mp that you should be 
aware of. Here is the relevant section:

        4. When a task sends a message to another task, only the pointer
        is passed. The receiving task should call pvm_freebuf() to "destroy"
        the receive buffer after the data have been unpacked, and allow the
        sender to reuse that space.

        5. If you save any PVM buffers (with pvm_setsbuf, for example), 
        it is important that you free them before calling pvm_exit(), otherwise
        your process will hang.


I hope one of these points helps to fix your problem,

/jim

+---------------------------------------------------------------+
| Jim Lawton                      Digital Equipment Corporation | 
|                    Technical Computing Group, Galway, Ireland |
| Phone: +353-91-754759                     Fax: +353-91-754435 | 
| Email: jim@ilo.dec.com (MIME OK) Not an offical spokesperson! |
+---------------------------------------------------------------+

