Newsgroups: comp.parallel
From: nmm1@cus.cam.ac.uk (Nick Maclaren)
Subject: Re: parallel pseudo random number generator
Organization: University of Cambridge, England
Date: 13 Apr 1995 18:52:49 GMT
Message-ID: <3n370n$jt3@usenet.srv.cis.pitt.edu>

In article <3m95kj$kln@usenet.srv.cis.pitt.edu>,
J.J.A. Koot <koot@sara.nl> wrote:
>May this is a FAQ, may be other news groups are better suited for this
>question. If you think so, please mail me personally (koot@sara.nl).
>I am looking for a pseudo random number generator for use in a
>parallel program using message passing. Efficiency requires that each
>task has its own generator and that this generator can produce most of
>the random numbers without communication with other tasks. The
>sequences produced in different tasks must be statistically
>independent from each other.  I have asked this question earlier, but
>received no answer. 
>
>In principle it is simple (i think) to prepare the desired pseudo
>random number generator from a normal sequential one, namely as
>follows. 
>
>1: define a subset of all the states the generator can have
>   (usually one or more seeds)
>2: make sure that there is a simple test telling whether or not
>   a state belongs to the subset. (In fact the test is not
>   necessarily simple. It must be fast in most cases).
>3: There is a server for initial states. On demand of a task
>   the server delivers an initial state from the subset. The
>   server never delivers a state already delivered earlier.
>4: The generator of a task asks the server for an initial state.
>   After every produced number test 2 is made. If the new state
>   belongs to the subset (1) then the server must be contacted
>   for another state.
>In this way the generators of the different tasks produce
>non overlapping sequences.

This will NOT give independent sequences, but only disjoint ones!
In particular, this is easy to do with multiplicative congruential
generators, and gives catastrophic dependence between the processes.
The same is true (theoretically) for ones based on polynomials
(including shift register etc.) but the dependence is not as
catastrophic.

There are three known, reliable strategies:

    1) To work out the interdependence of the sequences, and ensure
that it is within acceptable bounds.  See papers by De Matteis and
others.

    2) To use multiple generators with known 'independence' properties.
There are precisely 2 papers that I know of on this - mine in Applied
Statistics, and a more recent Russian one (both based on coprime
moduli for multiplicative congruential generators).

    3) To use a cryptographic generator and pray :-)  If you find a
serious dependence in DES, for example, please publish it and give the
NSA sleepless nights ....


Nick Maclaren,
University of Cambridge Computer Laboratory,
New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.
Email:  nmm1@cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

