I think there is an issue about boundaries as to what is ParallelProgramming
. I had tried to be clear about it when I started ParallelProgramming
, but it seems that I have not been clear and that shows up as a problem worth discussing.
There are pages which link from DistributedComputing
which discuss issues of the patterns of programming. They include quite a lot of topics as can be seen from the fact that it has 56 back links.
I intended ParallelProgramming
to be about taking a serial program solving some numerical problem and turning it into a parallel program which will run on several computers in a closecoupled way. Maybe it is wrongly named, in which case it could be renamed something more specific. I now suggest ProgrammingForParallelComputing
. -- JohnFletcher
I think of DistributedComputing
as distributing a task among separate computers where it is expensive to share resources between nodes. I think of ParallelProgramming
as exploiting fine-grain parallelism in a problem, where sharing resources is easier. Examples: MultiThreading?
to take advantage of SymmetricMultiprocessing
, and decomposing algorithms to use SIMD (SingleInstructionMultipleData?
) to take advantage of vector units such as AltiVec
s. -- IanOsgood
Here are some definitions from WikiPedia
On the page http://en.wikipedia.org/wiki/Parallel_computing.
is the simultaneous execution of the same task
(split up and specially adapted) on multiple processors in order to obtain results faster. The idea is based on the fact that the process of solving a problem usually can be divided into smaller tasks, which may be carried out simultaneously with some coordination.
A parallel programming model
is a set of software technologies to express parallel algorithms and match applications with the underlying parallel systems. It encloses the areas of applications, languages, compilers, libraries, communication systems, and parallel I/O. People have to choose a proper parallel programming model or a form of mixture of them to develop their parallel applications on a particular platform.
The items listed include ParallelVirtualMachine and MessagePassingInterface.
On the page http://en.wikipedia.org/wiki/Parallel_programming
is a computer programming
technique that provides for the execution of operations in parallel, either within a single computer, or across a number of systems. In the latter case, the term distributed computing
On the basis of the above I think that what I am discussing can be described as ProgrammingForParallelComputing
which has to use a ParallelProgrammingModel
. That has the virtue of being consistent with the WikiPedia
usage. -- JohnFletcher
Category and Concurrency
There are two categories CategoryConcurrency
There are also related pages called SynchronizationStrategies
. There is some discussion about how these topics are related.
- Concurrency is when two tasks or threads are active at the same time.
- Synchronization is when the two tasks need to interact.
There is an Edit
Hint on CategoryConcurrencyPatterns
is the same thing and that they should be merged.
I think that ProgrammingForParallelComputing is a distinct area from the above which could have a category of its own but does not yet. I would welcome suggestions or contrary views.
There is also the interesting book PatternsForParallelProgramming