Collective Communication

All or None:

• Collective communication must involve all processes in the scope of a communicator. All processes are by default, members in the communicator MPI_COMM_WORLD.

• It is the programmer's responsibility to insure that all processes within a communicator participate in any collective operations.

Types of Collective Operations:

• Synchronization - processes wait until all members of the group have reached the synchronization point.

• Data Movement - broadcast, scatter/gather, all to all.

• Collective Computation (reductions) - one member of the group collects data from the other members and performs an operation (min, max, add, multiply, etc.) on that data.

Programming Considerations and Restrictions:

• Collective operations are blocking.

• Collective communication routines do not take message tag arguments.

• Collective operations within subsets of processes are accomplished by first partitioning the subsets into new groups and then attaching the new groups to new communicators (discussed in the Group and Communicator Management Routines section).

• Can only be used with MPI predefined datatypes - not with MPI Derived Data Types.