568
社区成员
发帖
与我相关
我的任务
分享Getting Started with OpenMP－II
Managing Shared and Private Data
Nearly every loop (at least if it's a useful one) reads or writes memory, and it's the programmer's job to tell the compiler which pieces of memory should be shared among the threads and which pieces should be kept private. When memory is identified as shared, all threads access the exact same memory location. When memory is identified as private, however, a separate copy of the variable is made for each thread to access in private. When the loop ends, these private copies are destroyed. By default, all variables are shared except for the loop variable, which is private. Memory can be declared as private in the following two ways.
* Declare the variable inside the loop-really inside the parallel OpenMP directive-without the static keyword.
* Specify the private clause on an OpenMP directive.
The following loop fails to function correctly because the variable temp is shared. It needs to be private
// WRONG. Fails due to shared memory.
// Variable temp is shared among all threads, so while one thread
// is reading variable temp another thread might be writing to it
#pragma omp parallel for
for (i=0; i < 100; i++)
{
temp = array[i];
array[i] = do_something(temp);
}
The following two examples both declare the variable temp as private memory, which fixes the problem.
// This works. The variable temp is now private
#pragma omp parallel for
for (i=0; i < 100; i++)
{
int temp; // variables declared within a
parallel construct are, by definition, private
temp = array[i];
array[i] = do_something(temp);
}
//This also works. The variable temp is declared private
#pragma omp parallel for private(temp)
for (i=0; i < 100; i++)
{
temp = array[i];
array[i] = do_something(temp);
}
Every time you instruct OpenMP to parallelize a loop, you should carefully examine all memory references, including the references made by called functions. Variables declared within a parallel construct are defined as private except when they are declared with the static declarator, because static variables are not allocated on the stack.
Reductions
Loops that accumulate a value are fairly common, and OpenMP has a specific clause to accommodate them. Consider the following loop that calculates the sum of an array of integers.
sum = 0;
for (i=0; i < 100; i++){
sum += array[i]; // this
//variable needs to be shared to generate
//the correct results, but private to avoid
//race conditions from parallel execution
}
The variable sum in the previous loop must be shared to generate the correct result, but it also must be private to permit access by multiple threads. To solve this case, OpenMP provides the reduction clause that is used to efficiently combine the mathematical reduction of one or more variables in a loop. The following loop uses the reduction clause to generate the correct results.
sum = 0;
#pragma omp parallel for reduction(+:sum)
for (i=0; i < 100; i++){
sum += array[i];
}
Under the hood, OpenMP provides private copies of the variable sum for each thread, and when the threads exit, it adds the values together and places the result in the one global copy of the variable.
The following table lists the possible reductions, along with the initial variables-which is also the mathematical identify value-for the temporary private variables.
Multiple reductions in a loop are possible by specifying comma-separated variables and reductions on a given parallel construct. The only requirements are:
1. the reduction variables can be listed in just one reduction
2. they cannot be declared constant, and
3. they cannot be declared private in the parallel construct.
Nearly every loop (at least if it's a useful one) reads or writes memory, and it's the programmer's job to tell the compiler which pieces of memory should be shared among the threads and which pieces should be kept private. When memory is identified as shared, all threads access the exact same memory location. When memory is identified as private, however, a separate copy of the variable is made for each thread to access in private. When the loop ends, these private copies are destroyed. By default, all variables are shared except for the loop variable, which is private. Memory can be declared as private in the following two ways.
* Declare the variable inside the loop-really inside the parallel OpenMP directive-without the static keyword.
* Specify the private clause on an OpenMP directive.
The following loop fails to function correctly because the variable temp is shared. It needs to be private
// WRONG. Fails due to shared memory.
// Variable temp is shared among all threads, so while one thread
// is reading variable temp another thread might be writing to it
#pragma omp parallel for
for (i=0; i < 100; i++)
{
temp = array[i];
array[i] = do_something(temp);
}
The following two examples both declare the variable temp as private memory, which fixes the problem.
// This works. The variable temp is now private
#pragma omp parallel for
for (i=0; i < 100; i++)
{
int temp; // variables declared within a
parallel construct are, by definition, private
temp = array[i];
array[i] = do_something(temp);
}
//This also works. The variable temp is declared private
#pragma omp parallel for private(temp)
for (i=0; i < 100; i++)
{
temp = array[i];
array[i] = do_something(temp);
}
Every time you instruct OpenMP to parallelize a loop, you should carefully examine all memory references, including the references made by called functions. Variables declared within a parallel construct are defined as private except when they are declared with the static declarator, because static variables are not allocated on the stack.
Reductions
Loops that accumulate a value are fairly common, and OpenMP has a specific clause to accommodate them. Consider the following loop that calculates the sum of an array of integers.
sum = 0;
for (i=0; i < 100; i++){
sum += array[i]; // this
//variable needs to be shared to generate
//the correct results, but private to avoid
//race conditions from parallel execution
}
The variable sum in the previous loop must be shared to generate the correct result, but it also must be private to permit access by multiple threads. To solve this case, OpenMP provides the reduction clause that is used to efficiently combine the mathematical reduction of one or more variables in a loop. The following loop uses the reduction clause to generate the correct results.
sum = 0;
#pragma omp parallel for reduction(+:sum)
for (i=0; i < 100; i++){
sum += array[i];
}
Under the hood, OpenMP provides private copies of the variable sum for each thread, and when the threads exit, it adds the values together and places the result in the one global copy of the variable.
The following table lists the possible reductions, along with the initial variables-which is also the mathematical identify value-for the temporary private variables.
Multiple reductions in a loop are possible by specifying comma-separated variables and reductions on a given parallel construct. The only requirements are:
1. the reduction variables can be listed in just one reduction
2. they cannot be declared constant, and
3. they cannot be declared private in the parallel construct.
...全文
请发表友善的回复…
发表回复
flowersmokes122 2007-06-20
- 打赏
- 举报
Loop Scheduling
Load balancing, the equal division of work among threads, is among the most important attributes for parallel application performance. Load balancing is extremely important, because it ensures that the processors are busy most, if not all, of the time. Without a balanced load, some threads may finish significantly before others, leaving processor resources idle and wasting performance opportunities.
Within loop constructs, poor load balancing is usually caused by variations in compute time among loop iterations. It is usually easy to determine the variability of loop iteration compute time by examining the source code. In most cases, you will see that loop iterations consume a uniform amount of time. When that's not true, it may be possible to find a set of iterations that consume similar amounts of time. For example, sometimes the set of all even iterations consumes about as much time as the set of all odd iterations. Similarly it can happen that the set of the first half of the loop consumes about as much time as the second half. On the other hand, it may be impossible to find sets of loop iterations that have a uniform execution time. Regardless of which case an application falls into, you should provide this extra loop scheduling information to OpenMP so that it can better distribute the iterations of the loop across the threads (and therefore processors) for optimum load balancing.
OpenMP, by default, assumes that all loop iterations consume the same amount of time. This assumption leads OpenMP to distribute the iterations of the loop among the threads in roughly equal amounts and in such a way as to minimize the chances of memory conflicts due to false sharing. This is possible because loops generally touch memory sequentially, so splitting up the loop in large chunks-like the first half and second half when using two threads-will result in the least chance for overlapping memory. While this may be the best choice for memory issues, it may be bad for load balancing. Unfortunately, the reverse is also true; what may be best for load balancing may be bad for memory performance. Performance engineers, therefore, must strike a balance between optimal memory usage and optimal load balancing by measuring the performance to see what method produces the best results.
Loop scheduling information is conveyed to OpenMP on the parallel construct with the following syntax.
#pragma omp parallel for schedule(kind [, chunk size])
Four different loop scheduling types (hints) can be provided to OpenMP, as shown in the following table. The optional parameter (chunk), when specified, must be a loop-invariant positive integer.
Examples
Problem: Parallelize the following loop
for (i=0; i < NumElements; i++){
array[i] = StartVal;
StartVal++;
}
Solution: Watch for data dependencies
As written, the loop contains a data dependency, making it impossible to parallelize without a quick change. The new loop, shown below, fills the array in the same manner, but without data dependencies. As a bonus, the new loop can also be written using the SIMD instructions.
#pragma omp parallel for
for (i=0; i < NumElements; i++){
array[i] = StartVal + i;
}
Observe that the code is not 100% identical because the value of variable StartVal is not incremented. As a result, when the parallel loop is finished, the variable will have a value different from the one produced by the serial version. If the value of StartVal is needed after the loop, the additional statement, shown below, is needed.
// This works and is identical to the serial version.
#pragma omp parallel for
for (i=0; i < NumElements; i++){
array[i] = StartVal + i;
}
StartVal += NumElements;
Summary
OpenMP was written primarily to relieve the programmer from the details of threading, enabling a focus on the more important issues. Using the OpenMP pragmas, most loops can be threaded with one simple statement. This is the power of OpenMP and where a key benefit lies. This white paper has introduced the concepts and simpler side of OpenMP to get you started. The next two papers will build on this foundation making it possible to use OpenMP to thread more complex loops and more general constructs.
References
OpenMP Specification: http://www.openmp.org
Load balancing, the equal division of work among threads, is among the most important attributes for parallel application performance. Load balancing is extremely important, because it ensures that the processors are busy most, if not all, of the time. Without a balanced load, some threads may finish significantly before others, leaving processor resources idle and wasting performance opportunities.
Within loop constructs, poor load balancing is usually caused by variations in compute time among loop iterations. It is usually easy to determine the variability of loop iteration compute time by examining the source code. In most cases, you will see that loop iterations consume a uniform amount of time. When that's not true, it may be possible to find a set of iterations that consume similar amounts of time. For example, sometimes the set of all even iterations consumes about as much time as the set of all odd iterations. Similarly it can happen that the set of the first half of the loop consumes about as much time as the second half. On the other hand, it may be impossible to find sets of loop iterations that have a uniform execution time. Regardless of which case an application falls into, you should provide this extra loop scheduling information to OpenMP so that it can better distribute the iterations of the loop across the threads (and therefore processors) for optimum load balancing.
OpenMP, by default, assumes that all loop iterations consume the same amount of time. This assumption leads OpenMP to distribute the iterations of the loop among the threads in roughly equal amounts and in such a way as to minimize the chances of memory conflicts due to false sharing. This is possible because loops generally touch memory sequentially, so splitting up the loop in large chunks-like the first half and second half when using two threads-will result in the least chance for overlapping memory. While this may be the best choice for memory issues, it may be bad for load balancing. Unfortunately, the reverse is also true; what may be best for load balancing may be bad for memory performance. Performance engineers, therefore, must strike a balance between optimal memory usage and optimal load balancing by measuring the performance to see what method produces the best results.
Loop scheduling information is conveyed to OpenMP on the parallel construct with the following syntax.
#pragma omp parallel for schedule(kind [, chunk size])
Four different loop scheduling types (hints) can be provided to OpenMP, as shown in the following table. The optional parameter (chunk), when specified, must be a loop-invariant positive integer.
Examples
Problem: Parallelize the following loop
for (i=0; i < NumElements; i++){
array[i] = StartVal;
StartVal++;
}
Solution: Watch for data dependencies
As written, the loop contains a data dependency, making it impossible to parallelize without a quick change. The new loop, shown below, fills the array in the same manner, but without data dependencies. As a bonus, the new loop can also be written using the SIMD instructions.
#pragma omp parallel for
for (i=0; i < NumElements; i++){
array[i] = StartVal + i;
}
Observe that the code is not 100% identical because the value of variable StartVal is not incremented. As a result, when the parallel loop is finished, the variable will have a value different from the one produced by the serial version. If the value of StartVal is needed after the loop, the additional statement, shown below, is needed.
// This works and is identical to the serial version.
#pragma omp parallel for
for (i=0; i < NumElements; i++){
array[i] = StartVal + i;
}
StartVal += NumElements;
Summary
OpenMP was written primarily to relieve the programmer from the details of threading, enabling a focus on the more important issues. Using the OpenMP pragmas, most loops can be threaded with one simple statement. This is the power of OpenMP and where a key benefit lies. This white paper has introduced the concepts and simpler side of OpenMP to get you started. The next two papers will build on this foundation making it possible to use OpenMP to thread more complex loops and more general constructs.
References
OpenMP Specification: http://www.openmp.org
