出错提示term does not evaluate to a function什么意思

chinesefys2012 2011-08-24 09:17:27
出错提示term does not evaluate to a function什么意思
...全文
67685 11 打赏 收藏 转发到动态 举报
写回复
用AI写文章
11 条回复
切换为时间正序
请发表友善的回复…
发表回复
vipcxj 2014-01-01
  • 打赏
  • 举报
回复
http://msdn.microsoft.com/en-us/library/z72c1dfd.aspx 为啥就不会用google,baidu,msdn呢? 人家一下就举了5个例子,没有比例子更容易理解的东西了
mr三零四 2014-01-01
  • 打赏
  • 举报
回复
你也在编勒让德函数?
赵4老师 2013-10-30
  • 打赏
  • 举报
回复
偶遇到类似问题都是用 “每次用/*...*/注释掉不同部分再重新编译,直到定位到具体语法出错的位置。” 的方法解决的。
逍遥翔傲 2013-10-30
  • 打赏
  • 举报
回复
我也遇到这个错误了,可是不明觉厉,很是痛苦,感谢分享,试试看行不行!
xiaomiehhz 2012-12-14
  • 打赏
  • 举报
回复
引用 4 楼 babilife 的回复:
贴代码~~
//编写一个函数chg,在字符串中,将其中字母大小写翻转,字符串长度不超过50。 #include<stdio.h> int main() { char str[50]; char *q; char chg(*q); gets(str); chg(str[50]); puts(str); return 0; } char chg(char str[]) { char *p=str; while(*p!='\0') { if((*p)>='a' && (*p)<='z') (*p)-=32; else if((*p)>='A' && (*p)<='Z') (*p)+=32; p++; } return 0; }
laomashitu 2011-08-24
  • 打赏
  • 举报
回复
一个没有定义为函数的表达式被当做函数来调用啦。
例如::
int i, j;
char* p;
void func()
{
j = i(); // error, i is not a function
p(); // error, p doesn't point to a function
}
这个例子里i只不过是个整数,却被当做函数来用啦,因而导致了错误!
laomashitu 2011-08-24
  • 打赏
  • 举报
回复
Compiler Error C2064
term does not evaluate to a function

A call was made to a function through an expression that did not evaluate to a function pointer.

This error is probably caused by attempting to call a nonfunction.

The following is an example of this error:

int i, j;
char* p;
void func()
{
j = i(); // error, i is not a function
p(); // error, p doesn't point to a function
}

至善者善之敌 2011-08-24
  • 打赏
  • 举报
回复
贴代码~~
ryfdizuo 2011-08-24
  • 打赏
  • 举报
回复
根据错误提示,贴出那一行的代码。
ningto.com 2011-08-24
  • 打赏
  • 举报
回复
error C2064: term does not evaluate to a function
解决 error C2064 这种错误,关键是找到它所指向的错误行里面所有调用的函数名,他们中一定至少有一个是未被定义过的。
error C2064: term does not evaluate to a function!
今天这个错误让我痛苦了30分钟!

double Legendre[30];
Vect VV;

serial_legendre(Legendre,cos(VV.th()));

解释一下,serial_legendre 是我定义的一个函数,其函数原型为
void serial_legendre(double *, double);

Vect是我定义的一个类,其中定义了成员变量th ,
看起来很简单吧?可是每次我compile的时候,就会出现错误提示:error C2064: term does not evaluate to a function! 而且这个错误指向serial_legendre(Legendre,cos(VV.th()));

哈,编译器说这一行的函数调用有问题,打眼一看,这一行似乎只调用了serial_legendre这个很明显的函数,于是我开始花大量精力检查serial_legendre是不是有问题,结果查来查去都没有发现任何问题,郁闷死了!甚至开始怀疑是不是编译器出问题了!天啊!还是基本功不扎实,我怎么就没有注意到VV.th()呢!,th是VV 的成员变量,而不是它的成员函数,所以我在th后面加了(),当然就相当于调用类的成员函数了!这就直接导致了C2064这个错误!

这个教训太深刻了,写在这里,万一哪位难友遇到了同样的麻烦,看到我的经历也许会有所启发。记住:千万别把成员变量当作成员函数一样引用。

总结:解决 error C2064 这种错误,关键是找到它所指向的错误行里面所有调用的函数名,他们中一定至少有一个是未被定义过的。

http://apps.hi.baidu.com/share/detail/15490743
luciferisnotsatan 2011-08-24
  • 打赏
  • 举报
回复
代码贴出来
Table of Contents Header Files The #define Guard Header File Dependencies Inline Functions The -inl.h Files Function Parameter Ordering Names and Order of Includes Scoping Namespaces Nested Classes Nonmember, Static Member, and Global Functions Local Variables Static and Global Variables Classes Doing Work in Constructors Default Constructors Explicit Constructors Copy Constructors Structs vs. Classes Inheritance Multiple Inheritance Interfaces Operator Overloading Access Control Declaration Order Write Short Functions Google-Specific Magic Smart Pointers cpplint Other C++ Features Reference Arguments Function Overloading Default Arguments Variable-Length Arrays and alloca() Friends Exceptions Run-Time Type Information (RTTI) Casting Streams Preincrement and Predecrement Use of const Integer Types 64-bit Portability Preprocessor Macros 0 and NULL sizeof Boost C++0x Naming General Naming Rules File Names Type Names Variable Names Constant Names Function Names Namespace Names Enumerator Names Macro Names Exceptions to Naming Rules Comments Comment Style File Comments Class Comments Function Comments Variable Comments Implementation Comments Punctuation, Spelling and Grammar TODO Comments Deprecation Comments Formatting Line Length Non-ASCII Characters Spaces vs. Tabs Function Declarations and Definitions Function Calls Conditionals Loops and Switch Statements Pointer and Reference Expressions Boolean Expressions Return Values Variable and Array Initialization Preprocessor Directives Class Format Constructor Initializer Lists Namespace Formatting Horizontal Whitespace Vertical Whitespace Exceptions to the Rules Existing Non-conformant Code Windows Code Important Note Displaying Hidden Details in this Guide link ▶This style guide contains many details that are initially hidden from view. They are marked by the triangle icon, which you see here on your left. Click it now. You should see "Hooray" appear below. Hooray! Now you know you can expand points to get more details. Alternatively, there's an "expand all" at the top of this document. Background C++ is the main development language used by many of Google's open-source projects. As every C++ programmer knows, the language has many powerful features, but this power brings with it complexity, which in turn can make code more bug-prone and harder to read and maintain. The goal of this guide is to manage this complexity by describing in detail the dos and don'ts of writing C++ code. These rules exist to keep the code base manageable while still allowing coders to use C++ language features productively. Style, also known as readability, is what we call the conventions that govern our C++ code. The term Style is a bit of a misnomer, since these conventions cover far more than just source file formatting. One way in which we keep the code base manageable is by enforcing consistency. It is very important that any programmer be able to look at another's code and quickly understand it. Maintaining a uniform style and following conventions means that we can more easily use "pattern-matching" to infer what various symbols are and what invariants are true about them. Creating common, required idioms and patterns makes code much easier to understand. In some cases there might be good arguments for changing certain style rules, but we nonetheless keep things as they are in order to preserve consistency. Another issue this guide addresses is that of C++ feature bloat. C++ is a huge language with many advanced features. In some cases we constrain, or even ban, use of certain features. We do this to keep code simple and to avoid the various common errors and problems that these features can cause. This guide lists these features and explains why their use is restricted. Open-source projects developed by Google conform to the requirements in this guide. Note that this guide is not a C++ tutorial: we assume that the reader is familiar with the language. Header Files In general, every .cc file should have an associated .h file. There are some common exceptions, such as unittests and small .cc files containing just a main() function. Correct use of header files can make a huge difference to the readability, size and performance of your code. The following rules will guide you through the various pitfalls of using header files. The #define Guard link ▶All header files should have #define guards to prevent multiple inclusion. The format of the symbol name should be ___H_. To guarantee uniqueness, they should be based on the full path in a project's source tree. For example, the file foo/src/bar/baz.h in project foo should have the following guard: #ifndef FOO_BAR_BAZ_H_ #define FOO_BAR_BAZ_H_ ... #endif // FOO_BAR_BAZ_H_ Header File Dependencies link ▶Don't use an #include when a forward declaration would suffice. When you include a header file you introduce a dependency that will cause your code to be recompiled whenever the header file changes. If your header file includes other header files, any change to those files will cause any code that includes your header to be recompiled. Therefore, we prefer to minimize includes, particularly includes of header files in other header files. You can significantly minimize the number of header files you need to include in your own header files by using forward declarations. For example, if your header file uses the File class in ways that do not require access to the declaration of the File class, your header file can just forward declare class File; instead of having to #include "file/base/file.h". How can we use a class Foo in a header file without access to its definition? We can declare data members of type Foo* or Foo&. We can declare (but not define) functions with arguments, and/or return values, of type Foo. (One exception is if an argument Foo or const Foo& has a non-explicit, one-argument constructor, in which case we need the full definition to support automatic type conversion.) We can declare static data members of type Foo. This is because static data members are defined outside the class definition. On the other hand, you must include the header file for Foo if your class subclasses Foo or has a data member of type Foo. Sometimes it makes sense to have pointer (or better, scoped_ptr) members instead of object members. However, this complicates code readability and imposes a performance penalty, so avoid doing this transformation if the only purpose is to minimize includes in header files. Of course, .cc files typically do require the definitions of the classes they use, and usually have to include several header files. Note: If you use a symbol Foo in your source file, you should bring in a definition for Foo yourself, either via an #include or via a forward declaration. Do not depend on the symbol being brought in transitively via headers not directly included. One exception is if Foo is used in myfile.cc, it's ok to #include (or forward-declare) Foo in myfile.h, instead of myfile.cc. Inline Functions link ▶Define functions inline only when they are small, say, 10 lines or less. Definition: You can declare functions in a way that allows the compiler to expand them inline rather than calling them through the usual function call mechanism. Pros: Inlining a function can generate more efficient object code, as long as the inlined function is small. Feel free to inline accessors and mutators, and other short, performance-critical functions. Cons: Overuse of inlining can actually make programs slower. Depending on a function's size, inlining it can cause the code size to increase or decrease. Inlining a very small accessor function will usually decrease code size while inlining a very large function can dramatically increase code size. On modern processors smaller code usually runs faster due to better use of the instruction cache. Decision: A decent rule of thumb is to not inline a function if it is more than 10 lines long. Beware of destructors, which are often longer than they appear because of implicit member- and base-destructor calls! Another useful rule of thumb: it's typically not cost effective to inline functions with loops or switch statements (unless, in the common case, the loop or switch statement is never executed). It is important to know that functions are not always inlined even if they are declared as such; for example, virtual and recursive functions are not normally inlined. Usually recursive functions should not be inline. The main reason for making a virtual function inline is to place its definition in the class, either for convenience or to document its behavior, e.g., for accessors and mutators. The -inl.h Files link ▶You may use file names with a -inl.h suffix to define complex inline functions when needed. The definition of an inline function needs to be in a header file, so that the compiler has the definition available for inlining at the call sites. However, implementation code properly belongs in .cc files, and we do not like to have much actual code in .h files unless there is a readability or performance advantage. If an inline function definition is short, with very little, if any, logic in it, you should put the code in your .h file. For example, accessors and mutators should certainly be inside a class definition. More complex inline functions may also be put in a .h file for the convenience of the implementer and callers, though if this makes the .h file too unwieldy you can instead put that code in a separate -inl.h file. This separates the implementation from the class definition, while still allowing the implementation to be included where necessary. Another use of -inl.h files is for definitions of function templates. This can be used to keep your template definitions easy to read. Do not forget that a -inl.h file requires a #define guard just like any other header file. Function Parameter Ordering link ▶When defining a function, parameter order is: inputs, then outputs. Parameters to C/C++ functions are either input to the function, output from the function, or both. Input parameters are usually values or const references, while output and input/output parameters will be non-const pointers. When ordering function parameters, put all input-only parameters before any output parameters. In particular, do not add new parameters to the end of the function just because they are new; place new input-only parameters before the output parameters. This is not a hard-and-fast rule. Parameters that are both input and output (often classes/structs) muddy the waters, and, as always, consistency with related functions may require you to bend the rule. Names and Order of Includes link ▶Use standard order for readability and to avoid hidden dependencies: C library, C++ library, other libraries' .h, your project's .h. All of a project's header files should be listed as descentants of the project's source directory without use of UNIX directory shortcuts . (the current directory) or .. (the parent directory). For example, google-awesome-project/src/base/logging.h should be included as #include "base/logging.h" In dir/foo.cc, whose main purpose is to implement or test the stuff in dir2/foo2.h, order your includes as follows: dir2/foo2.h (preferred location — see details below). C system files. C++ system files. Other libraries' .h files. Your project's .h files. The preferred ordering reduces hidden dependencies. We want every header file to be compilable on its own. The easiest way to achieve this is to make sure that every one of them is the first .h file #included in some .cc. dir/foo.cc and dir2/foo2.h are often in the same directory (e.g. base/basictypes_test.cc and base/basictypes.h), but can be in different directories too. Within each section it is nice to order the includes alphabetically. For example, the includes in google-awesome-project/src/foo/internal/fooserver.cc might look like this: #include "foo/public/fooserver.h" // Preferred location. #include #include #include #include #include "base/basictypes.h" #include "base/commandlineflags.h" #include "foo/public/bar.h" Scoping Namespaces link ▶Unnamed namespaces in .cc files are encouraged. With named namespaces, choose the name based on the project, and possibly its path. Do not use a using-directive. Definition: Namespaces subdivide the global scope into distinct, named scopes, and so are useful for preventing name collisions in the global scope. Pros: Namespaces provide a (hierarchical) axis of naming, in addition to the (also hierarchical) name axis provided by classes. For example, if two different projects have a class Foo in the global scope, these symbols may collide at compile time or at runtime. If each project places their code in a namespace, project1::Foo and project2::Foo are now distinct symbols that do not collide. Cons: Namespaces can be confusing, because they provide an additional (hierarchical) axis of naming, in addition to the (also hierarchical) name axis provided by classes. Use of unnamed spaces in header files can easily cause violations of the C++ One Definition Rule (ODR). Decision: Use namespaces according to the policy described below. Unnamed Namespaces Unnamed namespaces are allowed and even encouraged in .cc files, to avoid runtime naming conflicts: namespace { // This is in a .cc file. // The content of a namespace is not indented enum { kUnused, kEOF, kError }; // Commonly used tokens. bool AtEof() { return pos_ == kEOF; } // Uses our namespace's EOF. } // namespace However, file-scope declarations that are associated with a particular class may be declared in that class as types, static data members or static member functions rather than as members of an unnamed namespace. Terminate the unnamed namespace as shown, with a comment // namespace. Do not use unnamed namespaces in .h files. Named Namespaces Named namespaces should be used as follows: Namespaces wrap the entire source file after includes, gflags definitions/declarations, and forward declarations of classes from other namespaces: // In the .h file namespace mynamespace { // All declarations are within the namespace scope. // Notice the lack of indentation. class MyClass { public: ... void Foo(); }; } // namespace mynamespace // In the .cc file namespace mynamespace { // Definition of functions is within scope of the namespace. void MyClass::Foo() { ... } } // namespace mynamespace The typical .cc file might have more complex detail, including the need to reference classes in other namespaces. #include "a.h" DEFINE_bool(someflag, false, "dummy flag"); class C; // Forward declaration of class C in the global namespace. namespace a { class A; } // Forward declaration of a::A. namespace b { ...code for b... // Code goes against the left margin. } // namespace b Do not declare anything in namespace std, not even forward declarations of standard library classes. Declaring entities in namespace std is undefined behavior, i.e., not portable. To declare entities from the standard library, include the appropriate header file. You may not use a using-directive to make all names from a namespace available. // Forbidden -- This pollutes the namespace. using namespace foo; You may use a using-declaration anywhere in a .cc file, and in functions, methods or classes in .h files. // OK in .cc files. // Must be in a function, method or class in .h files. using ::foo::bar; Namespace aliases are allowed anywhere in a .cc file, anywhere inside the named namespace that wraps an entire .h file, and in functions and methods. // Shorten access to some commonly used names in .cc files. namespace fbz = ::foo::bar::baz; // Shorten access to some commonly used names (in a .h file). namespace librarian { // The following alias is available to all files including // this header (in namespace librarian): // alias names should therefore be chosen consistently // within a project. namespace pd_s = ::pipeline_diagnostics::sidetable; inline void my_inline_function() { // namespace alias local to a function (or method). namespace fbz = ::foo::bar::baz; ... } } // namespace librarian Note that an alias in a .h file is visible to everyone #including that file, so public headers (those available outside a project) and headers transitively #included by them, should avoid defining aliases, as part of the general goal of keeping public APIs as small as possible. Nested Classes link ▶Although you may use public nested classes when they are part of an interface, consider a namespace to keep declarations out of the global scope. Definition: A class can define another class within it; this is also called a member class. class Foo { private: // Bar is a member class, nested within Foo. class Bar { ... }; }; Pros: This is useful when the nested (or member) class is only used by the enclosing class; making it a member puts it in the enclosing class scope rather than polluting the outer scope with the class name. Nested classes can be forward declared within the enclosing class and then defined in the .cc file to avoid including the nested class definition in the enclosing class declaration, since the nested class definition is usually only relevant to the implementation. Cons: Nested classes can be forward-declared only within the definition of the enclosing class. Thus, any header file manipulating a Foo::Bar* pointer will have to include the full class declaration for Foo. Decision: Do not make nested classes public unless they are actually part of the interface, e.g., a class that holds a set of options for some method. Nonmember, Static Member, and Global Functions link ▶Prefer nonmember functions within a namespace or static member functions to global functions; use completely global functions rarely. Pros: Nonmember and static member functions can be useful in some situations. Putting nonmember functions in a namespace avoids polluting the global namespace. Cons: Nonmember and static member functions may make more sense as members of a new class, especially if they access external resources or have significant dependencies. Decision: Sometimes it is useful, or even necessary, to define a function not bound to a class instance. Such a function can be either a static member or a nonmember function. Nonmember functions should not depend on external variables, and should nearly always exist in a namespace. Rather than creating classes only to group static member functions which do not share static data, use namespaces instead. Functions defined in the same compilation unit as production classes may introduce unnecessary coupling and link-time dependencies when directly called from other compilation units; static member functions are particularly susceptible to this. Consider extracting a new class, or placing the functions in a namespace possibly in a separate library. If you must define a nonmember function and it is only needed in its .cc file, use an unnamed namespace or static linkage (eg static int Foo() {...}) to limit its scope. Local Variables link ▶Place a function's variables in the narrowest scope possible, and initialize variables in the declaration. C++ allows you to declare variables anywhere in a function. We encourage you to declare them in as local a scope as possible, and as close to the first use as possible. This makes it easier for the reader to find the declaration and see what type the variable is and what it was initialized to. In particular, initialization should be used instead of declaration and assignment, e.g. int i; i = f(); // Bad -- initialization separate from declaration. int j = g(); // Good -- declaration has initialization. Note that gcc implements for (int i = 0; i < 10; ++i) correctly (the scope of i is only the scope of the for loop), so you can then reuse i in another for loop in the same scope. It also correctly scopes declarations in if and while statements, e.g. while (const char* p = strchr(str, '/')) str = p + 1; There is one caveat: if the variable is an object, its constructor is invoked every time it enters scope and is created, and its destructor is invoked every time it goes out of scope. // Inefficient implementation: for (int i = 0; i < 1000000; ++i) { Foo f; // My ctor and dtor get called 1000000 times each. f.DoSomething(i); } It may be more efficient to declare such a variable used in a loop outside that loop: Foo f; // My ctor and dtor get called once each. for (int i = 0; i < 1000000; ++i) { f.DoSomething(i); } Static and Global Variables link ▶Static or global variables of class type are forbidden: they cause hard-to-find bugs due to indeterminate order of construction and destruction. Objects with static storage duration, including global variables, static variables, static class member variables, and function static variables, must be Plain Old Data (POD): only ints, chars, floats, or pointers, or arrays/structs of POD. The order in which class constructors and initializers for static variables are called is only partially specified in C++ and can even change from build to build, which can cause bugs that are difficult to find. Therefore in addition to banning globals of class type, we do not allow static POD variables to be initialized with the result of a function, unless that function (such as getenv(), or getpid()) does not itself depend on any other globals. Likewise, the order in which destructors are called is defined to be the reverse of the order in which the constructors were called. Since constructor order is indeterminate, so is destructor order. For example, at program-end time a static variable might have been destroyed, but code still running -- perhaps in another thread -- tries to access it and fails. Or the destructor for a static 'string' variable might be run prior to the destructor for another variable that contains a reference to that string. As a result we only allow static variables to contain POD data. This rule completely disallows vector (use C arrays instead), or string (use const char []). If you need a static or global variable of a class type, consider initializing a pointer (which will never be freed), from either your main() function or from pthread_once(). Note that this must be a raw pointer, not a "smart" pointer, since the smart pointer's destructor will have the order-of-destructor issue that we are trying to avoid. Classes Classes are the fundamental unit of code in C++. Naturally, we use them extensively. This section lists the main dos and don'ts you should follow when writing a class. Doing Work in Constructors link ▶In general, constructors should merely set member variables to their initial values. Any complex initialization should go in an explicit Init() method. Definition: It is possible to perform initialization in the body of the constructor. Pros: Convenience in typing. No need to worry about whether the class has been initialized or not. Cons: The problems with doing work in constructors are: There is no easy way for constructors to signal errors, short of using exceptions (which are forbidden). If the work fails, we now have an object whose initialization code failed, so it may be an indeterminate state. If the work calls virtual functions, these calls will not get dispatched to the subclass implementations. Future modification to your class can quietly introduce this problem even if your class is not currently subclassed, causing much confusion. If someone creates a global variable of this type (which is against the rules, but still), the constructor code will be called before main(), possibly breaking some implicit assumptions in the constructor code. For instance, gflags will not yet have been initialized. Decision: If your object requires non-trivial initialization, consider having an explicit Init() method. In particular, constructors should not call virtual functions, attempt to raise errors, access potentially uninitialized global variables, etc. Default Constructors link ▶You must define a default constructor if your class defines member variables and has no other constructors. Otherwise the compiler will do it for you, badly. Definition: The default constructor is called when we new a class object with no arguments. It is always called when calling new[] (for arrays). Pros: Initializing structures by default, to hold "impossible" values, makes debugging much easier. Cons: Extra work for you, the code writer. Decision: If your class defines member variables and has no other constructors you must define a default constructor (one that takes no arguments). It should preferably initialize the object in such a way that its internal state is consistent and valid. The reason for this is that if you have no other constructors and do not define a default constructor, the compiler will generate one for you. This compiler generated constructor may not initialize your object sensibly. If your class inherits from an existing class but you add no new member variables, you are not required to have a default constructor. Explicit Constructors link ▶Use the C++ keyword explicit for constructors with one argument. Definition: Normally, if a constructor takes one argument, it can be used as a conversion. For instance, if you define Foo::Foo(string name) and then pass a string to a function that expects a Foo, the constructor will be called to convert the string into a Foo and will pass the Foo to your function for you. This can be convenient but is also a source of trouble when things get converted and new objects created without you meaning them to. Declaring a constructor explicit prevents it from being invoked implicitly as a conversion. Pros: Avoids undesirable conversions. Cons: None. Decision: We require all single argument constructors to be explicit. Always put explicit in front of one-argument constructors in the class definition: explicit Foo(string name); The exception is copy constructors, which, in the rare cases when we allow them, should probably not be explicit. Classes that are intended to be transparent wrappers around other classes are also exceptions. Such exceptions should be clearly marked with comments. Copy Constructors link ▶Provide a copy constructor and assignment operator only when necessary. Otherwise, disable them with DISALLOW_COPY_AND_ASSIGN. Definition: The copy constructor and assignment operator are used to create copies of objects. The copy constructor is implicitly invoked by the compiler in some situations, e.g. passing objects by value. Pros: Copy constructors make it easy to copy objects. STL containers require that all contents be copyable and assignable. Copy constructors can be more efficient than CopyFrom()-style workarounds because they combine construction with copying, the compiler can elide them in some contexts, and they make it easier to avoid heap allocation. Cons: Implicit copying of objects in C++ is a rich source of bugs and of performance problems. It also reduces readability, as it becomes hard to track which objects are being passed around by value as opposed to by reference, and therefore where changes to an object are reflected. Decision: Few classes need to be copyable. Most should have neither a copy constructor nor an assignment operator. In many situations, a pointer or reference will work just as well as a copied value, with better performance. For example, you can pass function parameters by reference or pointer instead of by value, and you can store pointers rather than objects in an STL container. If your class needs to be copyable, prefer providing a copy method, such as CopyFrom() or Clone(), rather than a copy constructor, because such methods cannot be invoked implicitly. If a copy method is insufficient in your situation (e.g. for performance reasons, or because your class needs to be stored by value in an STL container), provide both a copy constructor and assignment operator. If your class does not need a copy constructor or assignment operator, you must explicitly disable them. To do so, add dummy declarations for the copy constructor and assignment operator in the private: section of your class, but do not provide any corresponding definition (so that any attempt to use them results in a link error). For convenience, a DISALLOW_COPY_AND_ASSIGN macro can be used: // A macro to disallow the copy constructor and operator= functions // This should be used in the private: declarations for a class #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ TypeName(const TypeName&); \ void operator=(const TypeName&) Then, in class Foo: class Foo { public: Foo(int f); ~Foo(); private: DISALLOW_COPY_AND_ASSIGN(Foo); }; Structs vs. Classes link ▶Use a struct only for passive objects that carry data; everything else is a class. The struct and class keywords behave almost identically in C++. We add our own semantic meanings to each keyword, so you should use the appropriate keyword for the data-type you're defining. structs should be used for passive objects that carry data, and may have associated constants, but lack any functionality other than access/setting the data members. The accessing/setting of fields is done by directly accessing the fields rather than through method invocations. Methods should not provide behavior but should only be used to set up the data members, e.g., constructor, destructor, Initialize(), Reset(), Validate(). If more functionality is required, a class is more appropriate. If in doubt, make it a class. For consistency with STL, you can use struct instead of class for functors and traits. Note that member variables in structs and classes have different naming rules. Inheritance link ▶Composition is often more appropriate than inheritance. When using inheritance, make it public. Definition: When a sub-class inherits from a base class, it includes the definitions of all the data and operations that the parent base class defines. In practice, inheritance is used in two major ways in C++: implementation inheritance, in which actual code is inherited by the child, and interface inheritance, in which only method names are inherited. Pros: Implementation inheritance reduces code size by re-using the base class code as it specializes an existing type. Because inheritance is a compile-time declaration, you and the compiler can understand the operation and detect errors. Interface inheritance can be used to programmatically enforce that a class expose a particular API. Again, the compiler can detect errors, in this case, when a class does not define a necessary method of the API. Cons: For implementation inheritance, because the code implementing a sub-class is spread between the base and the sub-class, it can be more difficult to understand an implementation. The sub-class cannot override functions that are not virtual, so the sub-class cannot change implementation. The base class may also define some data members, so that specifies physical layout of the base class. Decision: All inheritance should be public. If you want to do private inheritance, you should be including an instance of the base class as a member instead. Do not overuse implementation inheritance. Composition is often more appropriate. Try to restrict use of inheritance to the "is-a" case: Bar subclasses Foo if it can reasonably be said that Bar "is a kind of" Foo. Make your destructor virtual if necessary. If your class has virtual methods, its destructor should be virtual. Limit the use of protected to those member functions that might need to be accessed from subclasses. Note that data members should be private. When redefining an inherited virtual function, explicitly declare it virtual in the declaration of the derived class. Rationale: If virtual is omitted, the reader has to check all ancestors of the class in question to determine if the function is virtual or not. Multiple Inheritance link ▶Only very rarely is multiple implementation inheritance actually useful. We allow multiple inheritance only when at most one of the base classes has an implementation; all other base classes must be pure interface classes tagged with the Interface suffix. Definition: Multiple inheritance allows a sub-class to have more than one base class. We distinguish between base classes that are pure interfaces and those that have an implementation. Pros: Multiple implementation inheritance may let you re-use even more code than single inheritance (see Inheritance). Cons: Only very rarely is multiple implementation inheritance actually useful. When multiple implementation inheritance seems like the solution, you can usually find a different, more explicit, and cleaner solution. Decision: Multiple inheritance is allowed only when all superclasses, with the possible exception of the first one, are pure interfaces. In order to ensure that they remain pure interfaces, they must end with the Interface suffix. Note: There is an exception to this rule on Windows. Interfaces link ▶Classes that satisfy certain conditions are allowed, but not required, to end with an Interface suffix. Definition: A class is a pure interface if it meets the following requirements: It has only public pure virtual ("= 0") methods and static methods (but see below for destructor). It may not have non-static data members. It need not have any constructors defined. If a constructor is provided, it must take no arguments and it must be protected. If it is a subclass, it may only be derived from classes that satisfy these conditions and are tagged with the Interface suffix. An interface class can never be directly instantiated because of the pure virtual method(s) it declares. To make sure all implementations of the interface can be destroyed correctly, they must also declare a virtual destructor (in an exception to the first rule, this should not be pure). See Stroustrup, The C++ Programming Language, 3rd edition, section 12.4 for details. Pros: Tagging a class with the Interface suffix lets others know that they must not add implemented methods or non static data members. This is particularly important in the case of multiple inheritance. Additionally, the interface concept is already well-understood by Java programmers. Cons: The Interface suffix lengthens the class name, which can make it harder to read and understand. Also, the interface property may be considered an implementation detail that shouldn't be exposed to clients. Decision: A class may end with Interface only if it meets the above requirements. We do not require the converse, however: classes that meet the above requirements are not required to end with Interface. Operator Overloading link ▶Do not overload operators except in rare, special circumstances. Definition: A class can define that operators such as + and / operate on the class as if it were a built-in type. Pros: Can make code appear more intuitive because a class will behave in the same way as built-in types (such as int). Overloaded operators are more playful names for functions that are less-colorfully named, such as Equals() or Add(). For some template functions to work correctly, you may need to define operators. Cons: While operator overloading can make code more intuitive, it has several drawbacks: It can fool our intuition into thinking that expensive operations are cheap, built-in operations. It is much harder to find the call sites for overloaded operators. Searching for Equals() is much easier than searching for relevant invocations of ==. Some operators work on pointers too, making it easy to introduce bugs. Foo + 4 may do one thing, while &Foo + 4 does something totally different. The compiler does not complain for either of these, making this very hard to debug. Overloading also has surprising ramifications. For instance, if a class overloads unary operator&, it cannot safely be forward-declared. Decision: In general, do not overload operators. The assignment operator (operator=), in particular, is insidious and should be avoided. You can define functions like Equals() and CopyFrom() if you need them. Likewise, avoid the dangerous unary operator& at all costs, if there's any possibility the class might be forward-declared. However, there may be rare cases where you need to overload an operator to interoperate with templates or "standard" C++ classes (such as operator<<(ostream&, const T&) for logging). These are acceptable if fully justified, but you should try to avoid these whenever possible. In particular, do not overload operator== or operator< just so that your class can be used as a key in an STL container; instead, you should create equality and comparison functor types when declaring the container. Some of the STL algorithms do require you to overload operator==, and you may do so in these cases, provided you document why. See also Copy Constructors and Function Overloading. Access Control link ▶Make data members private, and provide access to them through accessor functions as needed (for technical reasons, we allow data members of a test fixture class to be protected when using Google Test). Typically a variable would be called foo_ and the accessor function foo(). You may also want a mutator function set_foo(). Exception: static const data members (typically called kFoo) need not be private. The definitions of accessors are usually inlined in the header file. See also Inheritance and Function Names. Declaration Order link ▶Use the specified order of declarations within a class: public: before private:, methods before data members (variables), etc. Your class definition should start with its public: section, followed by its protected: section and then its private: section. If any of these sections are empty, omit them. Within each section, the declarations generally should be in the following order: Typedefs and Enums Constants (static const data members) Constructors Destructor Methods, including static methods Data Members (except static const data members) Friend declarations should always be in the private section, and the DISALLOW_COPY_AND_ASSIGN macro invocation should be at the end of the private: section. It should be the last thing in the class. See Copy Constructors. Method definitions in the corresponding .cc file should be the same as the declaration order, as much as possible. Do not put large method definitions inline in the class definition. Usually, only trivial or performance-critical, and very short, methods may be defined inline. See Inline Functions for more details. Write Short Functions link ▶Prefer small and focused functions. We recognize that long functions are sometimes appropriate, so no hard limit is placed on functions length. If a function exceeds about 40 lines, think about whether it can be broken up without harming the structure of the program. Even if your long function works perfectly now, someone modifying it in a few months may add new behavior. This could result in bugs that are hard to find. Keeping your functions short and simple makes it easier for other people to read and modify your code. You could find long and complicated functions when working with some code. Do not be intimidated by modifying existing code: if working with such a function proves to be difficult, you find that errors are hard to debug, or you want to use a piece of it in several different contexts, consider breaking up the function into smaller and more manageable pieces. Google-Specific Magic There are various tricks and utilities that we use to make C++ code more robust, and various ways we use C++ that may differ from what you see elsewhere. Smart Pointers link ▶If you actually need pointer semantics, scoped_ptr is great. You should only use std::tr1::shared_ptr under very specific conditions, such as when objects need to be held by STL containers. You should never use auto_ptr. "Smart" pointers are objects that act like pointers but have added semantics. When a scoped_ptr is destroyed, for instance, it deletes the object it's pointing to. shared_ptr is the same way, but implements reference-counting so only the last pointer to an object deletes it. Generally speaking, we prefer that we design code with clear object ownership. The clearest object ownership is obtained by using an object directly as a field or local variable, without using pointers at all. On the other extreme, by their very definition, reference counted pointers are owned by nobody. The problem with this design is that it is easy to create circular references or other strange conditions that cause an object to never be deleted. It is also slow to perform atomic operations every time a value is copied or assigned. Although they are not recommended, reference counted pointers are sometimes the simplest and most elegant way to solve a problem. cpplint link ▶Use cpplint.py to detect style errors. cpplint.py is a tool that reads a source file and identifies many style errors. It is not perfect, and has both false positives and false negatives, but it is still a valuable tool. False positives can be ignored by putting // NOLINT at the end of the line. Some projects have instructions on how to run cpplint.py from their project tools. If the project you are contributing to does not, you can download cpplint.py separately. Other C++ Features Reference Arguments link ▶All parameters passed by reference must be labeled const. Definition: In C, if a function needs to modify a variable, the parameter must use a pointer, eg int foo(int *pval). In C++, the function can alternatively declare a reference parameter: int foo(int &val). Pros: Defining a parameter as reference avoids ugly code like (*pval)++. Necessary for some applications like copy constructors. Makes it clear, unlike with pointers, that NULL is not a possible value. Cons: References can be confusing, as they have value syntax but pointer semantics. Decision: Within function parameter lists all references must be const: void Foo(const string &in, string *out); In fact it is a very strong convention in Google code that input arguments are values or const references while output arguments are pointers. Input parameters may be const pointers, but we never allow non-const reference parameters. One case when you might want an input parameter to be a const pointer is if you want to emphasize that the argument is not copied, so it must exist for the lifetime of the object; it is usually best to document this in comments as well. STL adapters such as bind2nd and mem_fun do not permit reference parameters, so you must declare functions with pointer parameters in these cases, too. Function Overloading link ▶Use overloaded functions (including constructors) only if a reader looking at a call site can get a good idea of what is happening without having to first figure out exactly which overload is being called. Definition: You may write a function that takes a const string& and overload it with another that takes const char*. class MyClass { public: void Analyze(const string &text); void Analyze(const char *text, size_t textlen); }; Pros: Overloading can make code more intuitive by allowing an identically-named function to take different arguments. It may be necessary for templatized code, and it can be convenient for Visitors. Cons: If a function is overloaded by the argument types alone, a reader may have to understand C++'s complex matching rules in order to tell what's going on. Also many people are confused by the semantics of inheritance if a derived class overrides only some of the variants of a function. Decision: If you want to overload a function, consider qualifying the name with some information about the arguments, e.g., AppendString(), AppendInt() rather than just Append(). Default Arguments link ▶We do not allow default function parameters, except in a few uncommon situations explained below. Pros: Often you have a function that uses lots of default values, but occasionally you want to override the defaults. Default parameters allow an easy way to do this without having to define many functions for the rare exceptions. Cons: People often figure out how to use an API by looking at existing code that uses it. Default parameters are more difficult to maintain because copy-and-paste from previous code may not reveal all the parameters. Copy-and-pasting of code segments can cause major problems when the default arguments are not appropriate for the new code. Decision: Except as described below, we require all arguments to be explicitly specified, to force programmers to consider the API and the values they are passing for each argument rather than silently accepting defaults they may not be aware of. One specific exception is when default arguments are used to simulate variable-length argument lists. // Support up to 4 params by using a default empty AlphaNum. string StrCat(const AlphaNum &a, const AlphaNum &b = gEmptyAlphaNum, const AlphaNum &c = gEmptyAlphaNum, const AlphaNum &d = gEmptyAlphaNum); Variable-Length Arrays and alloca() link ▶We do not allow variable-length arrays or alloca(). Pros: Variable-length arrays have natural-looking syntax. Both variable-length arrays and alloca() are very efficient. Cons: Variable-length arrays and alloca are not part of Standard C++. More importantly, they allocate a data-dependent amount of stack space that can trigger difficult-to-find memory overwriting bugs: "It ran fine on my machine, but dies mysteriously in production". Decision: Use a safe allocator instead, such as scoped_ptr/scoped_array. Friends link ▶We allow use of friend classes and functions, within reason. Friends should usually be defined in the same file so that the reader does not have to look in another file to find uses of the private members of a class. A common use of friend is to have a FooBuilder class be a friend of Foo so that it can construct the inner state of Foo correctly, without exposing this state to the world. In some cases it may be useful to make a unittest class a friend of the class it tests. Friends extend, but do not break, the encapsulation boundary of a class. In some cases this is better than making a member public when you want to give only one other class access to it. However, most classes should interact with other classes solely through their public members. Exceptions link ▶We do not use C++ exceptions. Pros: Exceptions allow higher levels of an application to decide how to handle "can't happen" failures in deeply nested functions, without the obscuring and error-prone bookkeeping of error codes. Exceptions are used by most other modern languages. Using them in C++ would make it more consistent with Python, Java, and the C++ that others are familiar with. Some third-party C++ libraries use exceptions, and turning them off internally makes it harder to integrate with those libraries. Exceptions are the only way for a constructor to fail. We can simulate this with a factory function or an Init() method, but these require heap allocation or a new "invalid" state, respectively. Exceptions are really handy in testing frameworks. Cons: When you add a throw statement to an existing function, you must examine all of its transitive callers. Either they must make at least the basic exception safety guarantee, or they must never catch the exception and be happy with the program terminating as a result. For instance, if f() calls g() calls h(), and h throws an exception that f catches, g has to be careful or it may not clean up properly. More generally, exceptions make the control flow of programs difficult to evaluate by looking at code: functions may return in places you don't expect. This causes maintainability and debugging difficulties. You can minimize this cost via some rules on how and where exceptions can be used, but at the cost of more that a developer needs to know and understand. Exception safety requires both RAII and different coding practices. Lots of supporting machinery is needed to make writing correct exception-safe code easy. Further, to avoid requiring readers to understand the entire call graph, exception-safe code must isolate logic that writes to persistent state into a "commit" phase. This will have both benefits and costs (perhaps where you're forced to obfuscate code to isolate the commit). Allowing exceptions would force us to always pay those costs even when they're not worth it. Turning on exceptions adds data to each binary produced, increasing compile time (probably slightly) and possibly increasing address space pressure. The availability of exceptions may encourage developers to throw them when they are not appropriate or recover from them when it's not safe to do so. For example, invalid user input should not cause exceptions to be thrown. We would need to make the style guide even longer to document these restrictions! Decision: On their face, the benefits of using exceptions outweigh the costs, especially in new projects. However, for existing code, the introduction of exceptions has implications on all dependent code. If exceptions can be propagated beyond a new project, it also becomes problematic to integrate the new project into existing exception-free code. Because most existing C++ code at Google is not prepared to deal with exceptions, it is comparatively difficult to adopt new code that generates exceptions. Given that Google's existing code is not exception-tolerant, the costs of using exceptions are somewhat greater than the costs in a new project. The conversion process would be slow and error-prone. We don't believe that the available alternatives to exceptions, such as error codes and assertions, introduce a significant burden. Our advice against using exceptions is not predicated on philosophical or moral grounds, but practical ones. Because we'd like to use our open-source projects at Google and it's difficult to do so if those projects use exceptions, we need to advise against exceptions in Google open-source projects as well. Things would probably be different if we had to do it all over again from scratch. There is an exception to this rule (no pun intended) for Windows code. Run-Time Type Information (RTTI) link ▶We do not use Run Time Type Information (RTTI). Definition: RTTI allows a programmer to query the C++ class of an object at run time. Pros: It is useful in some unittests. For example, it is useful in tests of factory classes where the test has to verify that a newly created object has the expected dynamic type. In rare circumstances, it is useful even outside of tests. Cons: A query of type during run-time typically means a design problem. If you need to know the type of an object at runtime, that is often an indication that you should reconsider the design of your class. Decision: Do not use RTTI, except in unittests. If you find yourself in need of writing code that behaves differently based on the class of an object, consider one of the alternatives to querying the type. Virtual methods are the preferred way of executing different code paths depending on a specific subclass type. This puts the work within the object itself. If the work belongs outside the object and instead in some processing code, consider a double-dispatch solution, such as the Visitor design pattern. This allows a facility outside the object itself to determine the type of class using the built-in type system. If you think you truly cannot use those ideas, you may use RTTI. But think twice about it. :-) Then think twice again. Do not hand-implement an RTTI-like workaround. The arguments against RTTI apply just as much to workarounds like class hierarchies with type tags. Casting link ▶Use C++ casts like static_cast(). Do not use other cast formats like int y = (int)x; or int y = int(x);. Definition: C++ introduced a different cast system from C that distinguishes the types of cast operations. Pros: The problem with C casts is the ambiguity of the operation; sometimes you are doing a conversion (e.g., (int)3.5) and sometimes you are doing a cast (e.g., (int)"hello"); C++ casts avoid this. Additionally C++ casts are more visible when searching for them. Cons: The syntax is nasty. Decision: Do not use C-style casts. Instead, use these C++-style casts. Use static_cast as the equivalent of a C-style cast that does value conversion, or when you need to explicitly up-cast a pointer from a class to its superclass. Use const_cast to remove the const qualifier (see const). Use reinterpret_cast to do unsafe conversions of pointer types to and from integer and other pointer types. Use this only if you know what you are doing and you understand the aliasing issues. Do not use dynamic_cast except in test code. If you need to know type information at runtime in this way outside of a unittest, you probably have a design flaw. Streams link ▶Use streams only for logging. Definition: Streams are a replacement for printf() and scanf(). Pros: With streams, you do not need to know the type of the object you are printing. You do not have problems with format strings not matching the argument list. (Though with gcc, you do not have that problem with printf either.) Streams have automatic constructors and destructors that open and close the relevant files. Cons: Streams make it difficult to do functionality like pread(). Some formatting (particularly the common format string idiom %.*s) is difficult if not impossible to do efficiently using streams without using printf-like hacks. Streams do not support operator reordering (the %1s directive), which is helpful for internationalization. Decision: Do not use streams, except where required by a logging interface. Use printf-like routines instead. There are various pros and cons to using streams, but in this case, as in many other cases, consistency trumps the debate. Do not use streams in your code. Extended Discussion There has been debate on this issue, so this explains the reasoning in greater depth. Recall the Only One Way guiding principle: we want to make sure that whenever we do a certain type of I/O, the code looks the same in all those places. Because of this, we do not want to allow users to decide between using streams or using printf plus Read/Write/etc. Instead, we should settle on one or the other. We made an exception for logging because it is a pretty specialized application, and for historical reasons. Proponents of streams have argued that streams are the obvious choice of the two, but the issue is not actually so clear. For every advantage of streams they point out, there is an equivalent disadvantage. The biggest advantage is that you do not need to know the type of the object to be printing. This is a fair point. But, there is a downside: you can easily use the wrong type, and the compiler will not warn you. It is easy to make this kind of mistake without knowing when using streams. cout << this; // Prints the address cout << *this; // Prints the contents The compiler does not generate an error because << has been overloaded. We discourage overloading for just this reason. Some say printf formatting is ugly and hard to read, but streams are often no better. Consider the following two fragments, both with the same typo. Which is easier to discover? cerr << "Error connecting to '" hostname.first << ":" hostname.second << ": " hostname.first, foo->bar()->hostname.second, strerror(errno)); And so on and so forth for any issue you might bring up. (You could argue, "Things would be better with the right wrappers," but if it is true for one scheme, is it not also true for the other? Also, remember the goal is to make the language smaller, not add yet more machinery that someone has to learn.) Either path would yield different advantages and disadvantages, and there is not a clearly superior solution. The simplicity doctrine mandates we settle on one of them though, and the majority decision was on printf + read/write. Preincrement and Predecrement link ▶Use prefix form (++i) of the increment and decrement operators with iterators and other template objects. Definition: When a variable is incremented (++i or i++) or decremented (--i or i--) and the value of the expression is not used, one must decide whether to preincrement (decrement) or postincrement (decrement). Pros: When the return value is ignored, the "pre" form (++i) is never less efficient than the "post" form (i++), and is often more efficient. This is because post-increment (or decrement) requires a copy of i to be made, which is the value of the expression. If i is an iterator or other non-scalar type, copying i could be expensive. Since the two types of increment behave the same when the value is ignored, why not just always pre-increment? Cons: The tradition developed, in C, of using post-increment when the expression value is not used, especially in for loops. Some find post-increment easier to read, since the "subject" (i) precedes the "verb" (++), just like in English. Decision: For simple scalar (non-object) values there is no reason to prefer one form and we allow either. For iterators and other template types, use pre-increment. Use of const link ▶We strongly recommend that you use const whenever it makes sense to do so. Definition: Declared variables and parameters can be preceded by the keyword const to indicate the variables are not changed (e.g., const int foo). Class functions can have the const qualifier to indicate the function does not change the state of the class member variables (e.g., class Foo { int Bar(char c) const; };). Pros: Easier for people to understand how variables are being used. Allows the compiler to do better type checking, and, conceivably, generate better code. Helps people convince themselves of program correctness because they know the functions they call are limited in how they can modify your variables. Helps people know what functions are safe to use without locks in multi-threaded programs. Cons: const is viral: if you pass a const variable to a function, that function must have const in its prototype (or the variable will need a const_cast). This can be a particular problem when calling library functions. Decision: const variables, data members, methods and arguments add a level of compile-time type checking; it is better to detect errors as soon as possible. Therefore we strongly recommend that you use const whenever it makes sense to do so: If a function does not modify an argument passed by reference or by pointer, that argument should be const. Declare methods to be const whenever possible. Accessors should almost always be const. Other methods should be const if they do not modify any data members, do not call any non-const methods, and do not return a non-const pointer or non-const reference to a data member. Consider making data members const whenever they do not need to be modified after construction. However, do not go crazy with const. Something like const int * const * const x; is likely overkill, even if it accurately describes how const x is. Focus on what's really useful to know: in this case, const int** x is probably sufficient. The mutable keyword is allowed but is unsafe when used with threads, so thread safety should be carefully considered first. Where to put the const Some people favor the form int const *foo to const int* foo. They argue that this is more readable because it's more consistent: it keeps the rule that const always follows the object it's describing. However, this consistency argument doesn't apply in this case, because the "don't go crazy" dictum eliminates most of the uses you'd have to be consistent with. Putting the const first is arguably more readable, since it follows English in putting the "adjective" (const) before the "noun" (int). That said, while we encourage putting const first, we do not require it. But be consistent with the code around you! Integer Types link ▶Of the built-in C++ integer types, the only one used is int. If a program needs a variable of a different size, use a precise-width integer type from , such as int16_t. Definition: C++ does not specify the sizes of its integer types. Typically people assume that short is 16 bits, int is 32 bits, long is 32 bits and long long is 64 bits. Pros: Uniformity of declaration. Cons: The sizes of integral types in C++ can vary based on compiler and architecture. Decision: defines types like int16_t, uint32_t, int64_t, etc. You should always use those in preference to short, unsigned long long and the like, when you need a guarantee on the size of an integer. Of the C integer types, only int should be used. When appropriate, you are welcome to use standard types like size_t and ptrdiff_t. We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t. For integers we know can be "big", use int64_t. You should not use the unsigned integer types such as uint32_t, unless the quantity you are representing is really a bit pattern rather than a number, or unless you need defined twos-complement overflow. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this. On Unsigned Integers Some people, including some textbook authors, recommend using unsigned types to represent numbers that are never negative. This is intended as a form of self-documentation. However, in C, the advantages of such documentation are outweighed by the real bugs it can introduce. Consider: for (unsigned int i = foo.Length()-1; i >= 0; --i) ... This code will never terminate! Sometimes gcc will notice this bug and warn you, but often it will not. Equally bad bugs can occur when comparing signed and unsigned variables. Basically, C's type-promotion scheme causes unsigned types to behave differently than one might expect. So, document that a variable is non-negative using assertions. Don't use an unsigned type. 64-bit Portability link ▶Code should be 64-bit and 32-bit friendly. Bear in mind problems of printing, comparisons, and structure alignment. printf() specifiers for some types are not cleanly portable between 32-bit and 64-bit systems. C99 defines some portable format specifiers. Unfortunately, MSVC 7.1 does not understand some of these specifiers and the standard is missing a few, so we have to define our own ugly versions in some cases (in the style of the standard include file inttypes.h): // printf macros for size_t, in the style of inttypes.h #ifdef _LP64 #define __PRIS_PREFIX "z" #else #define __PRIS_PREFIX #endif // Use these macros after a % in a printf format string // to get correct 32/64 bit behavior, like this: // size_t size = records.size(); // printf("%"PRIuS"\n", size); #define PRIdS __PRIS_PREFIX "d" #define PRIxS __PRIS_PREFIX "x" #define PRIuS __PRIS_PREFIX "u" #define PRIXS __PRIS_PREFIX "X" #define PRIoS __PRIS_PREFIX "o" Type DO NOT use DO use Notes void * (or any pointer) %lx %p int64_t %qd, %lld %"PRId64" uint64_t %qu, %llu, %llx %"PRIu64", %"PRIx64" size_t %u %"PRIuS", %"PRIxS" C99 specifies %zu ptrdiff_t %d %"PRIdS" C99 specifies %zd Note that the PRI* macros expand to independent strings which are concatenated by the compiler. Hence if you are using a non-constant format string, you need to insert the value of the macro into the format, rather than the name. It is still possible, as usual, to include length specifiers, etc., after the % when using the PRI* macros. So, e.g. printf("x = %30"PRIuS"\n", x) would expand on 32-bit Linux to printf("x = %30" "u" "\n", x), which the compiler will treat as printf("x = %30u\n", x). Remember that sizeof(void *) != sizeof(int). Use intptr_t if you want a pointer-sized integer. You may need to be careful with structure alignments, particularly for structures being stored on disk. Any class/structure with a int64_t/uint64_t member will by default end up being 8-byte aligned on a 64-bit system. If you have such structures being shared on disk between 32-bit and 64-bit code, you will need to ensure that they are packed the same on both architectures. Most compilers offer a way to alter structure alignment. For gcc, you can use __attribute__((packed)). MSVC offers #pragma pack() and __declspec(align()). Use the LL or ULL suffixes a
Polycom RealPresence 桌面是一个易于使用的视频协作应用程序, 为 PC 用户提供高清质量的音频, 视频和内容共享。与类似的专有应用程序不同, RealPresence 桌面是基于标准的, 并扩展了组织现有的视频启用网络, 使最终用户能够在办公室内或在路上使用相同的高质量体验。 END USER LICENSE AGREEMENT FOR POLYCOM® SOFTWARE IMPORTANT-READ CAREFULLY BEFORE USING THE SOFTWARE: This End-User License Agreement ("Agreement") is a legal agreement between you and/or any company you represent, “you” and either Polycom (Netherlands) B.V. (if you are located in Europe, Middle East, or Africa), Polycom Asia Pacific PTE Ltd. (if you are located in Asia Pacific), or Polycom, Inc. (if you are located in the rest of the world) (each referred to individually and collectively herein as “POLYCOM”), for the SOFTWARE PRODUCT (as defined below) licensed by POLYCOM or its suppliers. Unless otherwise agreed in writing by POLYCOM, by accepting these terms or by installing, downloading, copying, or otherwise using the SOFTWARE PRODUCT, you agree to be and will be bound by the terms of this Agreement as a condition of your license and the terms and conditions of this Agreement will prevail over any inconsistent, conflicting, additional or preprinted terms. If you do not agree to the terms of this Agreement, your use is prohibited and you may not install or use the SOFTWARE PRODUCT. The SOFTWARE PRODUCT is protected by copyright laws and international copyright treaties, as well as other intellectual property laws and treaties. The SOFTWARE PRODUCT is licensed (not sold) to you, and its use is subject to the terms of this Agreement. This is NOT a sale contract. DEFINITIONS “SOFTWARE PRODUCT” means computer software, including any software updates or upgrades thereto, and associated media, printed materials, and "online" or electronic documentation identified on an AUTHORIZED PURCHASE ORDER. “CONCURRENT USER” is a type of license and means the number of personal devices, endpoints, applications or any other authorized means of access that may concurrently use or access the SOFTWARE PRODUCT or one or more of the RealPresence® Clariti™ SOFTWARE PRODUCT components. “DEVICE” means the POLYCOM hardware product with which the SOFTWARE PRODUCT is supplied or, if this SOFTWARE PRODUCT is delivered as software only, on a device (including but not limited to minimum hardware or software requirements) outlined in the SOFTWARE PRODUCT supporting documentation. “AUTHORIZED PURCHASE ORDER” means an ordering document describing the type, duration and number of SOFTWARE PRODUCT licenses ordered by you through, and accepted by, an AUTHORIZED SELLER. “AUTHORIZED SELLER” means POLYCOM or any of POLYCOM’s distributors, resellers or other business partners authorized to resell POLYCOM products. “LICENSE ACTIVATION” means the date you enter the license key provided to you by an AUTHORIZED SELLER. I. DEMONSTRATION/LABORATORY/DEVELOPER LICENSE If you have received the SOFTWARE PRODUCT for purposes of evaluation (“Demonstration”) or you have purchased a Laboratory or Developer License then this SOFTWARE PRODUCT is licensed to you per the terms set forth below. A.Demonstration License. License Grant. Subject to the terms of this Demonstration License, POLYCOM grants you a limited, non- exclusive, non-transferable license, to install and use, on a DEVICE, the number and type of SOFTWARE PRODUCT licenses listed in your AUTHORIZED PURCHASE ORDER solely to evaluate its suitability for your personal or internal business requirements. Demonstration Period. Your Demonstration License is limited to sixty (60) calendar days from LICENSE ACTIVATION (“DEMONSTRATION PERIOD”). Limited Use Software. Portions of the full-use version of the SOFTWARE PRODUCT may be restricted. Full use of the SOFTWARE PRODUCT may be provided when a Standard Use or Subscription license key is purchased. If you have entered data into the SOFTWARE PRODUCT during the DEMONSTRATION PERIOD and your DEMONSTRATION PERIOD ends, you may not have access to your data. POLYCOM accepts no liability for any lost data due to an expiring Demonstration License. 3725-50000-001B Release 3/17/2016 B.Laboratory and Developer License. License Grant. Subject to the terms of this Laboratory and Developer License, POLYCOM grants you a limited, non-exclusive, non-transferable license to install and use, on a DEVICE, the number and type of SOFTWARE PRODUCT licenses listed in your AUTHORIZED PURCHASE ORDER solely to use and test the SOFTWARE PRODUCT in a non-production, evaluation, and personal or internal business environment. Laboratory and Developer License Period. Your Laboratory and Developer License begins upon LICENSE ACTIVATION and ends after the duration listed in your AUTHORIZED PURCHASE ORDER (“LABORATORY AND DEVELOPER LICENSE PERIOD”). C.Disclaimer of Warranty. NOTWITHSTANDING ANYTHING TO THE CONTRARY IN THIS AGREEMENT, THE SOFTWARE PRODUCT IS PROVIDED ONLY FOR EVALUATION PURPOSES ON AN "AS IS" BASIS. POLYCOM EXPRESSLY DISCLAIMS ALL WARRANTIES, REPRESENTATIONS AND CONDITIONS INCLUDING THE IMPLIED WARRANTIES AND/OR CONDITIONS OF MERCHANTABILITY, SATISFACTORY QUALITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. D.Additional Terms. The following additional terms of the Agreement shall also apply to this Demonstration/Laboratory/Developer License: Sections IV. 1 (Restrictions), IV. 2 (Other Rights and Limitations), IV. 4 (Term, Termination and Survival), IV. 7 (Limitation of Liability), IV. 8 (Indemnity), IV. 9 (Disclaimer), IV. 10 (Export Controls), and IV. 11 (Miscellaneous). II. SUBSCRIPTION/TERM LICENSE If you have purchased a Subscription License then this SOFTWARE PRODUCT is licensed to you per the terms set forth below. A. Grant of License. Subject to the terms of this Agreement, and during the SUBSCRIPTION PERIOD, POLYCOM grants to you a non-exclusive, non-transferable (except as set forth herein) license to install and use, on a DEVICE, for personal or internal business purposes the number and type of SOFTWARE PRODUCT licenses listed in your AUTHORIZED PURCHASE ORDER . B.RealPresence One™ Definitions. “BURST” means a limited duration license, listed in your AUTHORIZED PURCHASE ORDER , allowing you to temporarily increase the number of SOFTWARE PRODUCT CONCURRENT CONNECTIONS licensed by you in your purchase order. “CONCURRENT CONNECTION” means a single audio or video communication endpoint in a multipoint conference call. A CONCURRENT CONNECTION is either a Standard Connection or a Universal Connection as those are defined in the SOFTWARE PRODUCT solution documentation. POINT TO POINT calls are not considered CONCURRENT CONNECTIONS and do not require a separate license. “GUEST” means a third party conference call participant invited to a conference call by a USER and who does not fit the definition of USER. GUESTS may not create or manage meetings on behalf of your company. “POINT TO POINT CALLS” means any call made directly between no more than two users on compatible devices that does not utilize POLYCOM’s multipoint bridging technology. POINT TO POINT CALLS may not have all of the same features and capabilities of a call made using a CONCURRENT CONNECTION. “SUBSCRIPTION PERIOD” means the period of time you have licensed the SOFTWARE PRODUCT, beginning upon LICENSE ACTIVATION and ending after the duration listed in your AUTHORIZED PURCHASE ORDER. "USER" means one individual employee, consultant, contractor, agent or student or one video conferencing device not assigned to a unique individual, within your company or legally recognized entity that is authorized by you to directly or indirectly use the SOFTWARE PRODUCT and to whom you have assigned a unique SOFTWARE PRODUCT user identification and password. A USER license is for an individual or single video conferencing device USER only and cannot be shared or used by more than one individual or multiple video conferencing devices not assigned to a unique individual. USER licenses may be reassigned to new USERs. You are not required to purchase a USER license for GUESTS; however, GUESTS do consume a CONCURRENT CONNECTION license. 3725-50000-001B Release 3/17/2016 C. Additional Terms. Except for Section I & III, all of the terms of this Agreement shall also apply to this Subscription License. III. STANDARD USE/PERPETUAL LICENSE If you have purchased a Standard Use License then this SOFTWARE PRODUCT is licensed to you per the terms set forth below. A. Grant of License. Subject to the terms of this Agreement, POLYCOM grants to you a non-exclusive, non- transferable (except as set forth herein), perpetual (unless otherwise terminated per the terms of this Agreement) license to install and use, on a DEVICE, for personal or internal business purposes the number and type of SOFTWARE PRODUCT licenses listed in your AUTHORIZED PURCHASE ORDER. B. Additional Terms. Except for Sections I & II, all of the terms of this Agreement shall also apply to this Standard Use License. IV. ADDITIONAL TERMS AND CONDITIONS 1. Restrictions 1.1You may use the SOFTWARE PRODUCT pursuant to the license grant above and subject to the following terms and the proprietary notices in the SOFTWARE PRODUCT or on the media upon which the SOFTWARE PRODUCT is provided. You are not permitted to lease, rent, distribute, assign, sell or sublicense the SOFTWARE PRODUCT, in whole or in part, or to use the SOFTWARE PRODUCT in a time-sharing, subscription service, service bureau, hosting or outsourcing arrangement or in any other unauthorized manner. Further, no license is granted to you in the human readable code of the SOFTWARE PRODUCT (source code). Except as expressly provided below, this License Agreement does not grant you any rights to patents, copyrights, trade secrets, trademarks, or any other rights in the SOFTWARE PRODUCT. You are solely responsible for use of the SOFTWARE PRODUCT by your agents, contractors, outsourcers, customers and suppliers and their compliance with this Agreement. 1.2You may not reverse engineer, decompile, modify or disassemble the SOFTWARE PRODUCT or otherwise reduce the SOFTWARE PRODUCT to human-perceivable form in whole or in part, except and only to the extent that such activity is expressly permitted by a third party license or applicable laws (in any instance where the law permits any such action, you agree to provide POLYCOM at least ninety (90) days advance written notice of your belief that such action is warranted and permitted and to provide POLYCOM with an opportunity to evaluate if the law’s requirements necessitate such action). The foregoing includes but is not limited to review of data structures or similar materials produced by SOFTWARE PRODUCT. The SOFTWARE PRODUCT is licensed as a single product. Its component parts may not be separated for use on more than one DEVICE. You may not use the SOFTWARE PRODUCT for any illegal purpose or conduct. 1.3You may not modify, translate or create derivative works of the SOFTWARE PRODUCT. 1.4You may not remove or obscure any proprietary notices, identification, label or trademarks on or in the SOFTWARE PRODUCT or the supporting documentation. 2.Other Rights and Limitations 2.1Back-up. Except as expressly provided for under this Agreement you may not copy the SOFTWARE PRODUCT; except, however, during the term of this Agreement you may keep or POLYCOM may provide one copy of the SOFTWARE PRODUCT and, if applicable, one copy of any previous version, for back-up purposes, only to be used in the event of and during a failure of the original. All copies of the SOFTWARE PRODUCT must be marked with the proprietary notices provided on the original SOFTWARE PRODUCT. You may not reproduce the supporting documentation accompanying the SOFTWARE PRODUCT. 2.2Software Transfer. If you have purchased a Standard Use license, you may permanently transfer all of your rights under this Agreement, solely in connection with transfer of the DEVICE, if applicable, provided you retain no copies, you transfer all of the SOFTWARE PRODUCT (including the media and printed materials, any upgrades or updates, and this Agreement), and the recipient agrees to the terms of this Agreement. If the SOFTWARE PRODUCT is an upgrade or update, any transfer must include all prior versions of the SOFTWARE PRODUCT. However, if the SOFTWARE PRODUCT is marked "Not for Resale" or "NFR", you may not resell it or otherwise transfer it for value. 3725-50000-001B Release 3/17/2016 2.3Copyright. All title and copyrights in and to the SOFTWARE PRODUCT (including but not limited to any images, photographs, animations, video, audio, music, text, programs and "applets" incorporated into the SOFTWARE PRODUCT), the accompanying printed materials, and any copies of the SOFTWARE PRODUCT are owned by POLYCOM or its suppliers. Title, ownership rights, and intellectual property rights in the SOFTWARE PRODUCT shall remain in POLYCOM or its suppliers. Title and related rights in the content accessed through the SOFTWARE PRODUCT is the property of such content owner and may be protected by applicable law. This Agreement gives you no rights in such content. 2.4Confidentiality. The SOFTWARE PRODUCT contains valuable proprietary information and trade secrets of POLYCOM and its suppliers that remain the property of POLYCOM. You shall protect the confidentiality of, and avoid disclosure and unauthorized use of, the SOFTWARE PRODUCT. 2.5Information. Using the SOFTWARE PRODUCT means you consent to the SOFTWARE PRODUCT’s transmission of device information (including but not limited to technical information about your device such as logs, statistics, device states, and IP addresses) to POLYCOM and/or POLYCOM partners. POLYCOM may use such device information, along with all technical information you provide to POLYCOM as part of your use of SOFTWARE PRODUCT or SUPPORT SERVICES, for its business purposes in accordance with POLYCOM’s Privacy Policy located at http://www.polycom.com/company/privacy-policy.html. POLYCOM has no obligation to store your data or information and POLYCOM shall not be liable for any data loss. 2.6Dual-Media Software. You may receive the SOFTWARE PRODUCT in more than one medium. Regardless of the type or size of medium you receive, you may use only one medium that is appropriate for your DEVICE. You may not use or install the other medium on another DEVICE. 2.7Reservation of Rights. POLYCOM and its suppliers reserve all rights in the SOFTWARE PRODUCT not expressly granted to you in this Agreement. 2.8Additional Obligations. You are responsible for all equipment and any third party fees (such as carrier charges, internet fees, or provider or airtime charges) necessary to access the SOFTWARE PRODUCT. 2.9Installation. You acknowledge that installation of the SOFTWARE PRODUCT and, as applicable, use of additional software features may involve a license key that may restrict installation of the SOFTWARE PRODUCT to the SOFTWARE PRODUCT licensed. POLYCOM may also embed algorithms in the SOFTWARE PRODUCT that periodically compare the SOFTWARE PRODUCT licenses enabled against the SOFTWARE PRODUCT licensed. You further acknowledge that the SOFTWARE PRODUCT requires activation on initial installation of the SOFTWARE PRODUCT and future events including, but not limited to, updates and changes to your hardware on which the SOFTWARE PRODUCT is installed. You acknowledge that the license key and internal controls in the SOFTWARE PRODUCT may not restrict usage to the licensed amounts and do not ensure compliance with this Agreement. 3.Support Services POLYCOM may provide you with support services, including support services related to the provisioning and availability of SOFTWARE PRODUCT updates and upgrades, related to the SOFTWARE PRODUCT ("SUPPORT SERVICES"). Unless otherwise agreed in writing by POLYCOM, Use of SUPPORT SERVICES and any supplemental software code provided to you as part of the SUPPORT SERVICES is governed by the terms and conditions of POLYCOM's Worldwide Service Program for End User Customers and the applicable Service Description. 4. Term, Termination and Survival This Agreement is in effect from the date of download of the SOFTWARE PRODUCT through the period in the applicable license grant as set forth above. This Agreement will terminate automatically if you fail to comply with any of the terms and conditions of this Agreement. POLYCOM shall have the right to audit your use of the SOFTWARE PRODUCT in conjunction with this Agreement, and you will provide reasonable assistance for this purpose. In the event of any termination, you must cease use of the SOFTWARE PRODUCT, and destroy all copies of the SOFTWARE PRODUCT and all of its component parts. You may terminate this Agreement at any time by destroying the SOFTWARE PRODUCT and all of its component parts. Termination of this Agreement shall not prevent POLYCOM or its suppliers from claiming any further damages. If you do not comply with any of the above restrictions, this license will terminate and you will be liable to POLYCOM and its suppliers for damages or losses 3725-50000-001B Release 3/17/2016 caused by your non-compliance. The waiver by POLYCOM of a specific breach or default shall not constitute the waiver of any subsequent breach or default. The following provisions shall survive the expiration or termination of this Agreement: Sections IV.1 (Restrictions), IV.2 (Other Rights and Limitations), IV.5 (Term, Termination and Survival), IV.7 (Limitation of Liability), IV.8 (Indemnity), IV.9 (Disclaimer), IV.10 (Export Controls), and IV.11 (Miscellaneous). 5. Upgrades If the SOFTWARE PRODUCT is labeled as an upgrade or update, you must be properly licensed to use the software identified by POLYCOM as being eligible for the upgrade or update in order to use the SOFTWARE PRODUCT. A SOFTWARE PRODUCT labeled as an upgrade or update replaces and/or supplements the software that formed the basis for your eligibility for the upgrade or update. You may use the resulting upgraded/updated SOFTWARE PRODUCT only in accordance with the terms of this Agreement. If the SOFTWARE PRODUCT is an upgrade or update of a component of a package of software programs that you licensed as a single product, the SOFTWARE PRODUCT may be used and transferred only as part of that single SOFTWARE PRODUCT package and may not be separated for use on more than one DEVICE. 6. Warranty and Warranty Exclusions 6.1Limited Warranty. Except as otherwise set forth in a Third Party License or in third party license terms set forth below, POLYCOM warrants that (a) the SOFTWARE PRODUCT will perform substantially in accordance with the accompanying documentation for a period of ninety (90) days from the date of shipment by POLYCOM or the SOFTWARE PRODUCT is initially downloaded by you, as applicable, and (b) any SUPPORT SERVICES provided by POLYCOM shall be substantially as described in applicable written materials provided to you by POLYCOM. This warranty is valid only for the original purchaser. POLYCOM DOES NOT WARRANT THAT YOUR USE OF THE SOFTWARE PRODUCT WILL BE UNINTERRUPTED OR ERROR FREE, OR THAT ALL DEFECTS IN THE SOFTWARE PRODUCT WILL BE CORRECTED. YOU ASSUME FULL RESPONSIBILITY FOR THE SELECTION OF THE SOFTWARE PRODUCT TO ACHIEVE YOUR INTENDED RESULTS AND FOR THE INSTALLATION, USE AND RESULTS OBTAINED FROM THE SOFTWARE PRODUCT. IF THE SOFTWARE PRODUCT DOES NOT OPERATE AS WARRANTED ABOVE, POLYCOM'S SOLE OBLIGATION UNDER THIS EXPRESS WARRANTY SHALL BE, AT POLYCOM'S OPTION AND EXPENSE, TO REPAIR OR REPLACE THE DEFECTIVE SOFTWARE, OR IF NEITHER OF THE TWO FOREGOING OPTIONS IS REASONABLY AVAILABLE, POLYCOM MAY, IN ITS SOLE DISCRETION REFUND TO YOU THE PURCHASE PRICE PAID FOR THE DEFECTIVE PRODUCT. Any replacement SOFTWARE PRODUCT will substantially conform to the accompanying documentation and be warranted for the remainder of the original warranty period or thirty (30) days, whichever is longer. 6.2Warranties Exclusive. TO THE FULL EXTENT ALLOWED BY LAW, THE FOREGOING WARRANTIES AND REMEDIES ARE EXCLUSIVE AND ARE IN LIEU OF ALL OTHER WARRANTIES, TERMS, OR CONDITIONS, EXPRESS OR IMPLIED, EITHER IN FACT OR BY OPERATION OF LAW, STATUTORY OR OTHERWISE, INCLUDING WARRANTIES, TERMS, OR CONDITIONS OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, SATISFACTORY QUALITY, CORRESPONDENCE WITH DESCRIPTION, AND NON- INFRINGEMENT, ALL OF WHICH ARE EXPRESSLY DISCLAIMED. POLYCOM NEITHER ASSUMES NOR AUTHORIZES ANY OTHER PERSON TO ASSUME FOR IT ANY OTHER LIABILITY IN CONNECTION WITH THE SALE, INSTALLATION, MAINTENANCE OR USE OF THE SOFTWARE PRODUCT. NO ADVICE OR INFORMATION, WHETHER ORAL OR WRITTEN, OBTAINED BY YOU FROM POLYCOM OR THROUGH OR FROM THE SOFTWARE PRODUCT SHALL CREATE ANY WARRANTY NOT EXPRESSLY STATED IN THIS AGREEMENT. NEITHER POLYCOM NOR ITS SUPPLIERS SHALL BE LIABLE UNDER THIS WARRANTY IF ITS TESTING AND EXAMINATION DISCLOSE THAT THE ALLEGED DEFECT OR MALFUNCTION IN THE SOFTWARE PRODUCT DOES NOT EXIST OR WAS CAUSED BY YOUR OR ANY THIRD PARTY'S MISUSE, NEGLECT, IMPROPER INSTALLATION OR TESTING, UNAUTHORIZED ATTEMPTS TO MODIFY THE SOFTWARE PRODUCT, OR ANY OTHER CAUSE BEYOND THE RANGE OF THE INTENDED USE, OR BY ACCIDENT, FIRE, LIGHTNING, POWER CUTS OR OUTAGES, OTHER HAZARDS, OR ACTS OF GOD. 7. Limitation of Liability YOUR USE OF THE SOFTWARE PRODUCT IS AT YOUR SOLE RISK. YOU WILL BE SOLELY RESPONSIBLE FOR ANY DAMAGE TO YOUR COMPUTER SYSTEM OR LOSS OF DATA THAT RESULTS FROM THE DOWNLOAD OR USE OF THE SOFTWARE PRODUCT. TO THE MAXIMUM EXTENT PERMITTED BY 3725-50000-001B Release 3/17/2016 APPLICABLE LAW, IN NO EVENT SHALL POLYCOM OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, WITHOUT LIMITATION DAMAGES FOR LOSS OF BUSINESS PROFITS OR REVENUE; BUSINESS INTERRUPTION OR WORK STOPPAGE; COMPUTER FAILURE OR MALFUNCTION; LOSS OF BUSINESS INFORMATION, DATA OR DATA USE; LOSS OF GOODWILL; OR ANY OTHER PECUNIARY LOSS) ARISING OUT OF THE USE OF OR INABILITY TO USE THE SOFTWARE PRODUCT OR THE PROVISION OF OR FAILURE TO PROVIDE SUPPORT SERVICES, EVEN IF POLYCOM OR ITS SUPPLIER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL POLYCOM’S SUPPLIERS BE LIABLE FOR ANY DIRECT DAMAGES WHATSOEVER ARISING OUT OF THE USE OR THE INABILITY TO USE THE SOFTWARE PRODUCT. IN ANY CASE, POLYCOM'S ENTIRE LIABILITY SHALL BE LIMITED TO THE GREATER OF THE AMOUNT ACTUALLY PAID BY YOU FOR THE SOFTWARE PRODUCT OR U.S. $1.00. NOTWITHSTANDING THE TERMS OF THIS SECTION 7, IF YOU HAVE ENTERED INTO A POLYCOM SUPPORT SERVICES AGREEMENT, POLYCOM'S ENTIRE LIABILITY REGARDING SUPPORT SERVICES SHALL BE GOVERNED BY THE TERMS OF THAT AGREEMENT. 8. Indemnity You agree to indemnify and hold harmless POLYCOM and its subsidiaries, affiliates, officers, agents, co-branders,customers, suppliers or other partners, and employees, from any loss, claim or demand, including reasonable attorneys' fees, made by any third party due to or arising out of your use of the SOFTWARE PRODUCT, your connection to the SOFTWARE PRODUCT, or your violation of the Terms. 9. Disclaimers 9.1Local Laws. Some countries, states, or provinces do not allow the exclusion or limitation of implied warranties or the limitation of incidental or consequential damages for certain products supplied to consumers, or the limitation of liability for death or personal injury, so the above limitations and exclusions may be limited in their application to you. When the implied warranties are not allowed to be excluded in their entirety due to local law, they will be limited to the duration of the applicable warranty. 9.2Quality. POLYCOM cannot guarantee that the SOFTWARE PRODUCT will always function without disruptions, delay or errors. A number of factors may impact the quality of your communications and use of the SOFTWARE PRODUCT, and may result in the failure of your communications including but not limited to: your local network, firewall, your internet service provider, the public internet, the public switched telephone network and your power supply. POLYCOM takes no responsibility for any disruption, interruption or delay caused by any failure of or inadequacy in any of these items or any other items over which we have no control. 9.3High Risk Use. The SOFTWARE PRODUCT is not fault-tolerant and is not designed or intended for use in hazardous environments requiring fail-safe performance or any application in which the failure of the SOFTWARE PRODUCT could lead directly to death, personal injury, or severe physical or property damage (collectively, "High Risk Use"). Because the SOFTWARE PRODUCT is not guaranteed to function without disruptions, delay or errors (see section 9.2, above), the SOFTWARE PRODUCT should never be utilized as the sole means of communication in High Risk Use. You assume full responsibility for the selection of the SOFTWARE PRODUCT for High Risk Use. POLYCOM AND ITS SUPPLIERS EXPRESSLY DISCLAIM ANY EXPRESS OR IMPLIED WARRANTY OF FITNESS FOR HIGH RISK USE. 10. Export Controls You acknowledge that the SOFTWARE PRODUCT may be subject to export and/or import laws and regulations of various countries, including but not limited to the US Export Administration Regulations, restricting the download, transfer, reexport, sale and import of the SOFTWARE PRODUCT to certain countries and persons. You further acknowledge that the SOFTWARE PRODUCT may include encryption/decryption features subject to licensing restrictions under US and other applicable laws for export, re-export, import or in-country transfer. You shall fully comply with all applicable export license restrictions and requirements as well as with all laws and regulations relating to the importation of the SOFTWARE PRODUCT, in the United States and in any foreign jurisdiction into which the SOFTWARE PRODUCT is downloaded or used. Without limiting the foregoing, the SOFTWARE PRODUCT may not be downloaded or otherwise exported or re-exported (i) into (or to a national or resident of, other than a national lawfully admitted for permanent residence in third-countries) Cuba, Iran, North Korea, Sudan, Syria or any other country against which the U.S. maintains comprehensive country-wide sanctions from time to time; (ii) any end user known, or having reason to be known, will utilize them in the design, development or production of nuclear, chemical or biological weapons; or (iii) to any restricted party identified on the U.S. Treasury Department's list of Specially Designated Nationals, the U.S. Commerce Department's Denied Persons List, Entity 3725-50000-001B Release 3/17/2016 List or Unverified List as in force from time to time (see: http://www.bis.doc.gov/complianceandenforcement/liststocheck.htm). By downloading or using the SOFTWARE PRODUCT, you are agreeing to the foregoing and you are representing and warranting that you are not located in, under the control of, acting on behalf of, or a national or resident of any such country or on any such list. If you obtained this SOFTWARE PRODUCT outside of the United States, you are also agreeing that you will not export or re-export it in violation of the laws of the country in which it was obtained including but not limited to export and sanctions laws. 11. Miscellaneous 11.1Governing Law. This Agreement shall be governed by the laws of the State of California, without reference to conflict of laws principles and any disputes will be subject to the exclusive jurisdiction of the Superior Court of Santa Clara County and/or the United States District Court for the Northern District of California. The losing party in any dispute will pay all court costs and legal fees finally awarded. . The United Nations Convention on Contracts for the International Sale of Goods (1980) and the Uniform Computer Information Transactions Act (UCITA) are hereby excluded in their entirety from application to this Agreement. 11.2Entire Agreement. This Agreement represents the complete agreement concerning the SOFTWARE PRODUCT and may be amended only by a writing executed by both parties. If any provision of this Agreement is held to be unenforceable, such provision shall be reformed only to the extent necessary to make it enforceable. 11.3Contact. If you have any questions concerning this Agreement, or if you desire to contact POLYCOM for any reason, please contact the POLYCOM office serving your country. 11.4U.S. Government Restricted Rights. The software and documentation provided by POLYCOM pursuant to this Agreement are “Commercial Items,” as the term is defined at 48 C.F.R. §2.101, consisting of “Commercial Computer Software” and “Commercial Computer Software Documentation,” as such terms are used in 48 C.F.R. §12.212 or 48 C.F.R. §227.7202, as applicable. Consistent with 48 C.F.R. §12.212 or 48 C.F.R. §§227.7202-1through 227.7202-4, as applicable, the Commercial Computer Software and Commercial Computer Software Documentation are licensed to United States Government end users (1) only as Commercial Items and (2) with only those rights as are granted to all other users pursuant to the terms of this Agreement. 11.5Third Party Software. The SOFTWARE PRODUCT may be distributed with software governed by licenses from third parties (“Third Party Software” and “Third Party License”). Any Third Party Software is licensed to you subject to the terms and conditions of this Agreement, notwithstanding anything to the contrary in this Agreement, the corresponding Third Party License. More information on Third Party Licenses terms applicable to the SOFTWARE PRODUCT can be found in the documentation for each SOFTWARE PRODUCT and at http://documents.polycom.com/indexes/licenses. POLYCOM makes no separate representation or warranty concerning Third Party Software and shall have no obligation or liability with respect to Third Party Software. If the Third Party Licenses include licenses that provide for the availability of source code and the corresponding source code is not included with the SOFTWARE PRODUCT, then check the documentation supplied with each SOFTWARE PRODUCT to learn how to obtain such source code. 11.6Translations. This Agreement may have been translated into various languages for the convenience of POLYCOM’s customers. While the translation is correct to the best of POLYCOM’s knowledge, POLYCOM is not responsible or liable in the event of an inaccuracy. English is the controlling language of this Agreement, and any translation has been prepared for you as a courtesy only. In the event of a conflict between the English-languageversion of this Agreement and a version that has been translated into another language, the English-languageversion of this Agreement shall control. 11.7Application Programming Interfaces (API). To the extent the SOFTWARE PRODUCT includes APIs, you agree that your access and use of the APIs will be governed by the terms of the current “Application Programming Interfaces License Agreement” located at www.support.polycom.com. BY INSTALLING, COPYING, OR OTHERWISE USING THIS SOFTWARE PRODUCT YOU ACKNOWLEDGE THAT YOU HAVE READ, UNDERSTAND AND AGREE TO BE BOUND BY THESE TERMS AND CONDITIONS. Polycom, Inc. © 2015. ALL RIGHTS RESERVED. 6001 America Center Drive San Jose, CA 95002 U.S.A. 3725-50000-001B Release 3/17/2016
Contents Overview 1 Lesson 1: Concepts – Locks and Lock Manager 3 Lesson 2: Concepts – Batch and Transaction 31 Lesson 3: Concepts – Locks and Applications 51 Lesson 4: Information Collection and Analysis 63 Lesson 5: Concepts – Formulating and Implementing Resolution 81 Module 4: Troubleshooting Locking and Blocking Overview At the end of this module, you will be able to:  Discuss how lock manager uses lock mode, lock resources, and lock compatibility to achieve transaction isolation.  Describe the various transaction types and how transactions differ from batches.  Describe how to troubleshoot blocking and locking issues.  Analyze the output of blocking scripts and Microsoft® SQL Server™ Profiler to troubleshoot locking and blocking issues.  Formulate hypothesis to resolve locking and blocking issues. Lesson 1: Concepts – Locks and Lock Manager This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Describe locking architecture used by SQL Server.  Identify the various lock modes used by SQL Server.  Discuss lock compatibility and concurrent access.  Identify different types of lock resources.  Discuss dynamic locking and lock escalation.  Differentiate locks, latches, and other SQL Server internal “locking” mechanism such as spinlocks and other synchronization objects. Recommended Reading  Chapter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  SOX000821700049 – SQL 7.0 How to interpret lock resource Ids  SOX000925700237 – TITLE: Lock escalation in SQL 7.0  SOX001109700040 – INF: Queries with PREFETCH in the plan hold lock until the end of transaction Locking Concepts Delivery Tip Prior to delivering this material, test the class to see if they fully understand the different isolation levels. If the class is not confident in their understanding, review appendix A04_Locking and its accompanying PowerPoint® file. Transactions in SQL Server provide the ACID properties: Atomicity A transaction either commits or aborts. If a transaction commits, all of its effects remain. If it aborts, all of its effects are undone. It is an “all or nothing” operation. Consistency An application should maintain the consistency of a database. For example, if you defer constraint checking, it is your responsibility to ensure that the database is consistent. Isolation Concurrent transactions are isolated from the updates of other incomplete transactions. These updates do not constitute a consistent state. This property is often called serializability. For example, a second transaction traversing the doubly linked list mentioned above would see the list before or after the insert, but it will see only complete changes. Durability After a transaction commits, its effects will persist even if there are system failures. Consistency and isolation are the most important in describing SQL Server’s locking model. It is up to the application to define what consistency means, and isolation in some form is needed to achieve consistent results. SQL Server uses locking to achieve isolation. Definition of Dependency: A set of transactions can run concurrently if their outputs are disjoint from the union of one another’s input and output sets. For example, if T1 writes some object that is in T2’s input or output set, there is a dependency between T1 and T2. Bad Dependencies These include lost updates, dirty reads, non-repeatable reads, and phantoms. ANSI SQL Isolation Levels An isolation level determines the degree to which data is isolated for use by one process and guarded against interference from other processes. Prior to SQL Server 7.0, REPEATABLE READ and SERIALIZABLE isolation levels were synonymous. There was no way to prevent non-repeatable reads while not preventing phantoms. By default, SQL Server 2000 operates at an isolation level of READ COMMITTED. To make use of either more or less strict isolation levels in applications, locking can be customized for an entire session by setting the isolation level of the session with the SET TRANSACTION ISOLATION LEVEL statement. To determine the transaction isolation level currently set, use the DBCC USEROPTIONS statement, for example: USE pubs GO SET TRANSACTION ISOLATION LEVEL REPEATABLE READ GO DBCC USEROPTIONS GO Multigranular Locking Multigranular Locking In our example, if one transaction (T1) holds an exclusive lock at the table level, and another transaction (T2) holds an exclusive lock at the row level, each of the transactions believe they have exclusive access to the resource. In this scenario, since T1 believes it locks the entire table, it might inadvertently make changes to the same row that T2 thought it has locked exclusively. In a multigranular locking environment, there must be a way to effectively overcome this scenario. Intent lock is the answer to this problem. Intent Lock Intent Lock is the term used to mean placing a marker in a higher-level lock queue. The type of intent lock can also be called the multigranular lock mode. An intent lock indicates that SQL Server wants to acquire a shared (S) lock or exclusive (X) lock on some of the resources lower down in the hierarchy. For example, a shared intent lock placed at the table level means that a transaction intends on placing shared (S) locks on pages or rows within that table. Setting an intent lock at the table level prevents another transaction from subsequently acquiring an exclusive (X) lock on the table containing that page. Intent locks improve performance because SQL Server examines intent locks only at the table level to determine whether a transaction can safely acquire a lock on that table. This removes the requirement to examine every row or page lock on the table to determine whether a transaction can lock the entire table. Lock Mode The code shown in the slide represents how the lock mode is stored internally. You can see these codes by querying the master.dbo.spt_values table: SELECT * FROM master.dbo.spt_values WHERE type = N'L' However, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. For example, value of req_mode = 3 represents the Shared lock mode rather than the Schema Modification lock mode. Lock Compatibility These locks can apply at any coarser level of granularity. If a row is locked, SQL Server will apply intent locks at both the page and the table level. If a page is locked, SQL Server will apply an intent lock at the table level. SIX locks imply that we have shared access to a resource and we have also placed X locks at a lower level in the hierarchy. SQL Server never asks for SIX locks directly, they are always the result of a conversion. For example, suppose a transaction scanned a page using an S lock and then subsequently decided to perform a row level update. The row would obtain an X lock, but now the page would require an IX lock. The resultant mode on the page would be SIX. Another type of table lock is a schema stability lock (Sch-S) and is compatible with all table locks except the schema modification lock (Sch-M). The schema modification lock (Sch-M) is incompatible with all table locks. Locking Resources Delivery Tip Note the differences between Key and Key Range locks. Key Range locks will be covered in a couple of slides. SQL Server can lock these resources: Item Description DB A database. File A database file Index An entire index of a table. Table An entire table, including all data and indexes. Extent A contiguous group of data pages or index pages. Page An 8-KB data page or index page. Key Row lock within an index. Key-range A key-range. Used to lock ranges between records in a table to prevent phantom insertions or deletions into a set of records. Ensures serializable transactions. RID A Row Identifier. Used to individually lock a single row within a table. Application A lock resource defined by an application. The lock manager knows nothing about the resource format. It simply compares the 'strings' representing the lock resources to determine whether it has found a match. If a match is found, it knows that resource is already locked. Some of the resources have “sub-resources.” The followings are sub-resources displayed by the sp_lock output: Database Lock Sub-Resources: Full Database Lock (default) [BULK-OP-DB] – Bulk Operation Lock for Database [BULK-OP-LOG] – Bulk Operation Lock for Log Table Lock Sub-Resources: Full Table Lock (default) [UPD-STATS] – Update statistics Lock [COMPILE] – Compile Lock Index Lock sub-Resources: Full Index Lock (default) [INDEX_ID] – Index ID Lock [INDEX_NAME] – Index Name Lock [BULK_ALLOC] – Bulk Allocation Lock [DEFRAG] – Defragmentation Lock For more information, see also… SOX000821700049 SQL 7.0 How to interpret lock resource Ids Lock Resource Block The resource type has the following resource block format: Resource Type (Code) Content DB (2) Data 1: sub-resource; Data 2: 0; Data 3: 0 File (3) Data 1: File ID; Data 2: 0; Data 3: 0 Index (4) Data 1: Object ID; Data 2: sub-resource; Data 3: Index ID Table (5) Data 1: Object ID; Data 2: sub-resource; Data 3: 0. Page (6) Data 1: Page Number; Data 3: 0. Key (7) Data 1: Object ID; Data 2: Index ID; Data 3: Hashed Key Extent (8) Data 1: Extent ID; Data 3: 0. RID (9) Data 1: RID; Data 3: 0. Application (10) Data 1: Application resource name The rsc_bin column of master..syslockinfo contains the resource block in hexadecimal format. For an example of how to decode value from this column using the information above, let us assume we have the following value: 0x000705001F83D775010002014F0BEC4E With byte swapping within each field, this can be decoded as: Byte 0: Flag – 0x00 Byte 1: Resource Type – 0x07 (Key) Byte 2-3: DBID – 0x0005 Byte 4-7: ObjectID – 0x 75D7831F (1977058079) Byte 8-9: IndexID – 0x0001 Byte 10-16: Hash Key value – 0x 02014F0BEC4E For more information about how to decode this value, see also… Inside SQL Server 2000, pages 803 and 806. Key Range Locking Key Range Locking To support SERIALIZABLE transaction semantics, SQL Server needs to lock sets of rows specified by a predicate, such as WHERE salary BETWEEN 30000 AND 50000 SQL Server needs to lock data that does not exist! If no rows satisfy the WHERE condition the first time the range is scanned, no rows should be returned on any subsequent scans. Key range locks are similar to row locks on index keys (whether clustered or not). The locks are placed on individual keys rather than at the node level. The hash value consists of all the key components and the locator. So, for a nonclustered index over a heap, where columns c1 and c2 where indexed, the hash would contain contributions from c1, c2 and the RID. A key range lock applied to a particular key means that all keys between the value locked and the next value would be locked for all data modification. Key range locks can lock a slightly larger range than that implied by the WHERE clause. Suppose the following select was executed in a transaction with isolation level SERIALIZABLE: SELECT * FROM members WHERE first_name between ‘Al’ and ‘Carl’ If 'Al', 'Bob', and 'Dave' are index keys in the table, the first two of these would acquire key range locks. Although this would prevent anyone from inserting either 'Alex' or 'Ben', it would also prevent someone from inserting 'Dan', which is not within the range of the WHERE clause. Prior to SQL Server 7.0, page locking was used to prevent phantoms by locking the entire set of pages on which the phantom would exist. This can be too conservative. Key Range locking lets SQL Server lock only a much more restrictive area of the table. Impact Key-range locking ensures that these scenarios are SERIALIZABLE:  Range scan query  Singleton fetch of nonexistent row  Delete operation  Insert operation However, the following conditions must be satisfied before key-range locking can occur:  The transaction-isolation level must be set to SERIALIZABLE.  The operation performed on the data must use an index range access. Range locking is activated only when query processing (such as the optimizer) chooses an index path to access the data. Key Range Lock Mode Again, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. Dynamic Locking When modifying individual rows, SQL Server typically would take row locks to maximize concurrency (for example, OLTP, order-entry application). When scanning larger volumes of data, it would be more appropriate to take page or table locks to minimize the cost of acquiring locks (for example, DSS, data warehouse, reporting). Locking Decision The decision about which unit to lock is made dynamically, taking many factors into account, including other activity on the system. For example, if there are multiple transactions currently accessing a table, SQL Server will tend to favor row locking more so than it otherwise would. It may mean the difference between scanning the table now and paying a bit more in locking cost, or having to wait to acquire a more coarse lock. A preliminary locking decision is made during query optimization, but that decision can be adjusted when the query is actually executed. Lock Escalation When the lock count for the transaction exceeds and is a multiple of ESCALATION_THRESHOLD (1250), the Lock Manager attempts to escalate. For example, when a transaction acquired 1250 locks, lock manager will try to escalate. The number of locks held may continue to increase after the escalation attempt (for example, because new tables are accessed, or the previous lock escalation attempts failed due to incompatible locks held by another spid). If the lock count for this transaction reaches 2500 (1250 * 2), Lock Manager will attempt escalation again. The Lock Manager looks at the lock memory it is using and if it is more than 40 percent of SQL Server’s allocated buffer pool memory, it tries to find a scan (SDES) where no escalation has already been performed. It then repeats the search operation until all scans have been escalated or until the memory used drops under the MEMORY_LOAD_ESCALATION_THRESHOLD (40%) value. If lock escalation is not possible or fails to significantly reduce lock memory footprint, SQL Server can continue to acquire locks until the total lock memory reaches 60 percent of the buffer pool (MAX_LOCK_RESOURCE_MEMORY_PERCENTAGE=60). Lock escalation may be also done when a single scan (SDES) holds more than LOCK_ESCALATION_THRESHOLD (765) locks. There is no lock escalation on temporary tables or system tables. Trace Flag 1211 disables lock escalation. Important Do not relay this to the customer without careful consideration. Lock escalation is a necessary feature, not something to be avoided completely. Trace flags are global and disabling lock escalation could lead to out of memory situations, extremely poor performing queries, or other problems. Lock escalation tracing can be seen using the Profiler or with the general locking trace flag, -T1200. However, Trace Flag 1200 shows all lock activity so it should not be usable on a production system. For more information, see also… SOX000925700237 “TITLE: SQL 7.0 Lock escalation in SQL 7.0” Lock Timeout Application Lock Timeout An application can set lock timeout for a session with the SET option: SET LOCK_TIMEOUT N where N is a number of milliseconds. A value of -1 means that there will be no timeout, which is equivalent to the version 6.5 behavior. A value of 0 means that there will be no waiting; if a process finds a resource locked, it will generate error message 1222 and continue with the next statement. The current value of LOCK_TIMEOUT is stored in the global variable @@lock_timeout. Note After a lock timeout any transaction containing the statement, is rolled back or canceled by SQL Server 2000 (bug#352640 was filed). This behavior is different from that of SQL Server 7.0. With SQL Server 7.0, the application must have an error handler that can trap error 1222 and if an application does not trap the error, it can proceed unaware that an individual statement within a transaction has been canceled, and errors can occur because statements later in the transaction may depend on the statement that was never executed. Bug#352640 is fixed in hotfix build 8.00.266 whereby a lock timeout will only Internal Lock Timeout At time, internal operations within SQL Server will attempt to acquire locks via lock manager. Typically, these lock requests are issued with “no waiting.” For example, the ghost record processing might try to clean up rows on a particular page, and before it can do that, it needs to lock the page. Thus, the ghost record manager will request a page lock with no wait so that if it cannot lock the page, it will just move on to other pages; it can always come back to this page later. If you look at SQL Profiler Lock: Timeout events, internal lock timeout typically have a duration value of zero. Lock Duration Lock Mode and Transaction Isolation Level For REPEATABLE READ transaction isolation level, update locks are held until data is read and processed, unless promoted to exclusive locks. "Data is processed" means that we have decided whether the row in question matched the search criteria; if not then the update lock is released, otherwise, we get an exclusive lock and make the modification. Consider the following query: use northwind go dbcc traceon(3604, 1200, 1211) -- turn on lock tracing -- and disable escalation go set transaction isolation level repeatable read begin tran update dbo.[order details] set discount = convert (real, discount) where discount = 0.0 exec sp_lock Update locks are promoted to exclusive locks when there is a match; otherwise, the update lock is released. The sp_lock output verifies that the SPID does not hold any update locks or shared locks at the end of the query. Lock escalation is turned off so that exclusive table lock is not held at the end. Warning Do not use trace flag 1200 in a production environment because it produces a lot of output and slows down the server. Trace flag 1211 should not be used unless you have done extensive study to make sure it helps with performance. These trace flags are used here for illustration and learning purposes only. Lock Ownership Most of the locking discussion in this lesson relates to locks owned by “transactions.” In addition to transaction, cursor and session can be owners of locks and they both affect how long locks are held. For every row that is fetched, when SCROLL_LOCKS option is used, regardless of the state of a transaction, a cursor lock is held until the next row is fetched or when the cursor is closed. Locks owned by session are outside the scope of a transaction. The duration of these locks are bounded by the connection and the process will continue to hold these locks until the process disconnects. A typical lock owned by session is the database (DB) lock. Locking – Read Committed Scan Under read committed isolation level, when database pages are scanned, shared locks are held when the page is read and processed. The shared locks are released “behind” the scan and allow other transactions to update rows. It is important to note that the shared lock currently acquired will not be released until shared lock for the next page is successfully acquired (this is commonly know as “crabbing”). If the same pages are scanned again, rows may be modified or deleted by other transactions. Locking – Repeatable Read Scan Under repeatable read isolation level, when database pages are scanned, shared locks are held when the page is read and processed. SQL Server continues to hold these shared locks, thus preventing other transactions to update rows. If the same pages are scanned again, previously scanned rows will not change but new rows may be added by other transactions. Locking – Serializable Read Scan Under serializable read isolation level, when database pages are scanned, shared locks are held not only on rows but also on scanned key range. SQL Server continues to hold these shared locks until the end of transaction. Because key range locks are held, not only will this prevent other transactions from modifying the rows, no new rows can be inserted. Prefetch and Isolation Level Prefetch and Locking Behavior The prefetch feature is available for use with SQL Server 7.0 and SQL Server 2000. When searching for data using a nonclustered index, the index is searched for a particular value. When that value is found, the index points to the disk address. The traditional approach would be to immediately issue an I/O for that row, given the disk address. The result is one synchronous I/O per row and, at most, one disk at a time working to evaluate the query. This does not take advantage of striped disk sets. The prefetch feature takes a different approach. It continues looking for more record pointers in the nonclustered index. When it has collected a number of them, it provides the storage engine with prefetch hints. These hints tell the storage engine that the query processor will need these particular records soon. The storage engine can now issue several I/Os simultaneously, taking advantage of striped disk sets to execute multiple operations simultaneously. For example, if the engine is scanning a nonclustered index to determine which rows qualify but will eventually need to visit the data page as well to access columns that are not in the index, it may decide to submit asynchronous page read requests for a group of qualifying rows. The prefetch data pages are then revisited later to avoid waiting for each individual page read to complete in a serial fashion. This data access path requires that a lock be held between the prefetch request and the row lookup to stabilize the row on the page so it is not to be moved by a page split or clustered key update. For our example, the isolation level of the query is escalated to REPEATABLE READ, overriding the transaction isolation level. With SQL Server 7.0 and SQL Server 2000, portions of a transaction can execute at a different transaction isolation level than the entire transaction itself. This is implemented as lock classes. Lock classes are used to control lock lifetime when portions of a transaction need to execute at a stricter isolation level than the underlying transaction. Unfortunately, in SQL Server 7.0 and SQL Server 2000, the lock class is created at the topmost operator of the query and hence released only at the end of the query. Currently there is no support to release the lock (lock class) after the row has been discarded or fetched by the filter or join operator. This is because isolation level can be set at the query level via a lock class, but no lower. Because of this, locks acquired during the query will not be released until the query completes. If prefetch is occurring you may see a single SPID that holds hundreds of Shared KEY or PAG locks even though the connection’s isolation level is READ COMMITTED. Isolation level can be determined from DBCC PSS output. For details about this behavior see “SOX001109700040 INF: Queries with PREFETCH in the plan hold lock until the end of transaction”. Other Locking Mechanism Lock manager does not manage latches and spinlocks. Latches Latches are internal mechanisms used to protect pages while doing operations such as placing a row physically on a page, compressing space on a page, or retrieving rows from a page. Latches can roughly be divided into I/O latches and non-I/O latches. If you see a high number of non-I/O related latches, SQL Server is usually doing a large number of hash or sort operations in tempdb. You can monitor latch activities via DBCC SQLPERF(‘WAITSTATS’) command. Spinlock A spinlock is an internal data structure that is used to protect vital information that is shared within SQL Server. On a multi-processor machine, when SQL Server tries to access a particular resource protected by a spinlock, it must first acquire the spinlock. If it fails, it executes a loop that will check to see if the lock is available and if not, decrements a counter. If the counter reaches zero, it yields the processor to another thread and goes into a “sleep” (wait) state for a pre-determined amount of time. When it wakes, hopefully, the lock is free and available. If not, the loop starts again and it is terminated only when the lock is acquired. The reason for implementing a spinlock is that it is probably less costly to “spin” for a short time rather than yielding the processor. Yielding the processor will force an expensive context switch where:  The old thread’s state must be saved  The new thread’s state must be reloaded  The data stored in the L1 and L2 cache are useless to the processor On a single-processor computer, the loop is not useful because no other thread can be running and thus, no one can release the spinlock for the currently executing thread to acquire. In this situation, the thread yields the processor immediately. Lesson 2: Concepts – Batch and Transaction This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Review batch processing and error checking.  Review explicit, implicit and autocommit transactions and transaction nesting level.  Discuss how commit and rollback transaction done in stored procedure and trigger affects transaction nesting level.  Discuss various transaction isolation level and their impact on locking.  Discuss the difference between aborting a statement, a transaction, and a batch.  Describe how @@error, @@transcount, and @@rowcount can be used for error checking and handling. Recommended Reading  Charter 12 “Transactions and Triggers”, Inside SQL Server 2000 by Kalen Delaney Batch Definition SQL Profiler Statements and Batches To help further your understanding of what is a batch and what is a statement, you can use SQL Profiler to study the definition of batch and statement.  Try This: Using SQL Profiler to Analyze Batch 1. Log on to a server with Query Analyzer 2. Startup the SQL Profiler against the same server 3. Start a trace using the “StandardSQLProfiler” template 4. Execute the following using Query Analyzer: SELECT @@VERSION SELECT @@SPID The ‘SQL:BatchCompleted’ event is captured by the trace. It shows both the statements as a single batch. 5. Now execute the following using Query Analyzer {call sp_who()} What shows up? The ‘RPC:Completed’ with the sp_who information. RPC is simply another entry point to the SQL Server to call stored procedures with native data types. This allows one to avoid parsing. The ‘RPC:Completed’ event should be considered the same as a batch for the purposes of this discussion. Stop the current trace and start a new trace using the “SQLProfilerTSQL_SPs” template. Issue the same command as outlines in step 5 above. Looking at the output, not only can you see the batch markers but each statement as executed within the batch. Autocommit, Explicit, and Implicit Transaction Autocommit Transaction Mode (Default) Autocommit mode is the default transaction management mode of SQL Server. Every Transact-SQL statement, whether it is a standalone statement or part of a batch, is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back. A SQL Server connection operates in autocommit mode whenever this default mode has not been overridden by either explicit or implicit transactions. Autocommit mode is also the default mode for ADO, OLE DB, ODBC, and DB-Library. A SQL Server connection operates in autocommit mode until a BEGIN TRANSACTION statement starts an explicit transaction, or implicit transaction mode is set on. When the explicit transaction is committed or rolled back, or when implicit transaction mode is turned off, SQL Server returns to autocommit mode. Explicit Transaction Mode An explicit transaction is a transaction that starts with a BEGIN TRANSACTION statement. An explicit transaction can contain one or more statements and must be terminated by either a COMMIT TRANSACTION or a ROLLBACK TRANSACTION statement. Implicit Transaction Mode SQL Server can automatically or, more precisely, implicitly start a transaction for you if a SET IMPLICIT_TRANSACTIONS ON statement is run or if the implicit transaction option is turned on globally by running sp_configure ‘user options’ 2. (Actually, the bit mask 0x2 must be turned on for the user option so you might have to perform an ‘OR’ operation with the existing user option value.) See SQL Server 2000 Books Online on how to turn on implicit transaction under ODBC and OLE DB (acdata.chm::/ac_8_md_06_2g6r.htm). Transaction Nesting Explicit transactions can be nested. Committing inner transactions is ignored by SQL Server other than to decrements @@TRANCOUNT. The transaction is either committed or rolled back based on the action taken at the end of the outermost transaction. If the outer transaction is committed, the inner nested transactions are also committed. If the outer transaction is rolled back, then all inner transactions are also rolled back, regardless of whether the inner transactions were individually committed. Each call to COMMIT TRANSACTION applies to the last executed BEGIN TRANSACTION. If the BEGIN TRANSACTION statements are nested, then a COMMIT statement applies only to the last nested transaction, which is the innermost transaction. Even if a COMMIT TRANSACTION transaction_name statement within a nested transaction refers to the transaction name of the outer transaction, the commit applies only to the innermost transaction. If a ROLLBACK TRANSACTION statement without a transaction_name parameter is executed at any level of a set of nested transaction, it rolls back all the nested transactions, including the outermost transaction. The @@TRANCOUNT function records the current transaction nesting level. Each BEGIN TRANSACTION statement increments @@TRANCOUNT by one. Each COMMIT TRANSACTION statement decrements @@TRANCOUNT by one. A ROLLBACK TRANSACTION statement that does not have a transaction name rolls back all nested transactions and decrements @@TRANCOUNT to 0. A ROLLBACK TRANSACTION that uses the transaction name of the outermost transaction in a set of nested transactions rolls back all the nested transactions and decrements @@TRANCOUNT to 0. When you are unsure if you are already in a transaction, SELECT @@TRANCOUNT to determine whether it is 1 or more. If @@TRANCOUNT is 0 you are not in a transaction. You can also find the transaction nesting level by checking the sysprocess.open_tran column. See SQL Server 2000 Books Online topic “Nesting Transactions” (acdata.chm::/ac_8_md_06_66nq.htm) for more information. Statement, Transaction, and Batch Abort One batch can have many statements and one transaction can have multiple statements, also. One transaction can span multiple batches and one batch can have multiple transactions. Statement Abort Currently executing statement is aborted. This can be a bit confusing when you start talking about statements in a trigger or stored procedure. Let us look closely at the following trigger: CREATE TRIGGER TRG8134 ON TBL8134 AFTER INSERT AS BEGIN SELECT 1/0 SELECT 'Next command in trigger' END To fire the INSERT trigger, the batch could be as simple as ‘INSERT INTO TBL8134 VALUES(1)’. However, the trigger contains two statements that must be executed as part of the batch to satisfy the clients insert request. When the ‘SELECT 1/0’ causes the divide by zero error, a statement abort is issued for the ‘SELECT 1/0’ statement. Batch and Transaction Abort On SQL Server 2000 (and SQL Server 7.0) whenever a non-informational error is encountered in a trigger, the statement abort is promoted to a batch and transactional abort. Thus, in the example the statement abort for ‘select 1/0’ promotion results in an entire batch abort. No further statements in the trigger or batch will be executed and a rollback is issued. On SQL Server 6.5, the statement aborts immediately and results in a transaction abort. However, the rest of the statements within the trigger are executed. This trigger could return ‘Next command in trigger’ as a result set. Once the trigger completes the batch abort promotion takes effect. Conversely, submitting a similar set of statements in a standalone batch can result in different behavior. SELECT 1/0 SELECT 'Next command in batch' Not considering the set option possibilities, a divide by zero error generally results in a statement abort. Since it is not in a trigger, the promotion to a batch abort is avoided and subsequent SELECT statement can execute. The programmer should add an “if @@ERROR” check immediately after the ‘select 1/0’ to T-SQL execution to control the flow correctly. Aborting and Set Options ARITHABORT If SET ARITHABORT is ON, these error conditions cause the query or batch to terminate. If the errors occur in a transaction, the transaction is rolled back. If SET ARITHABORT is OFF and one of these errors occurs, a warning message is displayed, and NULL is assigned to the result of the arithmetic operation. When an INSERT, DELETE, or UPDATE statement encounters an arithmetic error (overflow, divide-by-zero, or a domain error) during expression evaluation when SET ARITHABORT is OFF, SQL Server inserts or updates a NULL value. If the target column is not nullable, the insert or update action fails and the user receives an error. XACT_ABORT When SET XACT_ABORT is ON, if a Transact-SQL statement raises a run-time error, the entire transaction is terminated and rolled back. When OFF, only the Transact-SQL statement that raised the error is rolled back and the transaction continues processing. Compile errors, such as syntax errors, are not affected by SET XACT_ABORT. For example: CREATE TABLE t1 (a int PRIMARY KEY) CREATE TABLE t2 (a int REFERENCES t1(a)) GO INSERT INTO t1 VALUES (1) INSERT INTO t1 VALUES (3) INSERT INTO t1 VALUES (4) INSERT INTO t1 VALUES (6) GO SET XACT_ABORT OFF GO BEGIN TRAN INSERT INTO t2 VALUES (1) INSERT INTO t2 VALUES (2) /* Foreign key error */ INSERT INTO t2 VALUES (3) COMMIT TRAN SELECT 'Continue running batch 1...' GO SET XACT_ABORT ON GO BEGIN TRAN INSERT INTO t2 VALUES (4) INSERT INTO t2 VALUES (5) /* Foreign key error */ INSERT INTO t2 VALUES (6) COMMIT TRAN SELECT 'Continue running batch 2...' GO /* Select shows only keys 1 and 3 added. Key 2 insert failed and was rolled back, but XACT_ABORT was OFF and rest of transaction succeeded. Key 5 insert error with XACT_ABORT ON caused all of the second transaction to roll back. Also note that 'Continue running batch 2...' is not Returned to indicate that the batch is aborted. */ SELECT * FROM t2 GO DROP TABLE t2 DROP TABLE t1 GO Compile and Run-time Errors Compile Errors Compile errors are encountered during syntax checks, security checks, and other general operations to prepare the batch for execution. These errors can prevent the optimization of the query and thus lead to immediate abort. The statement is not run and the batch is aborted. The transaction state is generally left untouched. For example, assume there are four statements in a particular batch. If the third statement has a syntax error, none of the statements in the batch is executed. Optimization Errors Optimization errors would include rare situations where the statement encounters a problem when attempting to build an optimal execution plan. Example: “too many tables referenced in the query” error is reported because a “work table” was added to the plan. Runtime Errors Runtime errors are those that are encountered during the execution of the query. Consider the following batch: SELECT * FROM pubs.dbo.titles UPDATE pubs.dbo.authors SET au_lname = au_lname SELECT * FROM foo UPDATE pubs.dbo.authors SET au_lname = au_lname If you run the above statements in a batch, the first two statements will be executed, the third statement will fail because table foo does not exist, and the batch will terminate. Deferred Name Resolution is the feature that allows this batch to start executing before resolving the object foo. This feature allows SQL Server to delay object resolution and place a “placeholder” in the query’s execution. The object referenced by the placeholder is resolved until the query is executed. In our example, the execution of the statement “SELECT * FROM foo” will trigger another compile process to resolve the name again. This time, error message 208 is returned. Error: 208, Level 16, State 1, Line 1 Invalid object name 'foo'. Message 208 can be encountered as a runtime or compile error depending on whether the Deferred Name Resolution feature is available. In SQL Server 6.5 this would be considered a compile error and on SQL Server 2000 (and SQL Server7.0) as a runtime error due to Deferred Name Resolution. In the following example, if a trigger referenced authors2, the error is detected as SQL Server attempts to execute the trigger. However, under SQL Server 6.5 the create trigger statement fails because authors2 does not exist at compile time. When errors are encountered in a trigger, generally, the statement, batch, and transaction are aborted. You should be able to observe this by running the following script in pubs database: Create table tblTest(iID int) go create trigger trgInsert on tblTest for INSERT as begin select * from authors select * from authors2 select * from titles end go begin tran select 'Before' insert into tblTest values(1) select 'After' go select @@TRANCOUNT go When run in a batch, the statement and the batch are aborted but the transaction remains active. The follow script illustrates this: begin tran select 'Before' select * from authors2 select 'After' go select @@TRANCOUNT go One other factor in a compile versus runtime error is implicit data type conversions. If you were to run the following statements on SQL Server 6.5 and SQL Server 2000 (and SQL Server 7.0): create table tblData(dtData datetime) go select 1 insert into tblData values(12/13/99) go On SQL Server 6.5, you get an error before execution of the batch begins so no statements are executed and the batch is aborted. Error: 206, Level 16, State 2, Line 2 Operand type clash: int is incompatible with datetime On SQL Server 2000, you get the default value (1900-01-01 00:00:00.000) inserted into the table. SQL Server 2000 implicit data type conversion treats this as integer division. The integer division of 12/13/99 is 0, so the default date and time value is inserted, no error returned. To correct the problem on either version is to wrap the date string with quotes. See Bug #56118 (sqlbug_70) for more details about this situation. Another example of a runtime error is a 605 message. Error: 605 Attempt to fetch logical page %S_PGID in database '%.*ls' belongs to object '%.*ls', not to object '%.*ls'. A 605 error is always a runtime error. However, depending on the transaction isolation level, (e.g. using the NOLOCK lock hint), established by the SPID the handling of the error can vary. Specifically, a 605 error is considered an ACCESS error. Errors associated with buffer and page access are found in the 600 series of errors. When the error is encountered, the isolation level of the SPID is examined to determine proper handling based on information or fatal error level. Transaction Error Checking Not all errors cause transactions to automatically rollback. Although it is difficult to determine exactly which errors will rollback transactions and which errors will not, the main idea here is that programmers must perform error checking and handle errors appropriately. Error Handling Raiserror Details Raiserror seems to be a source of confusion but is really rather simple. Raiserror with severity levels of 20 or higher will terminate the connection. Of course, when the connection is terminated a full rollback of any open transaction will immediately be instantiated by the SQL Server (except distributed transaction with DTC involved). Severity levels lower than 20 will simply result in the error message being returned to the client. They do not affect the transaction scope of the connection. Consider the following batch: use pubs begin tran update authors set au_lname = 'smith' raiserror ('This is bad', 19, 1) with log select @@trancount With severity set at 19, the 'select @@trancount' will be executed after the raiserror statement and will return a value of 1. If severity is changed to 20, then the select statement will not run and the connection is broken. Important Error handling must occur not only in T-SQL batches and stored procedures, but also in application program code. Transactions and Triggers (1 of 2) Basic behavior assumes the implicit transactions setting is set to OFF. This behavior makes it possible to identify business logic errors in a trigger, raise an error, rollback the action, and add an audit table entry. Logically, the insert to the audit table cannot take place before the ROLLBACK action and you would not want to build in the audit table insert into every applications error handler that violated the business rule of the trigger. For more information, see also… SQL Server 2000 Books Online topic “Rollbacks in stored procedure and triggers“ (acdata.chm::/ac_8_md_06_4qcz.htm) IMPLICIT_TRANSACTIONS ON Behavior The behavior of firing other triggers on the same table can be tricky. Say you added a trigger that checks the CODE field. Read only versions of the rows contain the code ‘RO’ and read/write versions use ‘RW.’ Whenever someone tries to delete a row with a code ‘RO’ the trigger issues the rollback and logs an audit table entry. However, you also have a second trigger that is responsible for cascading delete operations. One client could issue the delete without implicit transactions on and only the current trigger would execute and then terminate the batch. However, a second client with implicit transactions on could issue the same delete and the secondary trigger would fire. You end up with a situation in which the cascading delete operations can take place (are committed) but the initial row remains in the table because of the rollback operation. None of the delete operations should be allowed but because the transaction scope was restarted because of the implicit transactions setting, they did. Transactions and Triggers (2 of 2) It is extremely difficult to determine the execution state of a trigger when using explicit rollback statements in combination with implicit transactions. The RETURN statement is not allowed to return a value. The only way I have found to set the @@ERROR is using a ‘raiserror’ as the last execution statement in the last trigger to execute. If you modify the example, this following RAISERROR statement will set @@ERROR to 50000: CREATE TRIGGER trgTest on tblTest for INSERT AS BEGIN ROLLBACK INSERT INTO tblAudit VALUES (1) RAISERROR('This is bad', 14,1) END However, this value does not carry over to a secondary trigger for the same table. If you raise an error at the end of the first trigger and then look at @@ERROR in the secondary trigger the @@ERROR remains 0. Carrying Forward an Active/Open Transaction It is possible to exit from a trigger and carry forward an open transaction by issuing a BEGIN TRAN or by setting implicit transaction on and doing INSERT, UPDATE, or DELETE. Warning It is never recommended that a trigger call BEGIN TRANSACTION. By doing this you increment the transaction count. Invalid code logic, not calling commit transaction, can lead to a situation where the transaction count remains elevated upon exit of the trigger. Transaction Count The behavior is better explained by understanding how the server works. It does not matter whether you are in a transaction, when a modification takes place the transaction count is incremented. So, in the simplest form, during the processing of an insert the transaction count is 1. On completion of the insert, the server will commit (and thus decrement the transaction count). If the commit identifies the transaction count has returned to 0, the actual commit processing is completed. Issuing a commit when the transaction count is greater than 1 simply decrements the nested transaction counter. Thus, when we enter a trigger, the transaction count is 1. At the completion of the trigger, the transaction count will be 0 due to the commit issued at the end of the modification statement (insert). In our example, if the connection was already in a transaction and called the second INSERT, since implicit transaction is ON, the transaction count in the trigger will be 2 as long as the ROLLBACK is not executed. At the end of the insert, the commit is again issued to decrement the transaction reference count to 1. However, the value does not return to 0 so the transaction remains open/active. Subsequent triggers are only fired if the transaction count at the end of the trigger remains greater than or equal to 1. The key to continuation of secondary triggers and the batch is the transaction count at the end of a trigger execution. If the trigger that performs a rollback has done an explicit begin transaction or uses implicit transactions, subsequent triggers and the batch will continue. If the transaction count is not 1 or greater, subsequent triggers and the batch will not execute. Warning Forcing the transaction count after issuing a rollback is dangerous because you can easily loose track of your transaction nesting level. When performing an explicit rollback in a trigger, you should immediately issue a return statement to maintain consistent behavior between a connection with and without implicit transaction settings. This will force the trigger(s) and batch to terminate immediately. One of the methods of dealing with this issue is to run ‘SET IMPLICIT_TRANSACTIONS OFF’ as the first statement of any trigger. Other methods may entails checking @@TRANCOUNT at the end of the trigger and continue to COMMIT the transaction as long as @@TRANCOUNT is greater than 1. Examples The following examples are based on this table: create table tbl50000Insert (iID int NOT NULL) go Note If more than one trigger is used, to guarantee the trigger firing sequence, the sp_settriggerorder command should be used. This command is omitted in these examples to simplify the complexity of the statements. First Example In the first example, the second trigger was never fired and the batch, starting with the insert statement, was aborted. Thus, the print statement was never issued. print('Trigger issues rollback - cancels batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran select 'End of trigger', @@TRANCOUNT as 'TRANCOUNT' end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(1) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Second Example The next example shows that since a new transaction is started, the second trigger will be fired and the print statement in the batch will be executed. Note that the insert is rolled back. print('Trigger issues rollback - increases tran count to continue batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(2) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Third Example In the third example, the raiserror statement is used to set the @@ERROR value and the BEGIN TRAN statement is used in the trigger to allow the batch to continue to run. print('Trigger issues rollback - uses raiserror to set @@ERROR') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran -- Increase @@trancount to allow -- batch to continue select @@trancount as ‘Trancount’ raiserror('This is from the trigger', 14,1) end go insert into tbl50000Insert values(3) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert go delete from tbl50000Insert Fourth Example For the fourth example, a second trigger is added to illustrate the fact that @@ERROR value set in the first trigger will not be seen in the second trigger nor will it show up in the batch after the second trigger is fired. print('Trigger issues rollback - uses raiserror to set @@ERROR, not seen in second trigger and cleared in batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback begin tran -- Increase @@trancount to -- allow batch to continue select @@TRANCOUNT as 'Trancount' raiserror('This is from the trigger', 14,1) end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' end go insert into tbl50000Insert values(4) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Lesson 3: Concepts – Locks and Applications This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Explain how lock hints are used and their impact.  Discuss the effect on locking when an application uses Microsoft Transaction Server.  Identify the different kinds of deadlocks including distributed deadlock. Recommended Reading  Charter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  Charter 16 “Query Tuning”, Inside SQL Server 2000 by Kalen Delaney Q239753 – Deadlock Situation Not Detected by SQL Server Q288752 – Blocked SPID Not Participating in Deadlock May Incorrectly be Chosen as victim Locking Hints UPDLOCK If update locks are used instead of shared locks while reading a table, the locks are held until the end of the statement or transaction. UPDLOCK has the advantage of allowing you to read data (without blocking other readers) and update it later with the assurance that the data has not changed since you last read it. READPAST READPAST is an optimizer hint for use with SELECT statements. When this hint is used, SQL Server will read past locked rows. For example, assume table T1 contains a single integer column with the values of 1, 2, 3, 4, and 5. If transaction A changes the value of 3 to 8 but has not yet committed, a SELECT * FROM T1 (READPAST) yields values 1, 2, 4, 5. Tip READPAST only applies to transactions operating at READ COMMITTED isolation and only reads past row-level locks. This lock hint can be used to implement a work queue on a SQL Server table. For example, assume there are many external work requests being thrown into a table and they should be serviced in approximate insertion order but they do not have to be completely FIFO. If you have 4 worker threads consuming work items from the queue they could each pick up a record using read past locking and then delete the entry from the queue and commit when they're done. If they fail, they could rollback, leaving the entry on the queue for the next worker thread to pick up. Caution The READPAST hint is not compatible with HOLDLOCK.  Try This: Using Locking Hints 1. Open a Query Window and connect to the pubs database. 2. Execute the following statements (--Conn 1 is optional to help you keep track of each connection): BEGIN TRANSACTION -- Conn 1 UPDATE titles SET price = price * 0.9 WHERE title_id = 'BU1032' 3. Open a second connection and execute the following statements: SELECT @@lock_timeout -- Conn 2 GO SELECT * FROM titles SELECT * FROM authors 4. Open a third connection and execute the following statements: SET LOCK_TIMEOUT 0 -- Conn 3 SELECT * FROM titles SELECT * FROM authors 5. Open a fourth connection and execute the following statement: SELECT * FROM titles (READPAST) -- Conn 4 WHERE title_ID < 'C' SELECT * FROM authors How many records were returned? 3 6. Open a fifth connection and execute the following statement: SELECT * FROM titles (NOLOCK) -- Conn 5 WHERE title_ID 0 the lock manager also checks for deadlocks every time a SPID gets blocked. So a single deadlock will trigger 20 seconds of more immediate deadlock detection, but if no additional deadlocks occur in that 20 seconds, the lock manager no longer checks for deadlocks at each block and detection again only happens every 5 seconds. Although normally not needed, you may use trace flag -T1205 to trace the deadlock detection process. Note Please note the distinction between application lock and other locks’ deadlock detection. For application lock, we do not rollback the transaction of the deadlock victim but simply return a -3 to sp_getapplock, which the application needs to handle itself. Deadlock Resolution How is a deadlock resolved? SQL Server picks one of the connections as a deadlock victim. The victim is chosen based on either which is the least expensive transaction (calculated using the number and size of the log records) to roll back or in which process “SET DEADLOCK_PRIORITY LOW” is specified. The victim’s transaction is rolled back, held locks are released, and SQL Server sends error 1205 to the victim’s client application to notify it that it was chosen as a victim. The other process can then obtain access to the resource it was waiting on and continue. Error 1205: Your transaction (process ID #%d) was deadlocked with another process and has been chosen as the deadlock victim. Rerun your transaction. Symptoms of deadlocking Error 1205 usually is not written to the SQL Server errorlog. Unfortunately, you cannot use sp_altermessage to cause 1205 to be written to the errorlog. If the client application does not capture and display error 1205, some of the symptoms of deadlock occurring are:  Clients complain of mysteriously canceled queries when using certain features of an application.  May be accompanied by excessive blocking. Lock contention increases the chances that a deadlock will occur. Triggers and Deadlock Triggers promote the deadlock priority of the SPID for the life of the trigger execution when the DEADLOCK PRIORITY is not set to low. When a statement in a trigger causes a deadlock to occur, the SPID executing the trigger is given preferential treatment and will not become the victim. Warning Bug 235794 is filed against SQL Server 2000 where a blocked SPID that is not a participant of a deadlock may incorrectly be chosen as a deadlock victim if the SPID is blocked by one of the deadlock participants and the SPID has the least amount of transaction logging. See KB article Q288752: “Blocked Spid Not Participating in Deadlock May Incorrectly be Chosen as victim” for more information. Distributed Deadlock – Scenario 1 Distributed Deadlocks The term distributed deadlock is ambiguous. There are many types of distributed deadlocks. Scenario 1 Client application opens connection A, begins a transaction, acquires some locks, opens connection B, connection B gets blocked by A but the application is designed to not commit A’s transaction until B completes. Note SQL Server has no way of knowing that connection A is somehow dependent on B – they are two distinct connections with two distinct transactions. This situation is discussed in scenario #4 in “Q224453 INF: Understanding and Resolving SQL Server 7.0 Blocking Problems”. Distributed Deadlock – Scenario 2 Scenario 2 Distributed deadlock involving bound connections. Two connections can be bound into a single transaction context with sp_getbindtoken/sp_bindsession or via DTC. Spid 60 enlists in a transaction with spid 61. A third spid 62 is blocked by spid 60, but spid 61 is blocked by spid 62. Because they are doing work in the same transaction, spid 60 cannot commit until spid 61 finishes his work, but spid 61 is blocked by 62 who is blocked by 60. This scenario is described in article “Q239753 - Deadlock Situation Not Detected by SQL Server.” Note SQL Server 6.5 and 7.0 do not detect this deadlock. The SQL Server 2000 deadlock detection algorithm has been enhanced to detect this type of distributed deadlock. The diagram in the slide illustrates this situation. Resources locked by a spid are below that spid (in a box). Arrows indicate blocking and are drawn from the blocked spid to the resource that the spid requires. A circle represents a transaction; spids in the same transaction are shown in the same circle. Distributed Deadlock – Scenario 3 Scenario 3 Distributed deadlock involving linked servers or server-to-server RPC. Spid 60 on Server 1 executes a stored procedure on Server 2 via linked server. This stored procedure does a loopback linked server query against a table on Server 1, and this connection is blocked by a lock held by Spid 60. Note No version of SQL Server is currently designed to detect this distributed deadlock. Lesson 4: Information Collection and Analysis This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Identify specific information needed for troubleshooting issues.  Locate and collect information needed for troubleshooting issues.  Analyze output of DBCC Inputbuffer, DBCC PSS, and DBCC Page commands.  Review information collected from master.dbo.sysprocesses table.  Review information collected from master.dbo.syslockinfo table.  Review output of sp_who, sp_who2, sp_lock.  Analyze Profiler log for query usage pattern.  Review output of trace flags to help troubleshoot deadlocks. Recommended Reading Q244455 - INF: Definition of Sysprocesses Waittype and Lastwaittype Fields Q244456 - INF: Description of DBCC PSS Command for SQL Server 7.0 Q271509 - INF: How to Monitor SQL Server 2000 Blocking Q251004 - How to Monitor SQL Server 7.0 Blocking Q224453 - Understanding and Resolving SQL Server 7.0 Blocking Problem Q282749 – BUG: Deadlock information reported with SQL Server 2000 Profiler Locking and Blocking  Try This: Examine Blocked Processes 1. Open a Query Window and connect to the pubs database. Execute the following statements: BEGIN TRAN -- connection 1 UPDATE titles SET price = price + 1 2. Open another connection and execute the following statement: SELECT * FROM titles-- connection 2 3. Open a third connection and execute sp_who; note the process id (spid) of the blocked process. (Connection 3) 4. In the same connection, execute the following: SELECT spid, cmd, waittype FROM master..sysprocesses WHERE waittype 0 -- connection 3 5. Do not close any of the connections! What was the wait type of the blocked process?  Try This: Look at locks held Assumes all your connections are still open from the previous exercise. • Execute sp_lock -- Connection 3 What locks is the process from the previous example holding? Make sure you run ROLLBACK TRAN in Connection 1 to clean up your transaction. Collecting Information See Module 2 for more about how to gather this information using various tools. Recognizing Blocking Problems How to Recognize Blocking Problems  Users complain about poor performance at a certain time of day, or after a certain number of users connect.  SELECT * FROM sysprocesses or sp_who2 shows non-zero values in the blocked or BlkBy column.  More severe blocking incidents will have long blocking chains or large sysprocesses.waittime values for blocked spids.  Possibl
湖 南 大 学 信息科学与工程学院 实 验 报 告 "实验名称 "香农编码 " "课程名称 "信息论与编码 " " " " 1、实验目的 (1)进一步熟悉Shannon编码算法; (2)掌握C语言程序设计和调试过程中数值的进制转换、数值与字符串之间的转换等技 术。 2、实验要求 (1)输入:信源符号个数q、信源的概率分布p; (2)输出:每个信源符号对应的Shannon编码的码字。 3、Shannon编码算法 1:procedure SHANNON(q,{}) 2: 降序排列{} 3: for i=1 q do 4: F() 5: 6:将累加概率F()(十进制小数)变换成二进制小数。 7:取小数点后个二进制数字作为第i个消息的码字。 8:end for 9:end procedure ---------------------------------------------------------------------------- -------------------------------------- 调试过程 1、fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory fatal error C1083: Cannot open include file: 'values.h': No such file or directory 原因:unistd.h和values.h是Unix操作系统下所使用的头文件 纠错:删去即可 2、error C2144: syntax error : missing ')' before type 'int' error C2064: term does not evaluate to a function 原因:l_i(int *)calloc(n,sizeof(int)); l_i后缺少赋值符号使之不能通过编译 纠错:添加上赋值符号 error C2018: unknown character '0xa1' 原因:有不能被识别的符号 纠错:在错误处将不能识别的符号改为符合C语言规范的符号 error C2021: expected exponent value, not ' ' 原因:if(fabs(sum-1.0)>DELTA); 这一行中DELTA宏定义不正确 纠错:# define DELTA 0.000001 error C2143: syntax error : missing ';' before '}' 原因:少写了";"号 纠错:在对应位置添加上";"号 参考代码 # include # include # include # include # define DELTA 0.000001/*精度*/ void sort(float*,int);/*排序*/ int main(void) { register int i,j; int n; /*符号个数*/ int temp;/*中间变量*/ float *p_i; /*符号的概率*/ float *P_i; /*累加概率*/ int *l_i; /*码长*/ char * *C; /*码集合*/ /*用sum来检验数据,用p来缓存了中间数据*/ float sum,p; /*输入符号数*/ fscanf(stdin,"%d",&n); /*分配内存地址 */ p_i=(float *)calloc(n,sizeof(float)); P_i=(float *)calloc(n,sizeof(float)); l_i=(int *)calloc(n,sizeof(int)); /* 存储信道传输的概率*/ for(i=0;iDELTA) fprintf(stderr,"Invalid input data \n"); fprintf(stdout,"Starting…\n\n"); /*以降序排列概率*/ sort (p_i,n); /*计算每个符号的码长*/ for(i=0;i
关于雷达方面的知识! EFFECTIVENESS OF EXTRACTING WATER SURFACE SLOPES FROM LIDAR DATA WITHIN THE ACTIVE CHANNEL: SANDY RIVER, OREGON, USA by JOHN THOMAS ENGLISH A THESIS Presented to the Department of Geography and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Master of Science March 2009 11 "Effectiveness of Extracting Water Surface Slopes from LiDAR Data within the Active Channel: Sandy River, Oregon, USA," a thesis prepared by John Thomas English in partial fulfillment of the requirements for the Master of Science degree in the Department of Geography. This thesis has been approved and accepted by: Date Committee in Charge: W. Andrew Marcus, Chair Patricia F. McDowell Accepted by: Dean of the Graduate School © 2009 John Thomas English 111 IV An Abstract of the Thesis of John Thomas English in the Department of Geography for the degree of to be taken Master of Science March 2009 Title: EFFECTIVENESS OF EXTRACTING WATER SURFACE SLOPES FROM LIDAR DATA WITHIN THE ACTIVE CHANNEL: SANDY RIVER, OREGON, USA Approved: _ W. Andrew Marcus This paper examines the capability ofLiDAR data to accurately map river water surface slopes in three reaches of the Sandy River, Oregon, USA. LiDAR data were compared with field measurements to evaluate accuracies and determine how water surface roughness and point density affect LiDAR measurements. Results show that LiDAR derived water surface slopes were accurate to within 0.0047,0.0025, and 0.0014 slope, with adjusted R2 values of 0.35, 0.47, and 0.76 for horizontal intervals of 5, 10, and 20m, respectively. Additionally, results show LiDAR provides greater data density where water surfaces are broken. This study provides conclusive evidence supporting use ofLiDAR to measure water surface slopes of channels with accuracies similar to field based approaches. CURRICULUM VITAE NAME OF AUTHOR: John Thomas English PLACE OF BIRTH: Eugene, Oregon DATE OF BIRTH: January 1st, 1980 GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene, Oregon Southern Oregon University, Ashland, Oregon DEGREES AWARDED: Master of Science, Geography, March 2009, University of Oregon Bachelor of Science, Geography, 2001, Southern Oregon University AREAS OF SPECIAL INTEREST: Fluvial Geomorphology Remote Sensing PROFESSIONAL EXPERIENCE: LiDAR Database Coordinator, Oregon Department of Geology & Mineral Industries, June 2008 - present. LiDAR & Remote Sensing Specialist, Sky Research Inc., 2003 - 2008 GRANTS, AWARDS AND HONORS: Gamma Theta Upsilon Geographic Society Member, 2006 Gradutate Teaching Fellowship, Social Science Instructional Laboratory, 20062007 v VI ACKNOWLEDGMENTS I wish to express special thanks to Professors W.A. Marcus and Patricia McDowell for their assistance in the preparation of this manuscript. In addition, special thanks are due to Mr. Paul Blanton who assisted with field data collection for this project. I also thank the members ofmy family who have been encouraging and supportive during the entirety of my graduate schooling. I wish to thank my parents Thomas and Nancy English for always being proud of me. Special thanks to my son Finn for always making me smile. Lastly, special thanks to my wife Kathryn for her unwavering support, love, and encouragement. Dedicated to my mother Bonita Claire English (1950-2004). Vll V111 TABLE OF CONTENTS Chapter Page I. INTRODUCTION 1 II. BACKGROlTND 5 Water Surface Slope 5 LiDAR Measurements of Active Channel Features 7 III. STUDY AREA 10 IV. METHODS 22 Overview 22 LiDAR Data and Image Acquisition 23 Field Data Acquisition 24 LiDAR Processing 25 Calculation of Water Surface Slopes 27 Evaluating LiDAR Slope Accuracies and Controls 33 V. RESULTS 35 Comparison of Absolute Elevations from Field and LiDAR Data in Reach 1 35 Slope Comparisons 41 Surface Roughness Analysis 46 VI. DiSCUSSiON 51 VII. CONCLUSION 57 APPENDIX: ARCGIS VBA SCRIPT CODE 58 REFERENCES 106 IX LIST OF FIGURES Figure Page 1. Return Factor vs. LiDAR Scan Angle 2 2. Angle of Incidence 3 3. Wave Action Relationship to LiDAR Echo 3 4. Site Map 11 5. Annual Hydrograph of Sandy River 13 6. Oregon GAP Vegetation within Study Area 15 7. Photo of Himalayan Blackberry on Sandy River 16 8. Reach 1 Site Area Map with photo 18 9. Reach 2 Site Area Map 20 10. Reach 3 Site Area Map 21 11. LiDAR Point Filtering Processing Step 26 12. Field DEM Interpolated using Kriging 29 13. Reach 1 LiDAR Cross Sections and Sample Point Location 31 14. Differences Between LiDAR and Field Based Elevations 37 15. Regression ofLiDAR and Field Cross section Elevations 38 16. Comparison of LiDAR and Field Longitudinal Profiles (5, 10,20 meters) 40 17. Regression ofField and LiDAR Based Slopes (5, 10,20 meters) 42 18. Differences Between LiDAR and Field Based Slopes (5, 10,20 meters) 44 19. Relationship of Water Surfaces to LiDAR Point Density 47 20. Marmot Dam: Orthophotographyand Colorized Slope Model 50 21. LiDAR Point Density versus Interpolation 53 LIST OF TABLES T~k p~ 1. Reported Accuracies of 2006 and 2007 LiDAR 24 2. Results of LiDAR and Field Elevation Comparison 38 3. Results ofLiDAR and Field Slope Comparison (5, 10,20 meters) 45 4. Results of Reach 1 Slope Comparison 46 5. Water Surface Roughness Results for Reach 1,2, and 3 48 6. Results of Reach 1 Water Surface Roughness Comparison 49 7. Subset of Reach 3 Water Surface Roughness Analysis Near Marmot Dam 50 x 1 CHAPTER I INTRODUCTION LiDAR (Light Detection and Ranging) has become a common tool for mapping and documenting floodplain environments by supplying individual point elevations and accurate Digital Terrain Models (DTM) (Bowen & Waltermire, 2002; Gilvear et aI., 2004; Glenn et aI., 2005; Magid et aI., 2005; Thoma, 2005; Smith et aI., 2006; Gangodagamage et aI., 2007). Active channel characteristics that have been extracted using LiDAR include bank profiles, longitudinal profiles (Magid et aI., 2005; Cavalli et aI., 2007) and transverse profiles of gullies under forest canopies (James et aI., 2007). To date, however, no one has tested if LiDAR returns from water surfaces can be used to measure local water surface slopes within the active channel. Much of the reason that researchers have not attempted to measure water surface slopes with LiDAR is because most LiDAR pulses are absorbed or not returned from the water surface. However, where the angle of incidence is close to nadir (i.e. the LiDAR pulse is fired near perpendicular to water surface plane), light is reflected and provides elevations off the water surface (Figure 1, Maslov et aI., 2000). Where LiDAR pulses glance the water surface at angles of incidence greater than 53 degrees, a LiDAR pulse is 2 more often lost to refraction (Figure 2) (Jenkins, 1957). In broken water surface conditions the water surface plane is angled, which produces perpendicular angles of incidence allowing for greater chance of return (Maslov et al. 2000). Su et al. (2007) documented this concept by examining LiDAR returns off disturbed surfaces in a controlled lab setting (Figure 3). LiDAR returns off the water surface potentially provide accurate surface elevations that can be used to calculate surface slopes. 1.0 08 ~ 0.6 o t5 ~ E .2 ~ 04 02 00 000 __d=2° d=10 ° --d=200 --d=300 d=40o d=50o I I 2000 4000 60.00 sensing angle, degree I 8000 Figure 1. Return Factor vs. LiDAR Scan Angle. Figure shows relationship between water surface return and scan angle. Return Factor versus sensing angle at different levels of the waving d (d = scan angle). Figure shows the relationship of scan angle of LiDAR to return from a water surface. Return factor is greatest at low scan angles relative to the nadir region of scan. (Maslov, D. V. et. al. (2000). A Shore-based LiDAR for Coastal Seawater Monitoring. Proceedings ofEARSeL-SIGWorkshop, Figure 1, pg. 47). 3 reflected\\ :.;/ incident 1 I 1 . '\ I lAIR \ •••••••• ••••••••••••• •••••• ••••••••••••••••••••• • •• eo ••••••••••• o •••••••••••• _0 •••••••••• 0 ••• .•.•.•.•.•.•00 ,••••• ' 0•••• 0 ••••••••••• 0 ••I' .•.•.•.•.•.,................. .".0 ••••••••••••• , •••••••••••• , ••••••••••0••••. .....................................~ . ••••••••••••••••••••••••••••••••••••• • •••••••••••••••••••••••••• 0 •••••••••••••••••••• 0 ••••• 0 •• ~~~)}))}))})))))))))\..)}))?()))))))))))))))))j((~j< Figure 2. Angle of Incidence. Figure displays concept of reflection and refraction of light according to angle of incidence. The intensity of light is greater as the angle of incidence approaches nadir. (Jenkins, F.A., White, RE. "Fundamentals of Optics". McGraw-Hili, 1957, Chapter 25) 09 08 0.7 0.6 0.5 0.4 0.3 0.2 0.1 r - 0.\ O,j/6Y3- -500 17.5 35 52.5 70 horizonral scanning dislancC(lllm) 0.9 0.8 0.7 06 0.5 0.4 0.3 0.2 0.1 a b Figure 3. Wave Action Relationship to LiDAR Echo. "LiDAR measurements of wake profiles generated by propeller at 6000 rpm (a) and 8000 rpm (b). Su's work definitively showed LiDAR's ability to measure water surfaces, and the relationship of wave action to capability of echo. From Su (2007) figure 5, p.844 . This study examines whether LiDAR can accurately measure water surface elevations and slopes. In order to address this topic, I assess the vertical accuracy of LiDAR and the effects of water surface roughness on LiDAR within the active channel. Findings shed light on the utility of LiDAR for measuring water surface slopes in different stream environments and methodological constraints to using LiDAR for this purpose. 4 5 CHAPTER II BACKGROlJND Water Surface Slope Water surface slope is a significant component to many equations for modeling hydraulics, sediment transport, and fluvial geomorphic processes (Knighton, 1999, Sing & Zang, in press). Traditional methods for measuring water surface slope include both direct and indirect methods. Direct water surface slope measurements typically use a device such as a total station or theodolite in combination with a stadia rod or drop line to measure water surface elevations (Harrelson, et ai., 1994, Western et ai., 1997). Inaccuracies in measurements stem from surface turbulence that makes it difficult to precisely locate the water surface, especially in fast water where flows pile up against the measuring device (Halwas, 2002). Direct survey methods often require a field team to occupy several known points throughout a reach. This is a time consuming process, especially if one wanted to document water surface slope along large portions of a river. This method can be dangerous in deep or fast water. 6 Indirect methods of water surface slope measurement consist of acquiring approximate water surface elevations using strand lines, water marks, secondary data sources such as contours from topographic maps, or hydraulic modeling to back calculate the water depth (USACE, 1993; Western et aI., 1997). Variable quality of data and modeling errors can lead to inaccuracies using these methods. The use of strand lines and water marks may not necessarily represent the peak flows or the water surface. Contours may be calculated or interpolated from survey points taken outside the channel area. The most commonly used hydraulic models are based on reconstruction of I-dimensional flow within the channel and do not account for channel variability between cross section locations. LiDAR water surface returns have a great deal of promise for improving measurement of water surfaces in several significant ways. LiDAR measurements eliminate hazards associated with surveyors being in the water. LiDAR also captures an immense amount of elevation data over a very short period of time, with hundreds of thousands of pulses collected within a few seconds for a single swath. Within this mass of pulses, hundreds or thousands of measurements off the water's surface may be collected depending on the nature of surface roughness, with broken water surfaces increasing the likelihood of measurements (Figure 3). In addition, most terrestrial LiDAR surveys collect data by flying multiple overlapping flight lines, thus increasing the number of returns in off nadir overlapping areas and the potential for returns from water surfaces. 7 The accuracy of high quality LiDAR measurements is comparable to field techniques. The relative variability of quality LiDAR vertical measurements typically ranges between 0.03-0.05 meters (Leica, 2007), where relative variability is the total range of vertical error within an individual scan on surface of consistent elevation. Lastly, LiDAR has the ability to collect water surface elevations over large stretches of river within a single flight of a few hours. LiDAR Measurements of Active Channel Features Recent studies evaluating the utility of LiDAR in the active channel environment have documented the effectiveness of using LiDAR DTMs to extract bank profiles. Magid et al. (2005) examined long term changes of longitudinal profiles along the Colorado River in the Grand Canyon. The study used historical survey data from 1923 and differenced topographic elevations with LiDAR data flown in 2000. LiDAR with three meter spot spacing was used to estimate water surface profiles based on the LiDAR elevations nearest to the known channel. Cavalli et al. (2007) extracted longitudinal profiles of the exposed bed of the Rio Cordon, Italy using 0.5 meter LiDAR DEM cells. This study successfully attributed LiDAR DEM roughness within the channel to instream habitats. Bowen and Waltermire (2002) found that LiDAR elevations within the floodplain were less accurate than advertised by vendors and sensor manufacturers. Dense vegetation within the riparian area prevented LiDAR pulses from reaching the 8 ground surface resulting in accuracies ranging 1-2 meters. Accuracies within unvegetated areas and flat surfaces met vendor specifications (l5-20cm). James et al. (2007) used LiDAR at 3 meter spot spacing to map transverse profiles of gullies under forest canopies. Results from this study showed that gully morphologies were underestimated by LiDAR data, possibly due to low density point spacing and biased filtering of the bare earth model. Today, point densities of 4-8 points/m2 are common and would likely alleviate some of the troubles found in this study. Additional studies have used LiDAR to extract geomorphic data from channel areas. Schumann et al. (2008) compared a variety of remotely sensed elevation models for floodplain mapping. The study used 2 meter LiDAR DEMs as topographic base data for floodplain modeling, and found that modeled flood stages based on the LiDAR DEM were accurate to within 0.35m. Ruesser and Bierman (2007) used high resolution LiDAR data to calculate erosion fluxes between strath terraces based on elevation. Gangodagamage et al. (2007) used LiDAR to extract river corridor width series, which help to quantify processes involved in valley formation. This study used a fixed water surface elevation and did not attempt to demonstrate the accuracy of LiDAR derived water surfaces. Green LiDAR also has been used to examine riverine environments. Green LiDAR functions much like terrestrial LiDAR (which uses an infrared laser) except that green LiDAR systems use green light that has the ability to penetrate the water surface and measure the elevation of the channel bed. Green LiDAR is far less common than terrestrial LiDAR and the majority of studies have been centered on studies of ocean shorelines. Wang and Philpot (2007) assessed attenuation parameters for measuring bathymetry in near shore shallow water, concluding that quality bathymetric models can be achieved through a number of post-processing steps. Hilldale and Raft (2007) assessed the accuracy and precision of bathymetric LiDAR and concluded that although the resulting models were informative, bathymetric LiDAR was less precise than traditional survey methods. In general, it is often difficult to assess the accuracy of bathymetric LiDAR given issues related to access of the channel bed at time of flight. 9 10 CHAPTER III STUDY AREA The study area is the Sandy River, Oregon, which flows from the western slopes ofMount Hood northwest to the Columbia River (Figure 4). Recent LiDAR data and aerial photography capture the variety of water surface characteristics in the Sandy River, which range from shooting flow to wide pool-riffle formations. The recent removal of the large run-of-river Marmot Dam upstream of the analysis sites has also generated interest in the river's hydraulics and geomorphology. 11 545000 ,·......,c' 550000 556000 560000 Washington, I 565000 -. Portland Sandy River .Eugene Oregon 570000 ooo '~" ooo ~ ooo~ • Gresham (""IIIII/hill /flIt'r Oregon Clack. fna County Marmot Dam IHillshaded area represents 2006 LiDAR extent. Ol1hophotography was collected only along the Sandy River channel within the LiDAR extent. 10 KiiomElt:IS t---+---+-~I--+--+----t-+--+---+----jl 545000 550000 555000 560000 565000 570000 Figure 4. Site Map. Site area map showing location of analysis reaches within the 2006 and 2007 LiDAR coverage areas. Olihophotography was also collected for the 2006 study, but was collected only along the Sandy River channel. 12 Floodplain longitudinal slopes along the Sandy River average 0.02 and reach a maximum of 0.04. The Sandy River has closely spaced pool-riffles and rapids in the upper reaches, transitioning to longer sequenced pool-riffle morphology in the middle and lower reaches. The Sandy River bed is dominated by sand. Cobbles and small boulders are present mostly in areas of riffles and rapids. Much of the channel is incised with steep slopes along the channel boundaries. The flow regime is typical of Pacific Northwest streams, with peak flows in the winter months ofNovember through February and in late spring with snowmelt runoff (Figure 5). Low flows occur between late September and early October. The average peak annual flow at the Sandy River station below Bull Run River (USGS 14142500) is 106cms. Average annual low flow for the same gauge is 13.9cms. 13 USGS 14142500 SRNDY RIVER BL~ BULL RUN RIVER, NR BULL RUN, OR 200 k.===_~~~=~~~=.......==",,=~-........==~ ~....J Jan 01Feb Ollar 01Rpr O:t1ay 01Jun 01Jul 01Rug OJSep 010ct 01Nov O:IJec 01 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 \ 11 ~I\\ ,1\ 1\ j\ 1"J'fn I\. I, ) \ , ,;' ) I I" 'I'•., I I' I' ] 30000 ~~-~----~-------------~-------, o ~ 20000 ~ 8'-. 10000 ~ Ql Ql ~ U '001 ~ ::::J U, Ql to 1000 to .= u Co? '001 Cl )- .....J. a: Cl Hedian daily statistic <59 years) Daily nean discharge --- Estinated daily nean discharge Period of approved data Period of provisional data Figure 5, Annual Hydrograph of Sandy River. US Geological Survey gaging station annual hydrograph of Sandy River, Oregon at Bull Run River. Data from http://waterdata.usgs.gov/or/nwis/annual/ Vegetation is mostly a mixture of Douglas fir and western red hemlock (Figure 6). Other vegetation includes palustrine forest found in the upper portions of the study area, and agricultural lands found in the middle and lower portions. Douglas fir and western red hemlock make up 87% of vegetated areas, palustrine forest 5%, and agricultural lands 5%, the remaining 3% is open water associated with the channel and reservoirs (Oregon GAP Analysis Program, 2002). The city of Troutdale, OR abuts the lower reaches of the Sandy River. Along this stretch of river Himalayan blackberry, an invasive species, dominates the western banks (Figure 7). The presence of Himalayan blackberry is significant because LiDAR has trouble penetrating through the dense clusters of vines. When this blackberry is close to the water's edge it is difficult to accurately define the channel boundary. 14 15 545000 550000 555000 560000 565000 570000 Reach 3 10 !' 0° 200 MetersO 0 ~~~~~~I O~~~OOO~ Figure 9. Reach 2 Site Area Map. Site map of Reach 2. Reach 2 contains 359 cross sections derived from LiDAR and 3,456 sample points. Inset map shows cross section sample locations derived from LiDAR and smooth/rough water surface delineations used in analysis. 21 Reach 3 is located 40.7km upstream from the mouth of the Sandy and is 2,815 meters in length (Figure 10). The widest portion of this section at approximate banle full is 88 meters. The upstream extent of the channel includes the supercritical flow of Marmot Dam. The channel is incised and relatively straight with a sinuosity of 1.08. Fine sands dominate the channel bed with some boulders likely present from mass wasting along valley walls. As with Reach 2, Douglas fir dominates bank vegetation along. 200 40) Inset mAp displays UDAR point I densily alol1g willl cross seellon Sanlpleing dala LiDAR cross section SAmple locations were used to eX1mcl poinl density values. 503 fOC I 000 '.1..Hrs 1-.,...--,.-+--=1..,=-,---4I--+-1---11 . Reach 3 Figure 10. Reach 3 Site Area Map. Site map of Reach 3. Inset map shows point LiDAR water surface points. Reach 3 contains 550 cross sections and 3,348 sample points. Visual examination of this map allows one to see how point density varies within the active channel. 22 CHAPTER IV METHODS Overview LiDAR data and orthophotography were collected in 2006 and additional LiDAR data were collected over the same area in 2007. Field measurements were obtained five days after the 2007 LiDAR flight in order to compare field measurements of water surface slope to LiDAR-based measurements. Time of flight field measurements of water surface elevations were not obtained for the 2006 flight, but the coincident collection of LiDAR data and orthophotos provide a basis for evaluating variability of LiDAR-based slopes over different channel types as identified from aerial photos. Following sections provide more detail regarding these methods. 23 LiDAR Data and Image Acquisition All LiDAR data were collected using a Leica ALS50 Phase II LiDAR system mounted on a Cessna Caravan C208 (see Table 1 for LiDAR acquisition specifications). The 2006 LiDAR data were collected October 2211d and encompassed 13,780 hectares of high resolution (2':4 points/m2 ) LiDAR data from the mouth of the Sandy River to Marmot Dam. Fifteen centimeter ground resolution orthophotography was collected September 26th , 2006 along the riparian corridor of the Sandy River from its mouth to just above the former site ofMarmot dam (Figure 4). The 2007 LiDAR were collected on October 8th and covered the same extent as the 2006 flight, but did not include orthophotography. Data included filtered XYZ ASCII point data, LiDAR DEMs as ESRI formatted grids at 0.5 meter cell size. Data were collected at 2':8 points per m2 providing a data set with significantly higher point density than the 2006 LiDAR data. The 2006 LiDAR data were collected in one continuous flight. 2006 orthophotography was collected using an RC30 camera system. Data were delivered in RGB geoTIFF format. LiDAR data were calibrated by the contractor to correct for IMU position errors (pitch, roll, heading, and mirror scale). Quality control points were collected along roads and other permanent flat features for absolute vertical correction of data. Horizontal accuracy ofLiDAR data is governed by flying height above ground with horizontal accuracy being equal to 1I3300th of flight altitude (meters) (Leica, 2007). 24 Table 1. Reported Accuracies of 2006 and 2007 LiDAR. Reported Accuracies and conditions for 2006 and 2007 LiDAR data. (Watershed Sciences PGE LiDAR Delivery Report, 2006, Watershed Sciences DOGAMI LiDAR Delivery Report, 2007). Relative Accuracy is a measure of flight line offsets resulting from sensor calibration. 2006 LiDAR 2007 LiDAR Flying height above ground level meters (AGL) 1100 1000 Absolute Vertical Accuracy in meters 0.063 0.034 Relative Accuracy in meters (calibration) 0.058 0.054 Horizontal Accuracy (l/3300th * AGL) meters 0.37 0.33 Discharge @ time of flight (cms) 13.05 20.8 - 21.8 LiDAR data collection over the Reach 1 field survey location was obtained in a single flight on October 8, 2007 between 1:30 and 6:00 pm. During the LiDAR flight, ground quality control data were collected along roads and other permanent flat surfaces within the collection area. These data were used to adjust for absolute vertical accuracy. Field Data Acquisition A river survey crew was dispatched at the soonest possible date (October 13, 2007) after the 2007 flight to collect ground truth data within the Reach 1. The initial aim was to survey water surface elevations at cross sections of the channel, but the survey was limited to near shore measurements due to high velocity conditions. We collected 187 measurements of bed elevation and depth one to fifteen meters from banks along both sides of the channel (Figure 8a) using standard total station longitudinal profile 25 survey methods (Harrelson, 1994). Seventy-six and 98 measurements were collected along the east and west banks, respectively, at intervals of approximately 1 to 2 meters. Thirteen additional measurements were collected along the east bank at approximately ten meter intervals. Depth measurements were added to bed elevations to derive water surface elevations. Discharge during the survey ranged between 22.5 and 22.7 cms during the survey of the east bank and remained steady at 22.5 cms during the survey of the west bank (USGS station 14142500). LiDAR Processing The goal ofLiDAR processing for this project was to classify LiDAR point data within the active channel as water and output this subset data for further analysis. The LiDAR imagery was first clipped to the active channel using a boundary digitized from the 2006 high resolution orthophotography. LiDAR point data were then reclassified to remove bars, banks, and overhanging vegetation (Figure 11). 26 Figure 11. LiDAR Point Filtering Processing Step. LiDAR processing steps. Top image shows entire LiDAR point cloud clipped to active channel boundary. Lower image shows the final processed LiDAR points representing only those points that reflect off the water surface. All bars and overhanging vegetation have been removed as well. 27 Water points were classified using the ground classification algorithm in Terrascan© (Soininen, 2005) to separate water surface returns from those off of vegetation or other surfaces elevated above the ground. The classification routine uses a proprietary mathematical model to accomplish this task. Once the ground classification was finished, classified points were visually inspected to add or remove false positives and remove in-channel features such as bar islands. A total of 11,593 of 1,854,219 LiDAR points were classified as water. Points classified as water were output as comma delimited x,y,z ASCII text files (XYZ), then converted to a 0.5 meter linearly interpolated ESRI formatted grid using ESRI geoprocessing model script. Calculation of Water Surface Slopes Water surface slopes were calculated using the rise over run dimensionless slope equation where the rise is the vertical difference between upstream and downstream water surface elevations and run is the longitudinal distance between elevation locations. LiDAR data is typically used in grid format. For this reason grid data were used for calculation of water surface slopes. We used linear interpolation to grid the LiDAR point data as this is the standard method used by the LiDAR contractor. In order to compare the LiDAR and field data it was also necessary to interpolate field 28 measurements to create a water surface for the entire stream. The field data-based DEM was created using kriging interpolation within ArcGIS Desktop Spatial Analyst (Figure 12). No quantitative analysis was performed to evaluate the interpolation method of the field-based water surface. The kriging interpolation was chosen because it producex the smoothest water surface based on visual inspection when compared to linear and natural neighbor interpolations, which generated irregular fluctuations that were unrealistic for a water surface. The kriged surface provided a water surface elevation model for comparative analysis with LiDAR. 29 Figure 12. Field DEM Interpolated using Kriging. Field DEM interpolated from field survey points using kriging method found in ArcGIS Spatial Analyst. DEM has been hiIlshaded to show surface characteristics. The very small differences in water surface elevations generate only slight variations in the hillshadeing. To compare LiDAR and field-based water surface slopes, water surface elevations from the LiDAR and field-based DEMS were extracted at the same locations along Reach I. To accomplish this, 37 cross sections were manually constructed at approximately Sm spacings (Figure 13). Cross sections comparisons were used rather than point-to-point comparisons between streamside field and LiDAR data points because the cross sections provide water surface slopes that are more representative of the entire channel. The Sm interval spacing was considered to be a sufficient for fine resolution slope extraction. Because cross section center points were used to calculate the longitudinal distance and because the stream was sinuous, the projection of the cross sections from the center line to the banks led to stream side distances between cross sections that differed from Sm. 30 31 Smooth 125 Meters I 100 I 75 I 50 I 25 I Cross Sections Cross Section Data Roughness Delineation Cross Section Sample Locations _ Rough oI ~ each 1 Figure 13. Reach 1 LiDAR Cross Sections and Sample Point Locations. Reach I LiDAR-derived cross section sample locations and areas of smooth and rough water surface delineations. 37 cross section and 444 sample points lie within Reach 1. 32 Cross sections were extracted using a custom ArcObjects VBA script (Appendix A). This script extracted 1 cell nearest neighbor elevations along the transverse cross sections at 5 meter intervals creating 444 cross section sample locations (Figure 13). Cross section averages were calculated using field-based and LiDAR-based elevation water surface grids. The average cross sectional elevation value for field and LiDAR data were then exported to Excel files, merged with longitudinal distance between cross section, and used to calculate field survey-based and LiDAR-based slopes between cross sections. Reaches 2 and 3, for which only LiDAR data were available, were sampled using the same cross sectional approach used in Reach 1. The data extracted from these reaches were used to characterize how LiDAR-based elevations, slopes and point densities interact with varying water surface roughness. Within Reach 2, 359 cross sections were drawn and elevations were sampled every five meters along each cross section creating 3,456 cross section sample locations (Figure 9). Reach 3 contained 550 cross sections and 3,348 cross section sample locations (Figure 10). Slopes were calculated between each cross section. 33 Evaluating LiDAR Slope Accuracies and Controls The accuracy of elevation data is the major control on slope accuracy, so a comparative analysis was performed using field survey and LiDAR elevations. First, field-based and LiDAR slopes were calculated at distance intervals of five, ten and twenty meters using average cross section elevations to test the sensitivity of the slopes to vertical inaccuracies in the LiDAR data. The field and LiDAR elevations were differenced using the same points used to create average cross section elevations. Differences were plotted in the form of histogram and cumulative frequency plot after transforming them into absolute values. Descriptive statistics were calculated to examine the range, minimum, maximum, and mean offset between data sets. Finally LiDAR and field-based values were compared using regression analysis. This study also examined the effects of water surface roughness on LiDAR elevation measurements, LiDAR point density, and LiDAR derived water surface slopes. Each reach was divided into smooth and rough sections based on visual analysis of the orthophoto data. One-meter resolution slope rasters were created from the LiDAR water surface grids using ArcGIS Spatial Analyst. One meter resolution point density grids were created from LiDAR point data (ArcGIS Spatial Analyst). Using the cross section sample points, values for water surface type, elevation, slope, and point density were extracted within each reach. Point sample data were transferred to tabular format, and average values were generated for each cross section. These tables were used to calculate 34 descriptive statistics associated with water surfaces such as elevation variance, average slope variance, average point density, and average slope. It is assumed in this study that smooth water surfaces are associated with pools and thus ought to have relatively low slopes. Conversely rough water surfaces are assumed to be representative of riffles and rapids, and thus ought to have relatively steeper slopes. Reach 1 contains field data, so slopes from LiDAR and field data were compared with respect to water surface conditions as determined from the aerial photos. 35 CHAPTER V RESULTS Results of this study encompass three analyses. Elevation analysis describes the statistical difference between LiDAR and field-based water surface elevations for Reach 1. Slope analysis compares LiDAR derived and field-based slopes calculated at 5, 10, and 20m longitudinal distances. These analyses aim to quantify both slope accuracy and slope sensitivity. Lastly, water surface analysis examines the relationship between LiDAR measured water surface slopes, point density, and water surface roughness. Comparison of Absolute Elevations from Field and LiDAR Data in Reach 1 The difference between water surface elevations from LiDAR affects the numerator within the rise over run equation, which in tum affects slope. This elevation analysis evaluation quantifies differences between field and LiDAR data. LiDAR-based cross section elevations were differenced from field-based cross section elevations. Difference values were examined through statistical analysis. 36 In terms of absolute elevations relative to sea level, the majority of LiDAR-based water surface elevations were lower than field-based elevations, although the LiDAR elevations were higher in the upper portion ofReach 1. Differences ranged between -0.04 and 0.05m with a mean absolute difference between field and LiDAR elevations of 0.02m (Figure 14 and Table 2). The range of differences is within the expected relative accuracies of LiDAR claimed by the LiDAR provider. Elevations for field and LiDAR data are significantly correlated with an R2 of 0.94 (Figure 15). The negative offset was expected given that discharge at time of LiDAR acquisition was lower than discharge at time of field data acquisition. Discharge during field acquisition ranged between 22.5 and 22.7 cfs, while discharge during LiDAR acquisition was between 20.8 and 21.8cfs. The portion of Reach 1 where LiDAR water surface measurements were higher than field measurements may be related to difference in discharge or change in bed configuration. Overall results showed that LiDAR data and field-based water surface measurements are comparable. 37 Distribution of Elevation Differences Between Field and LiDAR Water Surfaces 10 9 8 7 >. 6 u r:: ell 5 :l C'" ~ 4 u.. 3 2 0+---+ -0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 More Elevation Difference, Field - L1DAR (m) Figure 14. Differences Between LiDAR and Field Based Elevations. Elevation difference statistics between cross sections derived from field and LiDAR elevation data. Positive differences indicate that field-based elevations were higher than LiDAR; negative differences indicate LiDAR elevations were higher. Values on x axis represent minimum difference within range. For example, the 0.01 category includes values ranging from 0.01 to 0.0199. y-1.18x-1.03 .... R2 =0.94 ""..,; I •• ./... ./ .- ./ • ./ • ./. /""I ./iI ../. _._~. -? , 38 Table 2. Results of LiDAR and Field Elevation Comparison. Descriptive and regression statistics for absolute difference lField - LiDARI values between cross section elevations. All units in meters. Sample size is 37. Mean 0.028 Median 0.030 Standard Deviation 0.013 Kurtosis -0.640 Skewness -0.484 Range of difference 0.093 Minimum difference 0.002 Absolute maximum difference 0.047 Confidence Level(95.0%) (m) 0.004 Elevation Comparison of Field and LiDAR Water Surface Elevations 5.72 5.70 ~_ 5.68 g 5.66 :0:; I1l 5.64 > iii 5.62 ell 5.60 () ~ 5.58 ~ 5.56 ~ 5.54 1\1 5.52 ~ IX 5.50 terms of overall shape, the greatest differences occur in the upper 30 m, where LiDAR-based profiles demonstrate a higher slope than do field-based measurements. Because of the five day lag between LiDAR and field measurements in this mobile bed stream, it is impossible to know the degree to which this difference represents error in measurements or real change in the system. 40 5 meter Longitudinal Profile Comparison 20 40 60 80 100 120 140 160 180 5.75 .s 5.70 ~" _ • •• • :. 5 Cll 5.55 • • ~ • • w 5.50 • • • • • • • • • 5.45 5.40 0 20 40 60 80 100 120 140 160 180 Longitudinal Distance Down Stream (m) B 20 meter Longitudinal Profile Comparison 5.75 5.70 • ,. 20 Cll 5.55 •• Q) W 5.50 •• • , 5.45 . 5.40 0 20 40 60 80 100 120 140 160 180 Longitudinal Distance Down Stream (m) C Figure 16. Comparison of LiDAR and Field Longitudinal Profiles (5, 10, 20 meters). Longitudinal profiles of a) 5 meter, b) 10 meter, and c) 20 meter cross section elevations. 41 Slope Comparisons Slope in this study is calculated as the dimensionless ratio of rise over run. As noted in the Methods section, slopes were calculated over three different horizontal intervals to test the sensitivity of the LiDAR's internal relative accuracy. Differences in Sm LiDAR and field-based slopes derived from cross sections reveal substantial scatter (Figure l7a), although they clearly covary. Ten meter interval slopes show a stronger relationship (Figure 17b), while slopes based on cross sections spaced 20 m apart have the strongest relationship (Figure l7c). The slope associated with regression of field and LiDAR elevation data is not approximately 1 as one might expect. This is because LiDAR elevations are higher than field elevations at the upstream end of the reach, and lower at the downstream end. 42 5m Slope Comparison -c: ~ -0:: Q) (/l ~ ~.01 Q) C. .2 en 0:: « 0 ::i A -c: ~ 0:: --Q) (/l i2 -0.01 Q) C. 0 en 0:: « 0 ::i B 0.004 = 0.58x - 0.001 R2 = 0.38 ~.008 -0.008 Field Slope (Rise/Run) 10 meter Slope Comparison 0.004 y = 0.63x - 0.001 R2 = 0.51 -0.008 -0.008 Field Slope (Rise/Run) 20 meter Slope Comparison • 0.004 0.002 0.004 C :::l -0:: Q) (/l i2 ~.01 -Q) c. o Ci5 0:: « o~ 0.004 =0.66x - 0.001 R2 = 0.80 ~.008 ~.006 -0.008 Field Slope (Rise/Run) 0.002 0.004 C Figure 17. Regression of Field and LiDAR Based Slopes (5,10,20 meters). Scatter plots showing comparisons between slope values calculated at distance intervals of a) 5 meters, b) 10 meters, and c) 20 meters. 43 Figure 18 shows how the range of differences between LiDAR and field-based water surface slopes decrease as longitudinal distance increases. Five meter slope differences ranged between -0.004 and 0.004 (Figure 18a). Ten meter slope differences ranged between -0.002 and 0.003 (Figure 18b). Twenty meter slope differences ranged between 0 and 0.002 (Figure 18c). 44 Differences of Slope at 5m Between Field and LiDAR 10 » 8 0c Ql 6 :J 0" 4 .Q..l u. 2 0 SIll>< SIl"> SIll\- ~<::J <;:><::J <;:><::J SIl" ~ SIl" SIll\- SIl"> SIll>< ~/l, r;:,<::J ~'::; ~'::; ~'::; ~'::; ~o Slope Difference (Field-LiDAR) A Differences of Slope at 10m Between Field and L1DAR 7 6 ~ 5 lii 4 :J 0" 3 ~ u. 2 1 o +---+--~--;..J SIll>< ~<::J Slope Difference (Field-LiDAR) B Differences of Slope at 20m Between Field and LiDAR 4 ~~I\- ~~" ~ ~~" ~~I\- ~~"> ~~I>< o"/l, <;:>.~. ~.~.~.~. ~ Slope Difference (Field-LiDAR) o +---+--+--+--t- SIll>< SIl"> <;:><::J <;:><::J ~ 3 c Ql :J 2 0" ~ U. C Figure 18. Differences Between LiDAR and Field Based Slopes (5, 10,20 meters). Histogram charts showing difference values between field and LiDAR derived slopes at a) 5 meter slope distances, b) 10 meter slope distances, and c) 20 meter slope distances. 45 The mean difference between slopes decreases from 0.0017 to 0.0007 as slope distance interval is increased. Maximum slope difference and standard deviation of offsets decrease from 0.001 to 0.0005 and 0.0047 to 0.0014 respectively. Regression analysis of these data show a significant relationship for all three comparisons, and adjusted R2 increased from 0.357 to 0.763 with slope distance interval (Table 3). Table 3. Results of LiDAR and Field Slope Comparison (5, 10,20 meters). Descriptive and regression statistics for offsets between field and LiDAR derived slope values (Field minus LiDAR). Slope values are dimensionless rise / run. All data is significant at 0.01. Distance Interval 5m 10m 20m Mean 0.0017 0.0012 0.0007 Standard Deviation 0.0010 0.0007 0.0005 Range of Difference 0.0080 0.0047 0.0024 Minimum difference 0.0000 0.0000 0.0001 Maximum difference 0.0047 0.0026 0.0015 Count 36 16 8 Adjusted R squared 0.36 0.47 0.76 Water surface slope for the entire length of Reach 1 (l59.32m) was compared and yielded a difference of 0.0005. This difference is smaller (by 0.0002) than the difference between 20 meter slope (Table 4). Slope was calculated by differencing the most upstream and downstream cross sections and dividing by total length of reach. Differences between LiDAR and field-based slopes may represent real change due to the five day lag between data sets and difference in discharge. 46 Table 4. Results of Reach 1 Slope Comparison. Comparison of slopes calculated using the farthest upstream and downstream cross section elevation values. Slope values have dimensionless units stemming from rise over run. Upper Lower Reach Elevation (m) Elevation (m) Len2th (m) Slope Field 5.652 5.491 159.32 -0.0010 LiDAR 5.697 5.455 159.32 -0.0015 Surface Roughness Analysis Water surface condition was characterized as smooth or rough based on 2006 aerial photography (Figure 19). Surface roughness was examined to understand its effect on LiDAR data within the active channel, as well as LiDAR's ability to potentially capture difference in water surface turbulence. Table 5 shows statistics with relation to water surface condition for all three reaches. 47 Figure 19. Relationship of Water Surfaces to LiDAR Point Density. 2006 aerial photos were used to delineate rough and smooth water surfaces. Image on left shows a transition between rough water surface (seen as white water) and smooth water surface (seen as upstream pool). Image on right shows LiDAR point density in points per square meter. In all reaches point density, variance of elevations, and water surface slopes were significantly higher in rough surface conditions. These results indicate that LiDAR point density is directly related to the roughness of a water surface and that is capturing the rough water characteristics one would expect in areas where turbulence generates surface waves. 48 Table 5. Water Surface Roughness Results for Reach 1,2, and 3. Water surface statistical output for rough and smooth water surface of Reaches 1, 2, and 3. Results within table represent average values for each Reach. Slope values have dimensionless units from rise over run equation derived from ESRI generated slope grid. Point density values based on points/m2 • Elevation variance in meters. Reach 1 Reach 2 Reach 3 Rou~h water No. of Sample Points 153 1981 1968 Avg Slope -0.013 -0.011 -0.007 Point Density (pts/mL ) 1.195 1.002 1.217 Elevation Variance (m) 0.003 0.018 0.041 Smooth water No. of Sample Points 290 1474 1378 Avg Slope 0.0075 -0.0006 -0.0033 Point Density (pts/mL ) 0.149 0.550 0.480 Elevation Variance (m) 0.001 0.0077 0.024 Within Reach 1, cross section elevations were separated into rough and smooth water conditions and slopes were calculated using field and LiDAR data sets (Table 6). Again, results showed that rough water surfaces have greater slopes than smooth water surfaces. The smooth water surface of Reach 1 yielded a larger discrepancy between field and LiDAR derived slopes compared to rough water surface. This is because small differences between LiDAR and field elevations generate larger proportional error in the rise / run equation when total elevation differences between upstream and downstream are small. 49 Table 6. Results of Reach 1 Water Surface Roughness Comparison. Reach 1 water surface roughness slope analysis. Reach 1 was divided into smooth and rough water surfaces based upon visual characteristics present in aerial photography. Slopes were calculated for each area and compared with field data to examine accuracy. Surface Reach Upper Lower Slope Type Lenl!th (m) Elevation (m) Elevation (m) Slope Difference Field Smooth 83.11 5.652 5.642 -0.0001 N/A LiDAR Smooth 83.11 5.697 5.612 -0.0010 0.0009 Field Rough 71.73 5.635 5.491 -0.0020 N/A LiDAR Rough 71.73 5.592 5.455 -0.0019 -0.0001 Prior to collections of the 2007 data, Reach 3 contained the former Marmot Dam that was dismantled on October 19th , 2007 (Figure 20). The areas at and directly below the dam are rough water surfaces. The super critical flow at the dam yielded a slope of - 0.896 (Table 7). The run below the dam contained low slope values of less than -0.002. Both the dam fall and adjacent run yielded high point densities of greater than 2 points per square meter. 50 Cross Sections o Cross Section Sample Locations L1DAR derived Slope Model Value Higll 178814133 25 50 75 100 125 150 ~.',eters I I I I I I La,·, 0003936 Figure 20. Marmot Dam: Orthophotography and Colorized Slope Model. Mannot Dam at far upstream portion of Reach 3. Image on left shows dam site in 2006 orthophotography. Image on right shows the increase in slope associated with the dam. Marmot Dam was removed Oct. 19th , 2007. Table 7. Subset of Reach 3 Water Surface Roughness Analysis Near Marmot Dam. Subset of Reach 3 immediately surrounding Marmot Dam roughness analysis containing values for Mannot Dam. The roughness results fell within expectations showing increases in slope at the dam fall and high point densities at the dam fall and immediate down stream run. Habitat Type Avg Slope Point Density Point Density Variance Dam Fall -0.896 2.284 1.003 Dam Run -0.001 2.085 5.320 51 CHAPTER VI DISCUSSION The elevation analysis portion of this study shows that LiDAR can provide water surface profiles and slopes that are comparable to field-based data. The differences between LiDAR and field based measurements can be attributed to three potential sources. The first is the relative accuracy of the LiDAR data which has been reported between O.05m and O.06m by the vendor. The second source can be associated with the accuracy of field based measurements which are similar to the relative accuracy of the LiDAR (O.03m-O.05m). Lastly, the discharge differed between field data collection and LiDAR collection by O.02cms. It is possible that much of the O.05m difference observed through most of the Reach 1 profile (Figure 16) could be attributed to the difference in discharge and changes in bed configuration, but without further evidence, the degree of difference due to error or real change cannot be identified. Even if one attributes all the difference to error in LiDAR measurements, the overall correspondence ofLiDAR and field measurement (Figure 15 and 16) indicates that LiDAR-based surveys are useful for many hydrologic applications. 52 In the upper portion of the reach, the profiles display LiDAR elevations that are higher than the field data elevations, whereas the reverse is true at the base of the reach. This could be a function of difference in discharge between datasets, change in bed configuration, or an artifact of low point density. Low density of points forces greater lengths of interpolation between LiDAR points leading to a coarse DEM (Figure 21). Overall, the analysis Reach 1 profile indicates that LiDAR was able to match the fieldbased elevation measurements within ±O.05m. 53 Rough & Smooth Wa~t:e:-r~S~u=rf;:a~c:e:s~rz~~J,;~~ Grid Interpolation in Low Point Density Figure 21. LiDAR Point Density versus Interpolation. Side by side image showing long lines of interpolation associated with smooth water surfaces (right image). Smooth water surfaces tend to have low LiDAR point density. The image on the right shows a hillshade ofthe LiDAR DEM. The DEM has been visualized using a 2 standard deviation stretch to highlight long lines of interpolation. The comparability of LiDAR and field-based slopes showed a significant trend with increasing downstream distances between cross sections. Adjusted R2 values increased from 0.36 to 0.76 and the range of difference between field and LiDAR based slopes decreased from 0.0047 to 0.00 14 as longitudinal distance increased from 5 to 20- 54 m. This suggests that the 0.05m of expected variation of LiDAR derived water surface elevation has less effect on water surface slope accuracy as distance between elevation measurements points increases. Likewise, slopes accuracies along rivers with low gradients will improve as the longitudinal distance between elevation points increases. Overall, data has shown that LiDAR can measure water surface slopes with mean difference relative to field measurements of 0.017, 0.012, and 0.007 at horizontal distances of 5, 10, and 20 meters respectively. Although the discrepancy between field and LiDAR-based slopes is greatest at 5-m intervals, the overall slopes (Fig 17) and longitudinal profiles (Fig 16) even at this distance generally correspond. The use of a 5m interval water surface slope as a basis for comparison is really a worst case example, as water surface slopes are usually measured over longer reach scale distances where the discrepancy between LiDAR and field-based measurements is lower. The continuous channel coverage and accuracies derived from LiDAR represent a new level of accuracy and precision in terms of spatial extent and resolution of water surface slope measurements. Analysis of surface roughness found that rough water surfaces had significantly higher point densities than smooth water surfaces. Rough water surfaces averaged at least 1 point/m2 , while smooth water surfaces averaged less than 1 point/2m2 • Longitudinal profiles of Reach 1 indicate the most accurate water surface measurements occur in areas of higher point density (Fig. 16). Future applications that attempt to use 55 LiDAR to measure water surface slope ought to sample DEM elevations from high point density areas of channel. Water surface analysis also showed trends relating water surface roughness and slope. Rough water surfaces for all three analysis reaches averaged larger average slope values than smooth water surfaces. This is because rough water surfaces are commonly associated with steps, riffles, and rapids. All three of these habitat types are areas have higher slopes than smooth water habitats. Smooth water surfaces are commonly associated with pools or glides, which would be areas of lower slope. Future research should examine the potential for using LiDAR to characterize stream habitats based on in-stream point density and slope. This study is not without its limitations. The field area used to test the accuracy of LiDAR is only representative of a small portion of the Sandy River. Comparisons of field and LiDAR data would be improved by having mid-channel field data. One might also question the use of field based water surface slopes as control for measuring "accuracy". Water surface slope is difficult to measure for reasons stated earlier in this paper. One might make the argument that there is no real way to truly measure LiDAR accuracy of water surface slope, and that LiDAR and field based measurements are simply comparable. In this context, LiDAR holds an advantage over field based measurements given its ability to measure large sections of river in a single day. LiDAR has a distinct advantage over traditional methods of measurement in that measurements are returned from the water surface, and consequently not subject to errors 56 associated with variability of surface turbulence piling up against the measuring device. LiDAR can also capture long stretches of channel within a few seconds reducing the influence of changes in discharge. LiDAR data in general does have its limitations. LiDAR data are only as accurate as the instrumentation and vendor capabilities. LiDAR must be corrected for calibrations and GPS drift to create a reliable data set, and not all LiDAR vendors produce the same level of quality. LiDAR data may be more accurate in some river reaches than others. The study reaches of this study contained well defined open channels, which made identifying LiDAR returns off the water surface possible. Both LiDAR data sets were collected at low flows. Flows that are too low or channels that are too narrow may limit ability to extract water surface elevations because of protruding boulders or dense vegetation that hinders accurate measurements. In some cases vegetation within and adjacent to the channel may interfere with LiDAR's ability to reach the water surface. Researchers should consider flow, channel morphology, and biota when obtaining water surface slopes from LiDAR. 57 CHAPTER VII CONCLUSION This paper examined the ability of LiDAR data to accurately measure water surface slopes. This study has shown that LiDAR data provides sufficiently accurate elevation measurements within the active channel to accurately measure water surface slopes. Measurement of water surface slope with LiDAR provides researchers a tool which is both more efficient and cost effective in comparison with traditional field-based survey methods. Additionally, analysis showed that LiDAR point density is significantly higher in rough surface conditions. Water surface elevations should be gathered from high point density areas as low point density may hinder elevation accuracy. Channel morphology, gradient, flow, and biota should be considered when extracting water surface slopes as these attributes influence water surface measurement. Further study should examine accuracy of LiDAR derived water surface slopes in channel morphologies other than those in this study. Overall, the recognition that LiDAR can accurately measure water surface slopes allows researchers an unprecedented ability to study hydraulic processes for large stretches of river. Common: APPENDIX ARCGIS VBA SCRIPT CODE 58 Public g---.pStrmLayer As ILayer ' stream centerline layer selected by user (for step 1) Public g_StrearnLength As Double ' stream centerline length (for step 1) Public g_InputDistance As Integer 'As Double 'distance entered by user (for step 1) Public g_NumSegments As Integer I number of sample points entered by user (for step 1) Public gyPointLayer As ILayer I point layer created from stream centerline (for step 1) Public g]ntShpF1Name As String I point layer pathname (for step 1) Public gyMouseCursor As IMouseCursor 'mouse cursor Public g_LinearConverson As Double I linear conversion factor Public gyDEMLayer As IRasterLayer I DEM layer (for steps 3 and 4) Public g_DEMConvertUnits As Double I DEM vertical units conversion factor (for steps 3 and 4) Public g_MaxSearchDistance As Double 'maximum search distance (for step 4) Public L NumDirections As Integer I number of directions to search in (for step 4) Public g_SampleDistance As Double 'sample distance (for step 5) Public g_SampleNumber As Double ' total sample points (for step 5) Public g_VegBeginPoint As Boolean I where to start the calucaltion (for step 5) Public g_VegCaclMethod As Boolean 'which method for Vegetation Calculation (for step 5) Public gyContribLayer As ILayer ' contributing point layer (for step 6) Public gyReceivLayer As ILayer 'receiving point layer (for step 6) Public gyOutputLayerName As String I output shapefile (for step 6) Function VerifyField(fLayer As ILayer, fldName As String) As Boolean I verify that topo fields are in the stream centerline point layer Dim pFields As IFields Dim pField As IField Dim pFeatLayer As IFeatureLayer Dim pFeatClass As IFeatureClass Set pFeatLayer = fLayer Set pFeatClass = pFeatLayer.FeatureClass Set pFields = pFeatClass.Fields For i = 0 To pFields.FieldCount - 1 Set pField = pFields.Field(i) 'MsgBox pField.Name IfpField.Name = fldName Then VerifyField = True Exit Function End If Next VerifyField = False End Function Function Ca1cPointLatLong(inPnt As IPoint, inLayer As ILayer) As IPoint , in point layer Dim pFLayer As IFeatureLayer Set pFLayer = inLayer , spatial reference environment Dim pInSpatialRef As ISpatialReference Dim pOutSpatialRef As ISpatialReference Dim pGeoTrans As IGeoTransformation Dim pInGeoDataset As IGeoDataset Set pInGeoDataset = pFLayer Dim pSpatRefFact As ISpatialReferenceFactory , get map units of shapefile spatial reference Dim pPCS As IProjectedCoordinateSystem Set pPCS = pInGeoDataset.SpatialReference 'set spatial reference environment Set pSpatRefFact = New SpatialReferenceEnvironment Set pInSpatialRef= pInGeoDataset.SpatialReference 'MsgBox pInSpatialRef.Name Set pOutSpatialRef= pSpatRefFact.CreateGeographicCoordinateSystem(esriSRGeoCS_WGS1984) Set pGeoTrans = pSpatRefFact.CreateGeoTransformation(esriSRGeoTransformation_NADI983_To_WGS1984_1) Dim pOutGeom As IGeometry2 Set Ca1cPointLatLong = New Point Set CalcPointLatLong.SpatialReference = pInSpatialRef Ca1cPointLatLong.PutCoords inPnt.X, inPnt.Y Set pOutGeom = Ca1cPointLatLong pOutGeom.ProjectEx pOutSpatialRef, esriTransformForward, pGeoTrans, 0, 0, ° 'MsgBox inPnt.X &" "& inPnt.Y & vbCrLf& Ca1cPointLatLong.X &" "& Ca1cPointLatLong.Y End Function Sub OpenGxDialogO Dim pGxdial As IGxDialog Set pGxdial = New GxDialog pGxdial.ButtonCaption = "OK" pGxdial.Title = "Create Stream Centerline Point Shapefile" pGxdial.RememberLocation = True Dim pShapeFileObj As IGxObject Dim pGxFilter As IGxObjectFilter Set pGxFilter = New GxFilterShapefiles 'e.g shp Set pGxdial.ObjectFilter = pGxFilter If pGxdial.DoModaISave(ThisDocument.Parent.hWnd) Then Dim pLocation As IGxFile Dim fn As String 59 Set pLocation = pGxdial.FinalLocation fn = pGxdial.Name End If If Not pLocation Is Nothing Then LPntShpFlName = pLocation.Path & "\" & fn frmlB.tbxShpFileName.Text = g]ntShpFlName frmlB.cmdOK.Enabled = True End If End Sub Function GetAngle(pPolyline As IPolyline, dAlong As Double) As Double Dim pi As Double pi = 4 * Atn(l) Dim dAngle As Double Dim pLine As ILine Set pLine = New Line pPolyline.QueryTangent esriNoExtension, dAlong, False, 1, pLine , convert from radians to degrees dAngle = (180 * pLine.Angle) / pi I adjust angles , ESRI defines 0 degrees as the positive X-axis, increasing counter-clockwise I Ecology references 0 degrees as North, increasing clockwise If dAngle <= 90 Then GetAngle = 90 - dAngle Else GetAngle = 360 - (dAngle - 90) End If End Function Function FeatureExists(strFeatureFileName As String) As Boolean On Error GoTo ErrHandler: Dim pWSF As IWorkspaceFactory Set pWSF = New ShapefileWorkspaceFactory Dim pFeatWS As IFeatureWorksiJace Dim pFeatDS As IFeatureClass Dim strWorkspace As String Dim strFeatDS As String strWorkspace = SplitWorkspaceName(strFeatureFileName) & "\" strFeatDS = SplitFileName(strFeatureFileName) If PWSF.IsWorkspace(strWorkspace) Then Set pFeatWS = pWSF.OpenFromFile(strWorkspace, 0) Set pFeatDS = pFeatWS.OpenFeatureClass(strFeatDS) End If 60 FeatureExists = (Not pFeatDS Is Nothing) Set pWSF =Nothing Set pFeatWS = Nothing Set pFeatDS = Nothing Exit Function ErrHandler: FeatureExists = False End Function 'Returns a Workspace given for example C: \temp\dataset returns C:\temp Function SplitWorkspaceName(sWholeName As String) As String On Error GoTo ERH Dim pos As Integer pos = InStrRev(sWholeName, "\") If pos > 0 Then SplitWorkspaceName = Mid(sWholeName, 1, pos - 1) Else Exit Function End If Exit Function ERH: MsgBox "Workspace Split" & Err.Description End Function 'Returns a filename given for example C:\temp\dataset returns dataset Function SplitFileName(sWholeName As String) As String On Error GoTo ERH Dim pos As Integer Dim sT, sName As String pos = InStrRev(sWholeName, "\") Ifpos > 0 Then sT = Mid(sWholeName, 1, pos - 1) Ifpos = Len(sWholeName) Then Exit Function End If sName = Mid(sWholeName, pos + 1, Len(sWholeName) - Len(sT)) pos = InStr(sName, ".") If pos > 0 Then SplitFileName = Mid(sName, 1, pos - 1) Else SplitFileName = sName End If End If Exit Function ERH: 61 • MsgBox "Workspace Split:" & Err.Description End Function Public Sub BusyMouse(bolBusy As Boolean) 'Subroutine to change mouse cursor If g---'pMouseCursor Is Nothing Then Set g---'pMouseCursor = New MouseCursor End If IfbolBusy Then g---'pMouseCursor.SetCursor 2 Else g---'pMouseCursor.SetCursor 0 End If End Sub Function MakeColor(lRGB As Long) As IRgbColor Set MakeColor =New RgbColor MakeColor.RGB = lRGB End Function Function MakeDecoElement(pMarkerSym As IMarkerSymbol, _ dPos As Double)_ As ISimpleLineDecorationElement Set MakeDecoElement

64,648

社区成员

发帖
与我相关
我的任务
社区描述
C++ 语言相关问题讨论,技术干货分享,前沿动态等
c++ 技术论坛(原bbs)
社区管理员
  • C++ 语言社区
  • encoderlee
  • paschen
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
  1. 请不要发布与C++技术无关的贴子
  2. 请不要发布与技术无关的招聘、广告的帖子
  3. 请尽可能的描述清楚你的问题,如果涉及到代码请尽可能的格式化一下

试试用AI创作助手写篇文章吧