Special Database Object Can't Be Located !

yuhan21 2005-09-28 10:02:04

为什么我在打开数据库的时候 会出现 Special Database Object Can't Be Located !
这是什么原因啊!(以前我是能打开的)
...全文
107 1 打赏 收藏 转发到动态 举报
写回复
用AI写文章
1 条回复
切换为时间正序
请发表友善的回复…
发表回复
china1111 2005-09-28
  • 打赏
  • 举报
回复
说清楚点...
你是在Notes客户端上打开数据库时出现这个问题还是数据库打开后出现这个提示。
如果是数据库未打开出现这个问题,可能是这个数据库已经被删除了。
如果是数据库打开后出现这个提示,你去查看一下数据库资源中的数据库Script是否写了程序来获取其它数据库
Computer Networking: A Top-Down Approach, 6th Edition Solutions to Review Questions and Problems Version Date: May 2012 This document contains the solutions to review questions and problems for the 5th edition of Computer Networking: A Top-Down Approach by Jim Kurose and Keith Ross. These solutions are being made available to instructors ONLY. Please do NOT copy or distribute this document to others (even other instructors). Please do not post any solutions on a publicly-available Web site. We’ll be happy to provide a copy (up-to-date) of this solution manual ourselves to anyone who asks. Acknowledgments: Over the years, several students and colleagues have helped us prepare this solutions manual. Special thanks goes to HongGang Zhang, Rakesh Kumar, Prithula Dhungel, and Vijay Annapureddy. Also thanks to all the readers who have made suggestions and corrected errors. All material © copyright 1996-2012 by J.F. Kurose and K.W. Ross. All rights reserved Chapter 1 Review Questions There is no difference. Throughout this text, the words “host” and “end system” are used interchangeably. End systems include PCs, workstations, Web servers, mail servers, PDAs, Internet-connected game consoles, etc. From Wikipedia: Diplomatic protocol is commonly described as a set of international courtesy rules. These well-established and time-honored rules have made it easier for nations and people to live and work together. Part of protocol has always been the acknowledgment of the hierarchical standing of all present. Protocol rules are based on the principles of civility. Standards are important for protocols so that people can create networking systems and products that interoperate. 1. Dial-up modem over telephone line: home; 2. DSL over telephone line: home or small office; 3. Cable to HFC: home; 4. 100 Mbps switched Ethernet: enterprise; 5. Wifi (802.11): home and enterprise: 6. 3G and 4G: wide-area wireless. HFC bandwidth is shared among the users. On the downstream channel, all packets emanate from a single source, namely, the head end. Thus, there are no collisions in the downstream channel. In most American cities, the current possibilities include: dial-up; DSL; cable modem; fiber-to-the-home. 7. Ethernet LANs have transmission rates of 10 Mbps, 100 Mbps, 1 Gbps and 10 Gbps. 8. Today, Ethernet most commonly runs over twisted-pair copper wire. It also can run over fibers optic links. 9. Dial up modems: up to 56 Kbps, bandwidth is dedicated; ADSL: up to 24 Mbps downstream and 2.5 Mbps upstream, bandwidth is dedicated; HFC, rates up to 42.8 Mbps and upstream rates of up to 30.7 Mbps, bandwidth is shared. FTTH: 2-10Mbps upload; 10-20 Mbps download; bandwidth is not shared. 10. There are two popular wireless Internet access technologies today: Wifi (802.11) In a wireless LAN, wireless users transmit/receive packets to/from an base station (i.e., wireless access point) within a radius of few tens of meters. The base station is typically connected to the wired Internet and thus serves to connect wireless users to the wired network. 3G and 4G wide-area wireless access networks. In these systems, packets are transmitted over the same wireless infrastructure used for cellular telephony, with the base station thus being managed by a telecommunications provider. This provides wireless access to users within a radius of tens of kilometers of the base station. 11. At time t0 the sending host begins to transmit. At time t1 = L/R1, the sending host completes transmission and the entire packet is received at the router (no propagation delay). Because the router has the entire packet at time t1, it can begin to transmit the packet to the receiving host at time t1. At time t2 = t1 + L/R2, the router completes transmission and the entire packet is received at the receiving host (again, no propagation delay). Thus, the end-to-end delay is L/R1 + L/R2. 12. A circuit-switched network can guarantee a certain amount of end-to-end bandwidth for the duration of a call. Most packet-switched networks today (including the Internet) cannot make any end-to-end guarantees for bandwidth. FDM requires sophisticated analog hardware to shift signal into appropriate frequency bands. 13. a) 2 users can be supported because each user requires half of the link bandwidth. b) Since each user requires 1Mbps when transmitting, if two or fewer users transmit simultaneously, a maximum of 2Mbps will be required. Since the available bandwidth of the shared link is 2Mbps, there will be no queuing delay before the link. Whereas, if three users transmit simultaneously, the bandwidth required will be 3Mbps which is more than the available bandwidth of the shared link. In this case, there will be queuing delay before the link. c) Probability that a given user is transmitting = 0.2 d) Probability that all three users are transmitting simultaneously = = (0.2)3 = 0.008. Since the queue grows when all the users are transmitting, the fraction of time during which the queue grows (which is equal to the probability that all three users are transmitting simultaneously) is 0.008. 14. If the two ISPs do not peer with each other, then when they send traffic to each other they have to send the traffic through a provider ISP (intermediary), to which they have to pay for carrying the traffic. By peering with each other directly, the two ISPs can reduce their payments to their provider ISPs. An Internet Exchange Points (IXP) (typically in a standalone building with its own switches) is a meeting point where multiple ISPs can connect and/or peer together. An ISP earns its money by charging each of the the ISPs that connect to the IXP a relatively small fee, which may depend on the amount of traffic sent to or received from the IXP. 15. Google's private network connects together all its data centers, big and small. Traffic between the Google data centers passes over its private network rather than over the public Internet. Many of these data centers are located in, or close to, lower tier ISPs. Therefore, when Google delivers content to a user, it often can bypass higher tier ISPs. What motivates content providers to create these networks? First, the content provider has more control over the user experience, since it has to use few intermediary ISPs. Second, it can save money by sending less traffic into provider networks. Third, if ISPs decide to charge more money to highly profitable content providers (in countries where net neutrality doesn't apply), the content providers can avoid these extra payments. 16. The delay components are processing delays, transmission delays, propagation delays, and queuing delays. All of these delays are fixed, except for the queuing delays, which are variable. 17. a) 1000 km, 1 Mbps, 100 bytes b) 100 km, 1 Mbps, 100 bytes 18. 10msec; d/s; no; no 19. a) 500 kbps b) 64 seconds c) 100kbps; 320 seconds 20. End system A breaks the large file into chunks. It adds header to each chunk, thereby generating multiple packets from the file. The header in each packet includes the IP address of the destination (end system B). The packet switch uses the destination IP address in the packet to determine the outgoing link. Asking which road to take is analogous to a packet asking which outgoing link it should be forwarded on, given the packet’s destination address. 21. The maximum emission rate is 500 packets/sec and the maximum transmission rate is 350 packets/sec. The corresponding traffic intensity is 500/350 =1.43 > 1. Loss will eventually occur for each experiment; but the time when loss first occurs will be different from one experiment to the next due to the randomness in the emission process. 22. Five generic tasks are error control, flow control, segmentation and reassembly, multiplexing, and connection setup. Yes, these tasks can be duplicated at different layers. For example, error control is often provided at more than one layer. 23. The five layers in the Internet protocol stack are – from top to bottom – the application layer, the transport layer, the network layer, the link layer, and the physical layer. The principal responsibilities are outlined in Section 1.5.1. 24. Application-layer message: data which an application wants to send and passed onto the transport layer; transport-layer segment: generated by the transport layer and encapsulates application-layer message with transport layer header; network-layer datagram: encapsulates transport-layer segment with a network-layer header; link-layer frame: encapsulates network-layer datagram with a link-layer header. 25. Routers process network, link and physical layers (layers 1 through 3). (This is a little bit of a white lie, as modern routers sometimes act as firewalls or caching components, and process Transport layer as well.) Link layer switches process link and physical layers (layers 1 through2). Hosts process all five layers. 26. a) Virus Requires some form of human interaction to spread. Classic example: E-mail viruses. b) Worms No user replication needed. Worm in infected host scans IP addresses and port numbers, looking for vulnerable processes to infect. 27. Creation of a botnet requires an attacker to find vulnerability in some application or system (e.g. exploiting the buffer overflow vulnerability that might exist in an application). After finding the vulnerability, the attacker needs to scan for hosts that are vulnerable. The target is basically to compromise a series of systems by exploiting that particular vulnerability. Any system that is part of the botnet can automatically scan its environment and propagate by exploiting the vulnerability. An important property of such botnets is that the originator of the botnet can remotely control and issue commands to all the nodes in the botnet. Hence, it becomes possible for the attacker to issue a command to all the nodes, that target a single node (for example, all nodes in the botnet might be commanded by the attacker to send a TCP SYN message to the target, which might result in a TCP SYN flood attack at the target). 28. Trudy can pretend to be Bob to Alice (and vice-versa) and partially or completely modify the message(s) being sent from Bob to Alice. For example, she can easily change the phrase “Alice, I owe you $1000” to “Alice, I owe you $10,000”. Furthermore, Trudy can even drop the packets that are being sent by Bob to Alice (and vise-versa), even if the packets from Bob to Alice are encrypted. Chapter 1 Problems Problem 1 There is no single right answer to this question. Many protocols would do the trick. Here's a simple answer below: Messages from ATM machine to Server Msg name purpose -------- ------- HELO Let server know that there is a card in the ATM machine ATM card transmits user ID to Server PASSWD User enters PIN, which is sent to server BALANCE User requests balance WITHDRAWL User asks to withdraw money BYE user all done Messages from Server to ATM machine (display) Msg name purpose -------- ------- PASSWD Ask user for PIN (password) OK last requested operation (PASSWD, WITHDRAWL) OK ERR last requested operation (PASSWD, WITHDRAWL) in ERROR AMOUNT sent in response to BALANCE request BYE user done, display welcome screen at ATM Correct operation: client server HELO (userid) --------------> (check if valid userid) <------------- PASSWD PASSWD --------------> (check password) <------------- AMOUNT WITHDRAWL --------------> check if enough $ to cover withdrawl (check if valid userid) <------------- PASSWD PASSWD --------------> (check password) <------------- AMOUNT WITHDRAWL --------------> check if enough $ to cover withdrawl <------------- BYE Problem 2 At time N*(L/R) the first packet has reached the destination, the second packet is stored in the last router, the third packet is stored in the next-to-last router, etc. At time N*(L/R) + L/R, the second packet has reached the destination, the third packet is stored in the last router, etc. Continuing with this logic, we see that at time N*(L/R) + (P-1)*(L/R) = (N+P-1)*(L/R) all packets have reached the destination. Problem 3 a) A circuit-switched network would be well suited to the application, because the application involves long sessions with predictable smooth bandwidth requirements. Since the transmission rate is known and not bursty, bandwidth can be reserved for each application session without significant waste. In addition, the overhead costs of setting up and tearing down connections are amortized over the lengthy duration of a typical application session. b) In the worst case, all the applications simultaneously transmit over one or more network links. However, since each link has sufficient bandwidth to handle the sum of all of the applications' data rates, no congestion (very little queuing) will occur. Given such generous link capacities, the network does not need congestion control mechanisms. Problem 4 Between the switch in the upper left and the switch in the upper right we can have 4 connections. Similarly we can have four connections between each of the 3 other pairs of adjacent switches. Thus, this network can support up to 16 connections. We can 4 connections passing through the switch in the upper-right-hand corner and another 4 connections passing through the switch in the lower-left-hand corner, giving a total of 8 connections. Yes. For the connections between A and C, we route two connections through B and two connections through D. For the connections between B and D, we route two connections through A and two connections through C. In this manner, there are at most 4 connections passing through any link. Problem 5 Tollbooths are 75 km apart, and the cars propagate at 100km/hr. A tollbooth services a car at a rate of one car every 12 seconds. a) There are ten cars. It takes 120 seconds, or 2 minutes, for the first tollbooth to service the 10 cars. Each of these cars has a propagation delay of 45 minutes (travel 75 km) before arriving at the second tollbooth. Thus, all the cars are lined up before the second tollbooth after 47 minutes. The whole process repeats itself for traveling between the second and third tollbooths. It also takes 2 minutes for the third tollbooth to service the 10 cars. Thus the total delay is 96 minutes. b) Delay between tollbooths is 8*12 seconds plus 45 minutes, i.e., 46 minutes and 36 seconds. The total delay is twice this amount plus 8*12 seconds, i.e., 94 minutes and 48 seconds. Problem 6 a) seconds. b) seconds. c) seconds. d) The bit is just leaving Host A. e) The first bit is in the link and has not reached Host B. f) The first bit has reached Host B. g) Want km. Problem 7 Consider the first bit in a packet. Before this bit can be transmitted, all of the bits in the packet must be generated. This requires sec=7msec. The time required to transmit the packet is sec= sec. Propagation delay = 10 msec. The delay until decoding is 7msec + sec + 10msec = 17.224msec A similar analysis shows that all bits experience a delay of 17.224 msec. Problem 8 a) 20 users can be supported. b) . c) . d) . We use the central limit theorem to approximate this probability. Let be independent random variables such that . “21 or more users” when is a standard normal r.v. Thus “21 or more users” . Problem 9 10,000 Problem 10 The first end system requires L/R1 to transmit the packet onto the first link; the packet propagates over the first link in d1/s1; the packet switch adds a processing delay of dproc; after receiving the entire packet, the packet switch connecting the first and the second link requires L/R2 to transmit the packet onto the second link; the packet propagates over the second link in d2/s2. Similarly, we can find the delay caused by the second switch and the third link: L/R3, dproc, and d3/s3. Adding these five delays gives dend-end = L/R1 + L/R2 + L/R3 + d1/s1 + d2/s2 + d3/s3+ dproc+ dproc To answer the second question, we simply plug the values into the equation to get 6 + 6 + 6 + 20+16 + 4 + 3 + 3 = 64 msec. Problem 11 Because bits are immediately transmitted, the packet switch does not introduce any delay; in particular, it does not introduce a transmission delay. Thus, dend-end = L/R + d1/s1 + d2/s2+ d3/s3 For the values in Problem 10, we get 6 + 20 + 16 + 4 = 46 msec. Problem 12 The arriving packet must first wait for the link to transmit 4.5 *1,500 bytes = 6,750 bytes or 54,000 bits. Since these bits are transmitted at 2 Mbps, the queuing delay is 27 msec. Generally, the queuing delay is (nL + (L - x))/R. Problem 13 The queuing delay is 0 for the first transmitted packet, L/R for the second transmitted packet, and generally, (n-1)L/R for the nth transmitted packet. Thus, the average delay for the N packets is: (L/R + 2L/R + ....... + (N-1)L/R)/N = L/(RN) * (1 + 2 + ..... + (N-1)) = L/(RN) * N(N-1)/2 = LN(N-1)/(2RN) = (N-1)L/(2R) Note that here we used the well-known fact: 1 + 2 + ....... + N = N(N+1)/2 It takes seconds to transmit the packets. Thus, the buffer is empty when a each batch of packets arrive. Thus, the average delay of a packet across all batches is the average delay within one batch, i.e., (N-1)L/2R. Problem 14 The transmission delay is . The total delay is Let . Total delay = For x=0, the total delay =0; as we increase x, total delay increases, approaching infinity as x approaches 1/a. Problem 15 Total delay . Problem 16 The total number of packets in the system includes those in the buffer and the packet that is being transmitted. So, N=10+1. Because , so (10+1)=a*(queuing delay + transmission delay). That is, 11=a*(0.01+1/100)=a*(0.01+0.01). Thus, a=550 packets/sec. Problem 17 There are nodes (the source host and the routers). Let denote the processing delay at the th node. Let be the transmission rate of the th link and let . Let be the propagation delay across the th link. Then . Let denote the average queuing delay at node . Then . Problem 18 On linux you can use the command traceroute www.targethost.com and in the Windows command prompt you can use tracert www.targethost.com In either case, you will get three delay measurements. For those three measurements you can calculate the mean and standard deviation. Repeat the experiment at different times of the day and comment on any changes. Here is an example solution: Traceroutes between San Diego Super Computer Center and www.poly.edu The average (mean) of the round-trip delays at each of the three hours is 71.18 ms, 71.38 ms and 71.55 ms, respectively. The standard deviations are 0.075 ms, 0.21 ms, 0.05 ms, respectively. In this example, the traceroutes have 12 routers in the path at each of the three hours. No, the paths didn’t change during any of the hours. Traceroute packets passed through four ISP networks from source to destination. Yes, in this experiment the largest delays occurred at peering interfaces between adjacent ISPs. Traceroutes from www.stella-net.net (France) to www.poly.edu (USA). The average round-trip delays at each of the three hours are 87.09 ms, 86.35 ms and 86.48 ms, respectively. The standard deviations are 0.53 ms, 0.18 ms, 0.23 ms, respectively. In this example, there are 11 routers in the path at each of the three hours. No, the paths didn’t change during any of the hours. Traceroute packets passed three ISP networks from source to destination. Yes, in this experiment the largest delays occurred at peering interfaces between adjacent ISPs. Problem 19 An example solution: Traceroutes from two different cities in France to New York City in United States In these traceroutes from two different cities in France to the same destination host in United States, seven links are in common including the transatlantic link. In this example of traceroutes from one city in France and from another city in Germany to the same host in United States, three links are in common including the transatlantic link. Traceroutes to two different cities in China from same host in United States Five links are common in the two traceroutes. The two traceroutes diverge before reaching China Problem 20 Throughput = min{Rs, Rc, R/M} Problem 21 If only use one path, the max throughput is given by: . If use all paths, the max throughput is given by . Problem 22 Probability of successfully receiving a packet is: ps= (1-p)N. The number of transmissions needed to be performed until the packet is successfully received by the client is a geometric random variable with success probability ps. Thus, the average number of transmissions needed is given by: 1/ps . Then, the average number of re-transmissions needed is given by: 1/ps -1. Problem 23 Let’s call the first packet A and call the second packet B. If the bottleneck link is the first link, then packet B is queued at the first link waiting for the transmission of packet A. So the packet inter-arrival time at the destination is simply L/Rs. If the second link is the bottleneck link and both packets are sent back to back, it must be true that the second packet arrives at the input queue of the second link before the second link finishes the transmission of the first packet. That is, L/Rs + L/Rs + dprop = L/Rs + dprop + L/Rc Thus, the minimum value of T is L/Rc  L/Rs . Problem 24 40 terabytes = 40 * 1012 * 8 bits. So, if using the dedicated link, it will take 40 * 1012 * 8 / (100 *106 ) =3200000 seconds = 37 days. But with FedEx overnight delivery, you can guarantee the data arrives in one day, and it should cost less than $100. Problem 25 160,000 bits 160,000 bits The bandwidth-delay product of a link is the maximum number of bits that can be in the link. the width of a bit = length of link / bandwidth-delay product, so 1 bit is 125 meters long, which is longer than a football field s/R Problem 26 s/R=20000km, then R=s/20000km= 2.5*108/(2*107)= 12.5 bps Problem 27 80,000,000 bits 800,000 bits, this is because that the maximum number of bits that will be in the link at any given time = min(bandwidth delay product, packet size) = 800,000 bits. .25 meters Problem 28 ttrans + tprop = 400 msec + 80 msec = 480 msec. 20 * (ttrans + 2 tprop) = 20*(20 msec + 80 msec) = 2 sec. Breaking up a file takes longer to transmit because each data packet and its corresponding acknowledgement packet add their own propagation delays. Problem 29 Recall geostationary satellite is 36,000 kilometers away from earth surface. 150 msec 1,500,000 bits 600,000,000 bits Problem 30 Let’s suppose the passenger and his/her bags correspond to the data unit arriving to the top of the protocol stack. When the passenger checks in, his/her bags are checked, and a tag is attached to the bags and ticket. This is additional information added in the Baggage layer if Figure 1.20 that allows the Baggage layer to implement the service or separating the passengers and baggage on the sending side, and then reuniting them (hopefully!) on the destination side. When a passenger then passes through security and additional stamp is often added to his/her ticket, indicating that the passenger has passed through a security check. This information is used to ensure (e.g., by later checks for the security information) secure transfer of people. Problem 31 Time to send message from source host to first packet switch = With store-and-forward switching, the total time to move message from source host to destination host = Time to send 1st packet from source host to first packet switch = . . Time at which 2nd packet is received at the first switch = time at which 1st packet is received at the second switch = Time at which 1st packet is received at the destination host = . After this, every 5msec one packet will be received; thus time at which last (800th) packet is received = . It can be seen that delay in using message segmentation is significantly less (almost 1/3rd). Without message segmentation, if bit errors are not tolerated, if there is a single bit error, the whole message has to be retransmitted (rather than a single packet). Without message segmentation, huge packets (containing HD videos, for example) are sent into the network. Routers have to accommodate these huge packets. Smaller packets have to queue behind enormous packets and suffer unfair delays. Packets have to be put in sequence at the destination. Message segmentation results in many smaller packets. Since header size is usually the same for all packets regardless of their size, with message segmentation the total amount of header bytes is more. Problem 32 Yes, the delays in the applet correspond to the delays in the Problem 31.The propagation delays affect the overall end-to-end delays both for packet switching and message switching equally. Problem 33 There are F/S packets. Each packet is S=80 bits. Time at which the last packet is received at the first router is sec. At this time, the first F/S-2 packets are at the destination, and the F/S-1 packet is at the second router. The last packet must then be transmitted by the first router and the second router, with each transmission taking sec. Thus delay in sending the whole file is To calculate the value of S which leads to the minimum delay, Problem 34 The circuit-switched telephone networks and the Internet are connected together at "gateways". When a Skype user (connected to the Internet) calls an ordinary telephone, a circuit is established between a gateway and the telephone user over the circuit switched network. The skype user's voice is sent in packets over the Internet to the gateway. At the gateway, the voice signal is reconstructed and then sent over the circuit. In the other direction, the voice signal is sent over the circuit switched network to the gateway. The gateway packetizes the voice signal and sends the voice packets to the Skype user.   Chapter 2 Review Questions The Web: HTTP; file transfer: FTP; remote login: Telnet; e-mail: SMTP; BitTorrent file sharing: BitTorrent protocol Network architecture refers to the organization of the communication process into layers (e.g., the five-layer Internet architecture). Application architecture, on the other hand, is designed by an application developer and dictates the broad structure of the application (e.g., client-server or P2P). The process which initiates the communication is the client; the process that waits to be contacted is the server. No. In a P2P file-sharing application, the peer that is receiving a file is typically the client and the peer that is sending the file is typically the server. The IP address of the destination host and the port number of the socket in the destination process. You would use UDP. With UDP, the transaction can be completed in one roundtrip time (RTT) - the client sends the transaction request into a UDP socket, and the server sends the reply back to the client's UDP socket. With TCP, a minimum of two RTTs are needed - one to set-up the TCP connection, and another for the client to send the request, and for the server to send back the reply. One such example is remote word processing, for example, with Google docs. However, because Google docs runs over the Internet (using TCP), timing guarantees are not provided. a) Reliable data transfer TCP provides a reliable byte-stream between client and server but UDP does not. b) A guarantee that a certain value for throughput will be maintained Neither c) A guarantee that data will be delivered within a specified amount of time Neither d) Confidentiality (via encryption) Neither SSL operates at the application layer. The SSL socket takes unencrypted data from the application layer, encrypts it and then passes it to the TCP socket. If the application developer wants TCP to be enhanced with SSL, she has to include the SSL code in the application. A protocol uses handshaking if the two communicating entities first exchange control packets before sending data to each other. SMTP uses handshaking at the application layer whereas HTTP does not. The applications associated with those protocols require that all application data be received in the correct order and without gaps. TCP provides this service whereas UDP does not. When the user first visits the site, the server creates a unique identification number, creates an entry in its back-end database, and returns this identification number as a cookie number. This cookie number is stored on the user’s host and is managed by the browser. During each subsequent visit (and purchase), the browser sends the cookie number back to the site. Thus the site knows when this user (more precisely, this browser) is visiting the site. Web caching can bring the desired content “closer” to the user, possibly to the same LAN to which the user’s host is connected. Web caching can reduce the delay for all objects, even objects that are not cached, since caching reduces the traffic on links. Telnet is not available in Windows 7 by default. to make it available, go to Control Panel, Programs and Features, Turn Windows Features On or Off, Check Telnet client. To start Telnet, in Windows command prompt, issue the following command > telnet webserverver 80 where "webserver" is some webserver. After issuing the command, you have established a TCP connection between your client telnet program and the web server. Then type in an HTTP GET message. An example is given below: Since the index.html page in this web server was not modified since Fri, 18 May 2007 09:23:34 GMT, and the above commands were issued on Sat, 19 May 2007, the server returned "304 Not Modified". Note that the first 4 lines are the GET message and header lines inputed by the user, and the next 4 lines (starting from HTTP/1.1 304 Not Modified) is the response from the web server. FTP uses two parallel TCP connections, one connection for sending control information (such as a request to transfer a file) and another connection for actually transferring the file. Because the control information is not sent over the same connection that the file is sent over, FTP sends control information out of band. The message is first sent from Alice’s host to her mail server over HTTP. Alice’s mail server then sends the message to Bob’s mail server over SMTP. Bob then transfers the message from his mail server to his host over POP3. 17. Received: from 65.54.246.203 (EHLO bay0-omc3-s3.bay0.hotmail.com) (65.54.246.203) by mta419.mail.mud.yahoo.com with SMTP; Sat, 19 May 2007 16:53:51 -0700 Received: from hotmail.com ([65.55.135.106]) by bay0-omc3-s3.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.2668); Sat, 19 May 2007 16:52:42 -0700 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 19 May 2007 16:52:41 -0700 Message-ID: Received: from 65.55.135.123 by by130fd.bay130.hotmail.msn.com with HTTP; Sat, 19 May 2007 23:52:36 GMT From: "prithula dhungel" To: prithula@yahoo.com Bcc: Subject: Test mail Date: Sat, 19 May 2007 23:52:36 +0000 Mime-Version: 1.0 Content-Type: Text/html; format=flowed Return-Path: prithuladhungel@hotmail.com Figure: A sample mail message header Received: This header field indicates the sequence in which the SMTP servers send and receive the mail message including the respective timestamps. In this example there are 4 “Received:” header lines. This means the mail message passed through 5 different SMTP servers before being delivered to the receiver’s mail box. The last (forth) “Received:” header indicates the mail message flow from the SMTP server of the sender to the second SMTP server in the chain of servers. The sender’s SMTP server is at address 65.55.135.123 and the second SMTP server in the chain is by130fd.bay130.hotmail.msn.com. The third “Received:” header indicates the mail message flow from the second SMTP server in the chain to the third server, and so on. Finally, the first “Received:” header indicates the flow of the mail messages from the forth SMTP server to the last SMTP server (i.e. the receiver’s mail server) in the chain. Message-id: The message has been given this number BAY130-F26D9E35BF59E0D18A819AFB9310@phx.gbl (by bay0-omc3-s3.bay0.hotmail.com. Message-id is a unique string assigned by the mail system when the message is first created. From: This indicates the email address of the sender of the mail. In the given example, the sender is “prithuladhungel@hotmail.com” To: This field indicates the email address of the receiver of the mail. In the example, the receiver is “prithula@yahoo.com” Subject: This gives the subject of the mail (if any specified by the sender). In the example, the subject specified by the sender is “Test mail” Date: The date and time when the mail was sent by the sender. In the example, the sender sent the mail on 19th May 2007, at time 23:52:36 GMT. Mime-version: MIME version used for the mail. In the example, it is 1.0. Content-type: The type of content in the body of the mail message. In the example, it is “text/html”. Return-Path: This specifies the email address to which the mail will be sent if the receiver of this mail wants to reply to the sender. This is also used by the sender’s mail server for bouncing back undeliverable mail messages of mailer-daemon error messages. In the example, the return path is “prithuladhungel@hotmail.com”. With download and delete, after a user retrieves its messages from a POP server, the messages are deleted. This poses a problem for the nomadic user, who may want to access the messages from many different machines (office PC, home PC, etc.). In the download and keep configuration, messages are not deleted after the user retrieves the messages. This can also be inconvenient, as each time the user retrieves the stored messages from a new machine, all of non-deleted messages will be transferred to the new machine (including very old messages). Yes an organization’s mail server and Web server can have the same alias for a host name. The MX record is used to map the mail server’s host name to its IP address. You should be able to see the sender's IP address for a user with an .edu email address. But you will not be able to see the sender's IP address if the user uses a gmail account. It is not necessary that Bob will also provide chunks to Alice. Alice has to be in the top 4 neighbors of Bob for Bob to send out chunks to her; this might not occur even if Alice provides chunks to Bob throughout a 30-second interval. Recall that in BitTorrent, a peer picks a random peer and optimistically unchokes the peer for a short period of time. Therefore, Alice will eventually be optimistically unchoked by one of her neighbors, during which time she will receive chunks from that neighbor. The overlay network in a P2P file sharing system consists of the nodes participating in the file sharing system and the logical links between the nodes. There is a logical link (an “edge” in graph theory terms) from node A to node B if there is a semi-permanent TCP connection between A and B. An overlay network does not include routers. Mesh DHT: The advantage is in order to a route a message to the peer (with ID) that is closest to the key, only one hop is required; the disadvantage is that each peer must track all other peers in the DHT. Circular DHT: the advantage is that each peer needs to track only a few other peers; the disadvantage is that O(N) hops are needed to route a message to the peer that is closest to the key. 25. File Distribution Instant Messaging Video Streaming Distributed Computing With the UDP server, there is no welcoming socket, and all data from different clients enters the server through this one socket. With the TCP server, there is a welcoming socket, and each time a client initiates a connection to the server, a new socket is created. Thus, to support n simultaneous connections, the server would need n+1 sockets. For the TCP application, as soon as the client is executed, it attempts to initiate a TCP connection with the server. If the TCP server is not running, then the client will fail to make a connection. For the UDP application, the client does not initiate connections (or attempt to communicate with the UDP server) immediately upon execution Chapter 2 Problems Problem 1 a) F b) T c) F d) F e) F Problem 2 Access control commands: USER, PASS, ACT, CWD, CDUP, SMNT, REIN, QUIT. Transfer parameter commands: PORT, PASV, TYPE STRU, MODE. Service commands: RETR, STOR, STOU, APPE, ALLO, REST, RNFR, RNTO, ABOR, DELE, RMD, MRD, PWD, LIST, NLST, SITE, SYST, STAT, HELP, NOOP. Problem 3 Application layer protocols: DNS and HTTP Transport layer protocols: UDP for DNS; TCP for HTTP Problem 4 The document request was http://gaia.cs.umass.edu/cs453/index.html. The Host : field indicates the server's name and /cs453/index.html indicates the file name. The browser is running HTTP version 1.1, as indicated just before the first pair. The browser is requesting a persistent connection, as indicated by the Connection: keep-alive. This is a trick question. This information is not contained in an HTTP message anywhere. So there is no way to tell this from looking at the exchange of HTTP messages alone. One would need information from the IP datagrams (that carried the TCP segment that carried the HTTP GET request) to answer this question. Mozilla/5.0. The browser type information is needed by the server to send different versions of the same object to different types of browsers. Problem 5 The status code of 200 and the phrase OK indicate that the server was able to locate the document successfully. The reply was provided on Tuesday, 07 Mar 2008 12:39:45 Greenwich Mean Time. The document index.html was last modified on Saturday 10 Dec 2005 18:27:46 GMT. There are 3874 bytes in the document being returned. The first five bytes of the returned document are : ed to a persistent connection, as indicated by the Connection: Keep-Alive field Problem 6 Persistent connections are discussed in section 8 of RFC 2616 (the real goal of this question was to get you to retrieve and read an RFC). Sections 8.1.2 and 8.1.2.1 of the RFC indicate that either the client or the server can indicate to the other that it is going to close the persistent connection. It does so by including the connection-token "close" in the Connection-header field of the http request/reply. HTTP does not provide any encryption services. (From RFC 2616) “Clients that use persistent connections should limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy.” Yes. (From RFC 2616) “A client might have started to send a new request at the same time that the server has decided to close the "idle" connection. From the server's point of view, the connection is being closed while it was idle, but from the client's point of view, a request is in progress.” Problem 7 The total amount of time to get the IP address is . Once the IP address is known, elapses to set up the TCP connection and another elapses to request and receive the small object. The total response time is Problem 8 . . Problem 9 The time to transmit an object of size L over a link or rate R is L/R. The average time is the average size of the object divided by R:  = (850,000 bits)/(15,000,000 bits/sec) = .0567 sec The traffic intensity on the link is given by =(16 requests/sec)(.0567 sec/request) = 0.907. Thus, the average access delay is (.0567 sec)/(1 - .907)  .6 seconds. The total average response time is therefore .6 sec + 3 sec = 3.6 sec. The traffic intensity on the access link is reduced by 60% since the 60% of the requests are satisfied within the institutional network. Thus the average access delay is (.0567 sec)/[1 – (.4)(.907)] = .089 seconds. The response time is approximately zero if the request is satisfied by the cache (which happens with probability .6); the average response time is .089 sec + 3 sec = 3.089 sec for cache misses (which happens 40% of the time). So the average response time is (.6)(0 sec) + (.4)(3.089 sec) = 1.24 seconds. Thus the average response time is reduced from 3.6 sec to 1.24 sec. Problem 10 Note that each downloaded object can be completely put into one data packet. Let Tp denote the one-way propagation delay between the client and the server. First consider parallel downloads using non-persistent connections. Parallel downloads would allow 10 connections to share the 150 bits/sec bandwidth, giving each just 15 bits/sec. Thus, the total time needed to receive all objects is given by: (200/150+Tp + 200/150 +Tp + 200/150+Tp + 100,000/150+ Tp ) + (200/(150/10)+Tp + 200/(150/10) +Tp + 200/(150/10)+Tp + 100,000/(150/10)+ Tp ) = 7377 + 8*Tp (seconds) Now consider a persistent HTTP connection. The total time needed is given by: (200/150+Tp + 200/150 +Tp + 200/150+Tp + 100,000/150+ Tp ) + 10*(200/150+Tp + 100,000/150+ Tp ) =7351 + 24*Tp (seconds) Assuming the speed of light is 300*106 m/sec, then Tp=10/(300*106)=0.03 microsec. Tp is therefore negligible compared with transmission delay. Thus, we see that persistent HTTP is not significantly faster (less than 1 percent) than the non-persistent case with parallel download. Problem 11 Yes, because Bob has more connections, he can get a larger share of the link bandwidth. Yes, Bob still needs to perform parallel downloads; otherwise he will get less bandwidth than the other four users. Problem 12 Server.py from socket import * serverPort=12000 serverSocket=socket(AF_INET,SOCK_STREAM) serverSocket.bind(('',serverPort)) serverSocket.listen(1) connectionSocket, addr = serverSocket.accept() while 1: sentence = connectionSocket.recv(1024) print 'From Server:', sentence, '\n' serverSocket.close() Problem 13 The MAIL FROM: in SMTP is a message from the SMTP client that identifies the sender of the mail message to the SMTP server. The From: on the mail message itself is NOT an SMTP message, but rather is just a line in the body of the mail message. Problem 14 SMTP uses a line containing only a period to mark the end of a message body. HTTP uses “Content-Length header field” to indicate the length of a message body. No, HTTP cannot use the method used by SMTP, because HTTP message could be binary data, whereas in SMTP, the message body must be in 7-bit ASCII format. Problem 15 MTA stands for Mail Transfer Agent. A host sends the message to an MTA. The message then follows a sequence of MTAs to reach the receiver’s mail reader. We see that this spam message follows a chain of MTAs. An honest MTA should report where it receives the message. Notice that in this message, “asusus-4b96 ([58.88.21.177])” does not report from where it received the email. Since we assume only the originator is dishonest, so “asusus-4b96 ([58.88.21.177])” must be the originator. Problem 16 UIDL abbreviates “unique-ID listing”. When a POP3 client issues the UIDL command, the server responds with the unique message ID for all of the messages present in the user's mailbox. This command is useful for “download and keep”. By maintaining a file that lists the messages retrieved during earlier sessions, the client can use the UIDL command to determine which messages on the server have already been seen. Problem 17 a) C: dele 1 C: retr 2 S: (blah blah … S: ………..blah) S: . C: dele 2 C: quit S: +OK POP3 server signing off b) C: retr 2 S: blah blah … S: ………..blah S: . C: quit S: +OK POP3 server signing off C: list S: 1 498 S: 2 912 S: . C: retr 1 S: blah ….. S: ….blah S: . C: retr 2 S: blah blah … S: ………..blah S: . C: quit S: +OK POP3 server signing off Problem 18 For a given input of domain name (such as ccn.com), IP address or network administrator name, the whois database can be used to locate the corresponding registrar, whois server, DNS server, and so on. NS4.YAHOO.COM from www.register.com; NS1.MSFT.NET from ww.register.com Local Domain: www.mindspring.com Web servers : www.mindspring.com 207.69.189.21, 207.69.189.22, 207.69.189.23, 207.69.189.24, 207.69.189.25, 207.69.189.26, 207.69.189.27, 207.69.189.28 Mail Servers : mx1.mindspring.com (207.69.189.217) mx2.mindspring.com (207.69.189.218) mx3.mindspring.com (207.69.189.219) mx4.mindspring.com (207.69.189.220) Name Servers: itchy.earthlink.net (207.69.188.196) scratchy.earthlink.net (207.69.188.197) www.yahoo.com Web Servers: www.yahoo.com (216.109.112.135, 66.94.234.13) Mail Servers: a.mx.mail.yahoo.com (209.191.118.103) b.mx.mail.yahoo.com (66.196.97.250) c.mx.mail.yahoo.com (68.142.237.182, 216.39.53.3) d.mx.mail.yahoo.com (216.39.53.2) e.mx.mail.yahoo.com (216.39.53.1) f.mx.mail.yahoo.com (209.191.88.247, 68.142.202.247) g.mx.mail.yahoo.com (209.191.88.239, 206.190.53.191) Name Servers: ns1.yahoo.com (66.218.71.63) ns2.yahoo.com (68.142.255.16) ns3.yahoo.com (217.12.4.104) ns4.yahoo.com (68.142.196.63) ns5.yahoo.com (216.109.116.17) ns8.yahoo.com (202.165.104.22) ns9.yahoo.com (202.160.176.146) www.hotmail.com Web Servers: www.hotmail.com (64.4.33.7, 64.4.32.7) Mail Servers: mx1.hotmail.com (65.54.245.8, 65.54.244.8, 65.54.244.136) mx2.hotmail.com (65.54.244.40, 65.54.244.168, 65.54.245.40) mx3.hotmail.com (65.54.244.72, 65.54.244.200, 65.54.245.72) mx4.hotmail.com (65.54.244.232, 65.54.245.104, 65.54.244.104) Name Servers: ns1.msft.net (207.68.160.190) ns2.msft.net (65.54.240.126) ns3.msft.net (213.199.161.77) ns4.msft.net (207.46.66.126) ns5.msft.net (65.55.238.126) d) The yahoo web server has multiple IP addresses www.yahoo.com (216.109.112.135, 66.94.234.13) e) The address range for Polytechnic University: 128.238.0.0 – 128.238.255.255 f) An attacker can use the whois database and nslookup tool to determine the IP address ranges, DNS server addresses, etc., for the target institution. By analyzing the source address of attack packets, the victim can use whois to obtain information about domain from which the attack is coming and possibly inform the administrators of the origin domain. Problem 19 The following delegation chain is used for gaia.cs.umass.edu a.root-servers.net E.GTLD-SERVERS.NET ns1.umass.edu(authoritative) First command: dig +norecurse @a.root-servers.net any gaia.cs.umass.edu ;; AUTHORITY SECTION: edu. 172800 IN NS E.GTLD-SERVERS.NET. edu. 172800 IN NS A.GTLD-SERVERS.NET. edu. 172800 IN NS G3.NSTLD.COM. edu. 172800 IN NS D.GTLD-SERVERS.NET. edu. 172800 IN NS H3.NSTLD.COM. edu. 172800 IN NS L3.NSTLD.COM. edu. 172800 IN NS M3.NSTLD.COM. edu. 172800 IN NS C.GTLD-SERVERS.NET. Among all returned edu DNS servers, we send a query to the first one. dig +norecurse @E.GTLD-SERVERS.NET any gaia.cs.umass.edu umass.edu. 172800 IN NS ns1.umass.edu. umass.edu. 172800 IN NS ns2.umass.edu. umass.edu. 172800 IN NS ns3.umass.edu. Among all three returned authoritative DNS servers, we send a query to the first one. dig +norecurse @ns1.umass.edu any gaia.cs.umass.edu gaia.cs.umass.edu. 21600 IN A 128.119.245.12 The answer for google.com could be: a.root-servers.net E.GTLD-SERVERS.NET ns1.google.com(authoritative) Problem 20 We can periodically take a snapshot of the DNS caches in the local DNS servers. The Web server that appears most frequently in the DNS caches is the most popular server. This is because if more users are interested in a Web server, then DNS requests for that server are more frequently sent by users. Thus, that Web server will appear in the DNS caches more frequently. For a complete measurement study, see: Craig E. Wills, Mikhail Mikhailov, Hao Shang “Inferring Relative Popularity of Internet Applications by Actively Querying DNS Caches”, in IMC'03, October 27­29, 2003, Miami Beach, Florida, USA Problem 21 Yes, we can use dig to query that Web site in the local DNS server. For example, “dig cnn.com” will return the query time for finding cnn.com. If cnn.com was just accessed a couple of seconds ago, an entry for cnn.com is cached in the local DNS cache, so the query time is 0 msec. Otherwise, the query time is large. Problem 22 For calculating the minimum distribution time for client-server distribution, we use the following formula: Dcs = max {NF/us, F/dmin} Similarly, for calculating the minimum distribution time for P2P distribution, we use the following formula: Where, F = 15 Gbits = 15 * 1024 Mbits us = 30 Mbps dmin = di = 2 Mbps Note, 300Kbps = 300/1024 Mbps. Client Server N 10 100 1000 u 300 Kbps 7680 51200 512000 700 Kbps 7680 51200 512000 2 Mbps 7680 51200 512000 Peer to Peer N 10 100 1000 u 300 Kbps 7680 25904 47559 700 Kbps 7680 15616 21525 2 Mbps 7680 7680 7680 Problem 23 Consider a distribution scheme in which the server sends the file to each client, in parallel, at a rate of a rate of us/N. Note that this rate is less than each of the client’s download rate, since by assumption us/N ≤ dmin. Thus each client can also receive at rate us/N. Since each client receives at rate us/N, the time for each client to receive the entire file is F/( us/N) = NF/ us. Since all the clients receive the file in NF/ us, the overall distribution time is also NF/ us. Consider a distribution scheme in which the server sends the file to each client, in parallel, at a rate of dmin. Note that the aggregate rate, N dmin, is less than the server’s link rate us, since by assumption us/N ≥ dmin. Since each client receives at rate dmin, the time for each client to receive the entire file is F/ dmin. Since all the clients receive the file in this time, the overall distribution time is also F/ dmin. From Section 2.6 we know that DCS ≥ max {NF/us, F/dmin} (Equation 1) Suppose that us/N ≤ dmin. Then from Equation 1 we have DCS ≥ NF/us . But from (a) we have DCS ≤ NF/us . Combining these two gives: DCS = NF/us when us/N ≤ dmin. (Equation 2) We can similarly show that: DCS =F/dmin when us/N ≥ dmin (Equation 3). Combining Equation 2 and Equation 3 gives the desired result. Problem 24 Define u = u1 + u2 + ….. + uN. By assumption us <= (us + u)/N Equation 1 Divide the file into N parts, with the ith part having size (ui/u)F. The server transmits the ith part to peer i at rate ri = (ui/u)us. Note that r1 + r2 + ….. + rN = us, so that the aggregate server rate does not exceed the link rate of the server. Also have each peer i forward the bits it receives to each of the N-1 peers at rate ri. The aggregate forwarding rate by peer i is (N-1)ri. We have (N-1)ri = (N-1)(usui)/u = (us + u)/N Equation 2 Let ri = ui/(N-1) and rN+1 = (us – u/(N-1))/N In this distribution scheme, the file is broken into N+1 parts. The server sends bits from the ith part to the ith peer (i = 1, …., N) at rate ri. Each peer i forwards the bits arriving at rate ri to each of the other N-1 peers. Additionally, the server sends bits from the (N+1) st part at rate rN+1 to each of the N peers. The peers do not forward the bits from the (N+1)st part. The aggregate send rate of the server is r1+ …. + rN + N rN+1 = u/(N-1) + us – u/(N-1) = us Thus, the server’s send rate does not exceed its link rate. The aggregate send rate of peer i is (N-1)ri = ui Thus, each peer’s send rate does not exceed its link rate. In this distribution scheme, peer i receives bits at an aggregate rate of Thus each peer receives the file in NF/(us+u). (For simplicity, we neglected to specify the size of the file part for i = 1, …., N+1. We now provide that here. Let Δ = (us+u)/N be the distribution time. For i = 1, …, N, the ith file part is Fi = ri Δ bits. The (N+1)st file part is FN+1 = rN+1 Δ bits. It is straightforward to show that F1+ ….. + FN+1 = F.) The solution to this part is similar to that of 17 (c). We know from section 2.6 that Combining this with a) and b) gives the desired result. Problem 25 There are N nodes in the overlay network. There are N(N-1)/2 edges. Problem 26 Yes. His first claim is possible, as long as there are enough peers staying in the swarm for a long enough time. Bob can always receive data through optimistic unchoking by other peers. His second claim is also true. He can run a client on each host, let each client “free-ride,” and combine the collected chunks from the different hosts into a single file. He can even write a small scheduling program to make the different hosts ask for different chunks of the file. This is actually a kind of Sybil attack in P2P networks. Problem 27 Peer 3 learns that peer 5 has just left the system, so Peer 3 asks its first successor (Peer 4) for the identifier of its immediate successor (peer 8). Peer 3 will then make peer 8 its second successor. Problem 28 Peer 6 would first send peer 15 a message, saying “what will be peer 6’s predecessor and successor?” This message gets forwarded through the DHT until it reaches peer 5, who realizes that it will be 6’s predecessor and that its current successor, peer 8, will become 6’s successor. Next, peer 5 sends this predecessor and successor information back to 6. Peer 6 can now join the DHT by making peer 8 its successor and by notifying peer 5 that it should change its immediate successor to 6. Problem 29 For each key, we first calculate the distances (using d(k,p)) between itself and all peers, and then store the key in the peer that is closest to the key (that is, with smallest distance value). Problem 30 Yes, randomly assigning keys to peers does not consider the underlying network at all, so it very likely causes mismatches. Such mismatches may degrade the search performance. For example, consider a logical path p1 (consisting of only two logical links): ABC, where A and B are neighboring peers, and B and C are neighboring peers. Suppose that there is another logical path p2 from A to C (consisting of 3 logical links): ADEC. It might be the case that A and B are very far away physically (and separated by many routers), and B and C are very far away physically (and separated by many routers). But it may be the case that A, D, E, and C are all very close physically (and all separated by few routers). In other words, a shorter logical path may correspond to a much longer physical path. Problem 31 If you run TCPClient first, then the client will attempt to make a TCP connection with a non-existent server process. A TCP connection will not be made. UDPClient doesn't establish a TCP connection with the server. Thus, everything should work fine if you first run UDPClient, then run UDPServer, and then type some input into the keyboard. If you use different port numbers, then the client will attempt to establish a TCP connection with the wrong process or a non-existent process. Errors will occur. Problem 32 In the original program, UDPClient does not specify a port number when it creates the socket. In this case, the code lets the underlying operating system choose a port number. With the additional line, when UDPClient is executed, a UDP socket is created with port number 5432 . UDPServer needs to know the client port number so that it can send packets back to the correct client socket. Glancing at UDPServer, we see that the client port number is not “hard-wired” into the server code; instead, UDPServer determines the client port number by unraveling the datagram it receives from the client. Thus UDP server will work with any client port number, including 5432. UDPServer therefore does not need to be modified. Before: Client socket = x (chosen by OS) Server socket = 9876 After: Client socket = 5432 Problem 33 Yes, you can configure many browsers to open multiple simultaneous connections to a Web site. The advantage is that you will you potentially download the file faster. The disadvantage is that you may be hogging the bandwidth, thereby significantly slowing down the downloads of other users who are sharing the same physical links. Problem 34 For an application such as remote login (telnet and ssh), a byte-stream oriented protocol is very natural since there is no notion of message boundaries in the application. When a user types a character, we simply drop the character into the TCP connection. In other applications, we may be sending a series of messages that have inherent boundaries between them. For example, when one SMTP mail server sends another SMTP mail server several email messages back to back. Since TCP does not have a mechanism to indicate the boundaries, the application must add the indications itself, so that receiving side of the application can distinguish one message from the next. If each message were instead put into a distinct UDP segment, the receiving end would be able to distinguish the various messages without any indications added by the sending side of the application. Problem 35 To create a web server, we need to run web server software on a host. Many vendors sell web server software. However, the most popular web server software today is Apache, which is open source and free. Over the years it has been highly optimized by the open-source community. Problem 36 The key is the infohash, the value is an IP address that currently has the file designated by the infohash.   Chapter 3 Review Questions Call this protocol Simple Transport Protocol (STP). At the sender side, STP accepts from the sending process a chunk of data not exceeding 1196 bytes, a destination host address, and a destination port number. STP adds a four-byte header to each chunk and puts the port number of the destination process in this header. STP then gives the destination host address and the resulting segment to the network layer. The network layer delivers the segment to STP at the destination host. STP then examines the port number in the segment, extracts the data from the segment, and passes the data to the process identified by the port number. The segment now has two header fields: a source port field and destination port field. At the sender side, STP accepts a chunk of data not exceeding 1192 bytes, a destination host address, a source port number, and a destination port number. STP creates a segment which contains the application data, source port number, and destination port number. It then gives the segment and the destination host address to the network layer. After receiving the segment, STP at the receiving host gives the application process the application data and the source port number. No, the transport layer does not have to do anything in the core; the transport layer “lives” in the end systems. For sending a letter, the family member is required to give the delegate the letter itself, the address of the destination house, and the name of the recipient. The delegate clearly writes the recipient’s name on the top of the letter. The delegate then puts the letter in an envelope and writes the address of the destination house on the envelope. The delegate then gives the letter to the planet’s mail service. At the receiving side, the delegate receives the letter from the mail service, takes the letter out of the envelope, and takes note of the recipient name written at the top of the letter. The delegate then gives the letter to the family member with this name. No, the mail service does not have to open the envelope; it only examines the address on the envelope. Source port number y and destination port number x. An application developer may not want its application to use TCP’s congestion control, which can throttle the application’s sending rate at times of congestion. Often, designers of IP telephony and IP videoconference applications choose to run their applications over UDP because they want to avoid TCP’s congestion control. Also, some applications do not need the reliable data transfer provided by TCP. Since most firewalls are configured to block UDP traffic, using TCP for video and voice traffic lets the traffic though the firewalls. Yes. The application developer can put reliable data transfer into the application layer protocol. This would require a significant amount of work and debugging, however. Yes, both segments will be directed to the same socket. For each received segment, at the socket interface, the operating system will provide the process with the IP addresses to determine the origins of the individual segments. For each persistent connection, the Web server creates a separate “connection socket”. Each connection socket is identified with a four-tuple: (source IP address, source port number, destination IP address, destination port number). When host C receives and IP datagram, it examines these four fields in the datagram/segment to determine to which socket it should pass the payload of the TCP segment. Thus, the requests from A and B pass through different sockets. The identifier for both of these sockets has 80 for the destination port; however, the identifiers for these sockets have different values for source IP addresses. Unlike UDP, when the transport layer passes a TCP segment’s payload to the application process, it does not specify the source IP address, as this is implicitly specified by the socket identifier. Sequence numbers are required for a receiver to find out whether an arriving packet contains new data or is a retransmission. To handle losses in the channel. If the ACK for a transmitted packet is not received within the duration of the timer for the packet, the packet (or its ACK or NACK) is assumed to have been lost. Hence, the packet is retransmitted. A timer would still be necessary in the protocol rdt 3.0. If the round trip time is known then the only advantage will be that, the sender knows for sure that either the packet or the ACK (or NACK) for the packet has been lost, as compared to the real scenario, where the ACK (or NACK) might still be on the way to the sender, after the timer expires. However, to detect the loss, for each packet, a timer of constant duration will still be necessary at the sender. The packet loss caused a time out after which all the five packets were retransmitted. Loss of an ACK didn’t trigger any retransmission as Go-Back-N uses cumulative acknowledgements. The sender was unable to send sixth packet as the send window size is fixed to 5. When the packet was lost, the received four packets were buffered the receiver. After the timeout, sender retransmitted the lost packet and receiver delivered the buffered packets to application in correct order. Duplicate ACK was sent by the receiver for the lost ACK. The sender was unable to send sixth packet as the send win
RX Library 2.75 with Delphi 2009 support (by FlexGraphics software) ====================================================================== The Set of Native Delphi Components for Borland Delphi versions 1, 2, 3, 4, 5, 6, 7, 2005, 2006, 2009 and Borland C++ Builder 1, 3, 4, 5, 6, 2006 & 2009. 100% Source Code. Last revision date Oct 12, 1999. PLEASE FOLLOW THE INSTRUCTIONS PROVIDED IN THE INSTALLATION SECTION! TABLE OF CONTENTS ----------------- Latest Changes Overview History License Agreement Installation Demonstration Programs Source Files Using GIF Images Copyright Notes NEW FOR VERSION 2.75 -------------------- Delphi 5.0 & C++Builder 4.0 Compatibility New components: TRxLoginDialog New properties, events: TFormPlacement.RegistryRoot TFormPlacement.Version TFontComboBox.UseFonts TRxDBGrid.OnTopLeftChanged TRxDBLookupCombo.DisplayValues TStrHolder.Macros, TStrHolder.OnExpandMacros RxSpin.TValueType.vtHex New routines, methods, constants: SaveClipboardToStream, LoadClipboardFromStream (clipmon.pas) AppFileName, AppVerInfo (rxverinf.pas) XorString, XorEncode, XorDecode (strutils.pas) BUG FIXES. Overview -------- This version is the result of long unactivity of RX Library authors and some imperfections and bugs of other RX adaptations to Delphi 6. The authors of this version disclaim all warranties as to this software, whether express or implied, including without limitation any implied warranties of merchantability or fitness for a particular purpose. Use under your own responsibility, but comments (even critique) in English (or in Russian) are welcome. RX Library contains a large number of components, objects and routines for Borland Delphi with full source code. This library is compatible with Borland Delphi 1, 2, 3, 4, 5, 6 and Borland C++ Builder 1, 3, 4. This collection includes over 60 native Delphi components. RX Library is a freeware product. Feel free to distribute the library as long as all files are unmodified and kep
关于雷达方面的知识! EFFECTIVENESS OF EXTRACTING WATER SURFACE SLOPES FROM LIDAR DATA WITHIN THE ACTIVE CHANNEL: SANDY RIVER, OREGON, USA by JOHN THOMAS ENGLISH A THESIS Presented to the Department of Geography and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Master of Science March 2009 11 "Effectiveness of Extracting Water Surface Slopes from LiDAR Data within the Active Channel: Sandy River, Oregon, USA," a thesis prepared by John Thomas English in partial fulfillment of the requirements for the Master of Science degree in the Department of Geography. This thesis has been approved and accepted by: Date Committee in Charge: W. Andrew Marcus, Chair Patricia F. McDowell Accepted by: Dean of the Graduate School © 2009 John Thomas English 111 IV An Abstract of the Thesis of John Thomas English in the Department of Geography for the degree of to be taken Master of Science March 2009 Title: EFFECTIVENESS OF EXTRACTING WATER SURFACE SLOPES FROM LIDAR DATA WITHIN THE ACTIVE CHANNEL: SANDY RIVER, OREGON, USA Approved: _ W. Andrew Marcus This paper examines the capability ofLiDAR data to accurately map river water surface slopes in three reaches of the Sandy River, Oregon, USA. LiDAR data were compared with field measurements to evaluate accuracies and determine how water surface roughness and point density affect LiDAR measurements. Results show that LiDAR derived water surface slopes were accurate to within 0.0047,0.0025, and 0.0014 slope, with adjusted R2 values of 0.35, 0.47, and 0.76 for horizontal intervals of 5, 10, and 20m, respectively. Additionally, results show LiDAR provides greater data density where water surfaces are broken. This study provides conclusive evidence supporting use ofLiDAR to measure water surface slopes of channels with accuracies similar to field based approaches. CURRICULUM VITAE NAME OF AUTHOR: John Thomas English PLACE OF BIRTH: Eugene, Oregon DATE OF BIRTH: January 1st, 1980 GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene, Oregon Southern Oregon University, Ashland, Oregon DEGREES AWARDED: Master of Science, Geography, March 2009, University of Oregon Bachelor of Science, Geography, 2001, Southern Oregon University AREAS OF SPECIAL INTEREST: Fluvial Geomorphology Remote Sensing PROFESSIONAL EXPERIENCE: LiDAR Database Coordinator, Oregon Department of Geology & Mineral Industries, June 2008 - present. LiDAR & Remote Sensing Specialist, Sky Research Inc., 2003 - 2008 GRANTS, AWARDS AND HONORS: Gamma Theta Upsilon Geographic Society Member, 2006 Gradutate Teaching Fellowship, Social Science Instructional Laboratory, 20062007 v VI ACKNOWLEDGMENTS I wish to express special thanks to Professors W.A. Marcus and Patricia McDowell for their assistance in the preparation of this manuscript. In addition, special thanks are due to Mr. Paul Blanton who assisted with field data collection for this project. I also thank the members ofmy family who have been encouraging and supportive during the entirety of my graduate schooling. I wish to thank my parents Thomas and Nancy English for always being proud of me. Special thanks to my son Finn for always making me smile. Lastly, special thanks to my wife Kathryn for her unwavering support, love, and encouragement. Dedicated to my mother Bonita Claire English (1950-2004). Vll V111 TABLE OF CONTENTS Chapter Page I. INTRODUCTION 1 II. BACKGROlTND 5 Water Surface Slope 5 LiDAR Measurements of Active Channel Features 7 III. STUDY AREA 10 IV. METHODS 22 Overview 22 LiDAR Data and Image Acquisition 23 Field Data Acquisition 24 LiDAR Processing 25 Calculation of Water Surface Slopes 27 Evaluating LiDAR Slope Accuracies and Controls 33 V. RESULTS 35 Comparison of Absolute Elevations from Field and LiDAR Data in Reach 1 35 Slope Comparisons 41 Surface Roughness Analysis 46 VI. DiSCUSSiON 51 VII. CONCLUSION 57 APPENDIX: ARCGIS VBA SCRIPT CODE 58 REFERENCES 106 IX LIST OF FIGURES Figure Page 1. Return Factor vs. LiDAR Scan Angle 2 2. Angle of Incidence 3 3. Wave Action Relationship to LiDAR Echo 3 4. Site Map 11 5. Annual Hydrograph of Sandy River 13 6. Oregon GAP Vegetation within Study Area 15 7. Photo of Himalayan Blackberry on Sandy River 16 8. Reach 1 Site Area Map with photo 18 9. Reach 2 Site Area Map 20 10. Reach 3 Site Area Map 21 11. LiDAR Point Filtering Processing Step 26 12. Field DEM Interpolated using Kriging 29 13. Reach 1 LiDAR Cross Sections and Sample Point Location 31 14. Differences Between LiDAR and Field Based Elevations 37 15. Regression ofLiDAR and Field Cross section Elevations 38 16. Comparison of LiDAR and Field Longitudinal Profiles (5, 10,20 meters) 40 17. Regression ofField and LiDAR Based Slopes (5, 10,20 meters) 42 18. Differences Between LiDAR and Field Based Slopes (5, 10,20 meters) 44 19. Relationship of Water Surfaces to LiDAR Point Density 47 20. Marmot Dam: Orthophotographyand Colorized Slope Model 50 21. LiDAR Point Density versus Interpolation 53 LIST OF TABLES T~k p~ 1. Reported Accuracies of 2006 and 2007 LiDAR 24 2. Results of LiDAR and Field Elevation Comparison 38 3. Results ofLiDAR and Field Slope Comparison (5, 10,20 meters) 45 4. Results of Reach 1 Slope Comparison 46 5. Water Surface Roughness Results for Reach 1,2, and 3 48 6. Results of Reach 1 Water Surface Roughness Comparison 49 7. Subset of Reach 3 Water Surface Roughness Analysis Near Marmot Dam 50 x 1 CHAPTER I INTRODUCTION LiDAR (Light Detection and Ranging) has become a common tool for mapping and documenting floodplain environments by supplying individual point elevations and accurate Digital Terrain Models (DTM) (Bowen & Waltermire, 2002; Gilvear et aI., 2004; Glenn et aI., 2005; Magid et aI., 2005; Thoma, 2005; Smith et aI., 2006; Gangodagamage et aI., 2007). Active channel characteristics that have been extracted using LiDAR include bank profiles, longitudinal profiles (Magid et aI., 2005; Cavalli et aI., 2007) and transverse profiles of gullies under forest canopies (James et aI., 2007). To date, however, no one has tested if LiDAR returns from water surfaces can be used to measure local water surface slopes within the active channel. Much of the reason that researchers have not attempted to measure water surface slopes with LiDAR is because most LiDAR pulses are absorbed or not returned from the water surface. However, where the angle of incidence is close to nadir (i.e. the LiDAR pulse is fired near perpendicular to water surface plane), light is reflected and provides elevations off the water surface (Figure 1, Maslov et aI., 2000). Where LiDAR pulses glance the water surface at angles of incidence greater than 53 degrees, a LiDAR pulse is 2 more often lost to refraction (Figure 2) (Jenkins, 1957). In broken water surface conditions the water surface plane is angled, which produces perpendicular angles of incidence allowing for greater chance of return (Maslov et al. 2000). Su et al. (2007) documented this concept by examining LiDAR returns off disturbed surfaces in a controlled lab setting (Figure 3). LiDAR returns off the water surface potentially provide accurate surface elevations that can be used to calculate surface slopes. 1.0 08 ~ 0.6 o t5 ~ E .2 ~ 04 02 00 000 __d=2° d=10 ° --d=200 --d=300 d=40o d=50o I I 2000 4000 60.00 sensing angle, degree I 8000 Figure 1. Return Factor vs. LiDAR Scan Angle. Figure shows relationship between water surface return and scan angle. Return Factor versus sensing angle at different levels of the waving d (d = scan angle). Figure shows the relationship of scan angle of LiDAR to return from a water surface. Return factor is greatest at low scan angles relative to the nadir region of scan. (Maslov, D. V. et. al. (2000). A Shore-based LiDAR for Coastal Seawater Monitoring. Proceedings ofEARSeL-SIGWorkshop, Figure 1, pg. 47). 3 reflected\\ :.;/ incident 1 I 1 . '\ I lAIR \ •••••••• ••••••••••••• •••••• ••••••••••••••••••••• • •• eo ••••••••••• o •••••••••••• _0 •••••••••• 0 ••• .•.•.•.•.•.•00 ,••••• ' 0•••• 0 ••••••••••• 0 ••I' .•.•.•.•.•.,................. .".0 ••••••••••••• , •••••••••••• , ••••••••••0••••. .....................................~ . ••••••••••••••••••••••••••••••••••••• • •••••••••••••••••••••••••• 0 •••••••••••••••••••• 0 ••••• 0 •• ~~~)}))}))})))))))))\..)}))?()))))))))))))))))j((~j< Figure 2. Angle of Incidence. Figure displays concept of reflection and refraction of light according to angle of incidence. The intensity of light is greater as the angle of incidence approaches nadir. (Jenkins, F.A., White, RE. "Fundamentals of Optics". McGraw-Hili, 1957, Chapter 25) 09 08 0.7 0.6 0.5 0.4 0.3 0.2 0.1 r - 0.\ O,j/6Y3- -500 17.5 35 52.5 70 horizonral scanning dislancC(lllm) 0.9 0.8 0.7 06 0.5 0.4 0.3 0.2 0.1 a b Figure 3. Wave Action Relationship to LiDAR Echo. "LiDAR measurements of wake profiles generated by propeller at 6000 rpm (a) and 8000 rpm (b). Su's work definitively showed LiDAR's ability to measure water surfaces, and the relationship of wave action to capability of echo. From Su (2007) figure 5, p.844 . This study examines whether LiDAR can accurately measure water surface elevations and slopes. In order to address this topic, I assess the vertical accuracy of LiDAR and the effects of water surface roughness on LiDAR within the active channel. Findings shed light on the utility of LiDAR for measuring water surface slopes in different stream environments and methodological constraints to using LiDAR for this purpose. 4 5 CHAPTER II BACKGROlJND Water Surface Slope Water surface slope is a significant component to many equations for modeling hydraulics, sediment transport, and fluvial geomorphic processes (Knighton, 1999, Sing & Zang, in press). Traditional methods for measuring water surface slope include both direct and indirect methods. Direct water surface slope measurements typically use a device such as a total station or theodolite in combination with a stadia rod or drop line to measure water surface elevations (Harrelson, et ai., 1994, Western et ai., 1997). Inaccuracies in measurements stem from surface turbulence that makes it difficult to precisely locate the water surface, especially in fast water where flows pile up against the measuring device (Halwas, 2002). Direct survey methods often require a field team to occupy several known points throughout a reach. This is a time consuming process, especially if one wanted to document water surface slope along large portions of a river. This method can be dangerous in deep or fast water. 6 Indirect methods of water surface slope measurement consist of acquiring approximate water surface elevations using strand lines, water marks, secondary data sources such as contours from topographic maps, or hydraulic modeling to back calculate the water depth (USACE, 1993; Western et aI., 1997). Variable quality of data and modeling errors can lead to inaccuracies using these methods. The use of strand lines and water marks may not necessarily represent the peak flows or the water surface. Contours may be calculated or interpolated from survey points taken outside the channel area. The most commonly used hydraulic models are based on reconstruction of I-dimensional flow within the channel and do not account for channel variability between cross section locations. LiDAR water surface returns have a great deal of promise for improving measurement of water surfaces in several significant ways. LiDAR measurements eliminate hazards associated with surveyors being in the water. LiDAR also captures an immense amount of elevation data over a very short period of time, with hundreds of thousands of pulses collected within a few seconds for a single swath. Within this mass of pulses, hundreds or thousands of measurements off the water's surface may be collected depending on the nature of surface roughness, with broken water surfaces increasing the likelihood of measurements (Figure 3). In addition, most terrestrial LiDAR surveys collect data by flying multiple overlapping flight lines, thus increasing the number of returns in off nadir overlapping areas and the potential for returns from water surfaces. 7 The accuracy of high quality LiDAR measurements is comparable to field techniques. The relative variability of quality LiDAR vertical measurements typically ranges between 0.03-0.05 meters (Leica, 2007), where relative variability is the total range of vertical error within an individual scan on surface of consistent elevation. Lastly, LiDAR has the ability to collect water surface elevations over large stretches of river within a single flight of a few hours. LiDAR Measurements of Active Channel Features Recent studies evaluating the utility of LiDAR in the active channel environment have documented the effectiveness of using LiDAR DTMs to extract bank profiles. Magid et al. (2005) examined long term changes of longitudinal profiles along the Colorado River in the Grand Canyon. The study used historical survey data from 1923 and differenced topographic elevations with LiDAR data flown in 2000. LiDAR with three meter spot spacing was used to estimate water surface profiles based on the LiDAR elevations nearest to the known channel. Cavalli et al. (2007) extracted longitudinal profiles of the exposed bed of the Rio Cordon, Italy using 0.5 meter LiDAR DEM cells. This study successfully attributed LiDAR DEM roughness within the channel to instream habitats. Bowen and Waltermire (2002) found that LiDAR elevations within the floodplain were less accurate than advertised by vendors and sensor manufacturers. Dense vegetation within the riparian area prevented LiDAR pulses from reaching the 8 ground surface resulting in accuracies ranging 1-2 meters. Accuracies within unvegetated areas and flat surfaces met vendor specifications (l5-20cm). James et al. (2007) used LiDAR at 3 meter spot spacing to map transverse profiles of gullies under forest canopies. Results from this study showed that gully morphologies were underestimated by LiDAR data, possibly due to low density point spacing and biased filtering of the bare earth model. Today, point densities of 4-8 points/m2 are common and would likely alleviate some of the troubles found in this study. Additional studies have used LiDAR to extract geomorphic data from channel areas. Schumann et al. (2008) compared a variety of remotely sensed elevation models for floodplain mapping. The study used 2 meter LiDAR DEMs as topographic base data for floodplain modeling, and found that modeled flood stages based on the LiDAR DEM were accurate to within 0.35m. Ruesser and Bierman (2007) used high resolution LiDAR data to calculate erosion fluxes between strath terraces based on elevation. Gangodagamage et al. (2007) used LiDAR to extract river corridor width series, which help to quantify processes involved in valley formation. This study used a fixed water surface elevation and did not attempt to demonstrate the accuracy of LiDAR derived water surfaces. Green LiDAR also has been used to examine riverine environments. Green LiDAR functions much like terrestrial LiDAR (which uses an infrared laser) except that green LiDAR systems use green light that has the ability to penetrate the water surface and measure the elevation of the channel bed. Green LiDAR is far less common than terrestrial LiDAR and the majority of studies have been centered on studies of ocean shorelines. Wang and Philpot (2007) assessed attenuation parameters for measuring bathymetry in near shore shallow water, concluding that quality bathymetric models can be achieved through a number of post-processing steps. Hilldale and Raft (2007) assessed the accuracy and precision of bathymetric LiDAR and concluded that although the resulting models were informative, bathymetric LiDAR was less precise than traditional survey methods. In general, it is often difficult to assess the accuracy of bathymetric LiDAR given issues related to access of the channel bed at time of flight. 9 10 CHAPTER III STUDY AREA The study area is the Sandy River, Oregon, which flows from the western slopes ofMount Hood northwest to the Columbia River (Figure 4). Recent LiDAR data and aerial photography capture the variety of water surface characteristics in the Sandy River, which range from shooting flow to wide pool-riffle formations. The recent removal of the large run-of-river Marmot Dam upstream of the analysis sites has also generated interest in the river's hydraulics and geomorphology. 11 545000 ,·......,c' 550000 556000 560000 Washington, I 565000 -. Portland Sandy River .Eugene Oregon 570000 ooo '~" ooo ~ ooo~ • Gresham (""IIIII/hill /flIt'r Oregon Clack. fna County Marmot Dam IHillshaded area represents 2006 LiDAR extent. Ol1hophotography was collected only along the Sandy River channel within the LiDAR extent. 10 KiiomElt:IS t---+---+-~I--+--+----t-+--+---+----jl 545000 550000 555000 560000 565000 570000 Figure 4. Site Map. Site area map showing location of analysis reaches within the 2006 and 2007 LiDAR coverage areas. Olihophotography was also collected for the 2006 study, but was collected only along the Sandy River channel. 12 Floodplain longitudinal slopes along the Sandy River average 0.02 and reach a maximum of 0.04. The Sandy River has closely spaced pool-riffles and rapids in the upper reaches, transitioning to longer sequenced pool-riffle morphology in the middle and lower reaches. The Sandy River bed is dominated by sand. Cobbles and small boulders are present mostly in areas of riffles and rapids. Much of the channel is incised with steep slopes along the channel boundaries. The flow regime is typical of Pacific Northwest streams, with peak flows in the winter months ofNovember through February and in late spring with snowmelt runoff (Figure 5). Low flows occur between late September and early October. The average peak annual flow at the Sandy River station below Bull Run River (USGS 14142500) is 106cms. Average annual low flow for the same gauge is 13.9cms. 13 USGS 14142500 SRNDY RIVER BL~ BULL RUN RIVER, NR BULL RUN, OR 200 k.===_~~~=~~~=.......==",,=~-........==~ ~....J Jan 01Feb Ollar 01Rpr O:t1ay 01Jun 01Jul 01Rug OJSep 010ct 01Nov O:IJec 01 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 \ 11 ~I\\ ,1\ 1\ j\ 1"J'fn I\. I, ) \ , ,;' ) I I" 'I'•., I I' I' ] 30000 ~~-~----~-------------~-------, o ~ 20000 ~ 8'-. 10000 ~ Ql Ql ~ U '001 ~ ::::J U, Ql to 1000 to .= u Co? '001 Cl )- .....J. a: Cl Hedian daily statistic <59 years) Daily nean discharge --- Estinated daily nean discharge Period of approved data Period of provisional data Figure 5, Annual Hydrograph of Sandy River. US Geological Survey gaging station annual hydrograph of Sandy River, Oregon at Bull Run River. Data from http://waterdata.usgs.gov/or/nwis/annual/ Vegetation is mostly a mixture of Douglas fir and western red hemlock (Figure 6). Other vegetation includes palustrine forest found in the upper portions of the study area, and agricultural lands found in the middle and lower portions. Douglas fir and western red hemlock make up 87% of vegetated areas, palustrine forest 5%, and agricultural lands 5%, the remaining 3% is open water associated with the channel and reservoirs (Oregon GAP Analysis Program, 2002). The city of Troutdale, OR abuts the lower reaches of the Sandy River. Along this stretch of river Himalayan blackberry, an invasive species, dominates the western banks (Figure 7). The presence of Himalayan blackberry is significant because LiDAR has trouble penetrating through the dense clusters of vines. When this blackberry is close to the water's edge it is difficult to accurately define the channel boundary. 14 15 545000 550000 555000 560000 565000 570000 Reach 3 10 !'lock-W. Red Cedar Forest Grass-shrub-saplillg/Regeneratillg Forest Mixed Conifer/Mixed Deciduous Forest Open Water Palustrine Forest Red Alder-Big Leaf Maple Forest Urban oo o o~ 545000 550000 555000 560000 565000 570000 Figure 6. Oregon GAP Vegetation within Study Area. 1999 Oregon GAP Analysis data for Sandy River area. Map shows how the Sandy River area is dominated by Douglas fir forest with areas of palustrine forest and agricultural lands (Oregon Natural Heritage Program, 1999). 16 Figure 7. Photo of Himalayan Blackberry on Sandy River. Himalayan blackberry near mouth of the Sandy River March, 25th 2007. Photo by John English. This study focuses on three reaches of channel that represent a range of water surface conditions along the river. Reach 1 is a I80-m long pool-riffle reach located 3.7 river kilometers upstream from the mouth, and is where we collected field data shortly after the 2007 LiDAR flight (Figure 8a). The bed is sandy in this reach and can change dramatically during high flows. The bank full width of Reach 1 is approximately 108 meters at its widest point. At the downstream end of the riffle, the channel is constricted 17 by riprap placed along the banks as the river flows under a bridge. Vegetation comprises deciduous and conifer trees such as Douglas fir, hemlock, and cottonwoods. Blackberry is present along the channel, but is not so dense that it obscures the active channel boundary. 18 b. Figure 8. Reach 1 Site Area Map with Photo. Reach 1 site area. Top figure (a) shows approximate width at bank full and length of field data collections. Yellow circles represent points along stream margins where water surface elevations were surveyed. Bottom photo (b) looks downstream from total station location. 19 Reach 2 (Figure 9) is located approximately 23.5 kIn upstream from the mouth of the Sandy River and is 1,815 meters in length. The widest portion of channel at approximate bank full is 116m. The channel consists of a large meander with sinuosity of 1.38 and consists of six riffles and five pools spaced at regular intervals. The substrate consists of sands with small boulders and large cobbles dominating riffle areas. Cobbles and boulders have likely been introduced to the channel as a result of mass wasting. Douglas fir dominates along banks. 20 oJ> 0° 200 MetersO 0 ~~~~~~I O~~~OOO~ Figure 9. Reach 2 Site Area Map. Site map of Reach 2. Reach 2 contains 359 cross sections derived from LiDAR and 3,456 sample points. Inset map shows cross section sample locations derived from LiDAR and smooth/rough water surface delineations used in analysis. 21 Reach 3 is located 40.7km upstream from the mouth of the Sandy and is 2,815 meters in length (Figure 10). The widest portion of this section at approximate banle full is 88 meters. The upstream extent of the channel includes the supercritical flow of Marmot Dam. The channel is incised and relatively straight with a sinuosity of 1.08. Fine sands dominate the channel bed with some boulders likely present from mass wasting along valley walls. As with Reach 2, Douglas fir dominates bank vegetation along. 200 40) Inset mAp displays UDAR point I densily alol1g willl cross seellon Sanlpleing dala LiDAR cross section SAmple locations were used to eX1mcl poinl density values. 503 fOC I 000 '.1..Hrs 1-.,...--,.-+--=1..,=-,---4I--+-1---11 . Reach 3 Figure 10. Reach 3 Site Area Map. Site map of Reach 3. Inset map shows point LiDAR water surface points. Reach 3 contains 550 cross sections and 3,348 sample points. Visual examination of this map allows one to see how point density varies within the active channel. 22 CHAPTER IV METHODS Overview LiDAR data and orthophotography were collected in 2006 and additional LiDAR data were collected over the same area in 2007. Field measurements were obtained five days after the 2007 LiDAR flight in order to compare field measurements of water surface slope to LiDAR-based measurements. Time of flight field measurements of water surface elevations were not obtained for the 2006 flight, but the coincident collection of LiDAR data and orthophotos provide a basis for evaluating variability of LiDAR-based slopes over different channel types as identified from aerial photos. Following sections provide more detail regarding these methods. 23 LiDAR Data and Image Acquisition All LiDAR data were collected using a Leica ALS50 Phase II LiDAR system mounted on a Cessna Caravan C208 (see Table 1 for LiDAR acquisition specifications). The 2006 LiDAR data were collected October 2211d and encompassed 13,780 hectares of high resolution (2':4 points/m2 ) LiDAR data from the mouth of the Sandy River to Marmot Dam. Fifteen centimeter ground resolution orthophotography was collected September 26th , 2006 along the riparian corridor of the Sandy River from its mouth to just above the former site ofMarmot dam (Figure 4). The 2007 LiDAR were collected on October 8th and covered the same extent as the 2006 flight, but did not include orthophotography. Data included filtered XYZ ASCII point data, LiDAR DEMs as ESRI formatted grids at 0.5 meter cell size. Data were collected at 2':8 points per m2 providing a data set with significantly higher point density than the 2006 LiDAR data. The 2006 LiDAR data were collected in one continuous flight. 2006 orthophotography was collected using an RC30 camera system. Data were delivered in RGB geoTIFF format. LiDAR data were calibrated by the contractor to correct for IMU position errors (pitch, roll, heading, and mirror scale). Quality control points were collected along roads and other permanent flat features for absolute vertical correction of data. Horizontal accuracy ofLiDAR data is governed by flying height above ground with horizontal accuracy being equal to 1I3300th of flight altitude (meters) (Leica, 2007). 24 Table 1. Reported Accuracies of 2006 and 2007 LiDAR. Reported Accuracies and conditions for 2006 and 2007 LiDAR data. (Watershed Sciences PGE LiDAR Delivery Report, 2006, Watershed Sciences DOGAMI LiDAR Delivery Report, 2007). Relative Accuracy is a measure of flight line offsets resulting from sensor calibration. 2006 LiDAR 2007 LiDAR Flying height above ground level meters (AGL) 1100 1000 Absolute Vertical Accuracy in meters 0.063 0.034 Relative Accuracy in meters (calibration) 0.058 0.054 Horizontal Accuracy (l/3300th * AGL) meters 0.37 0.33 Discharge @ time of flight (cms) 13.05 20.8 - 21.8 LiDAR data collection over the Reach 1 field survey location was obtained in a single flight on October 8, 2007 between 1:30 and 6:00 pm. During the LiDAR flight, ground quality control data were collected along roads and other permanent flat surfaces within the collection area. These data were used to adjust for absolute vertical accuracy. Field Data Acquisition A river survey crew was dispatched at the soonest possible date (October 13, 2007) after the 2007 flight to collect ground truth data within the Reach 1. The initial aim was to survey water surface elevations at cross sections of the channel, but the survey was limited to near shore measurements due to high velocity conditions. We collected 187 measurements of bed elevation and depth one to fifteen meters from banks along both sides of the channel (Figure 8a) using standard total station longitudinal profile 25 survey methods (Harrelson, 1994). Seventy-six and 98 measurements were collected along the east and west banks, respectively, at intervals of approximately 1 to 2 meters. Thirteen additional measurements were collected along the east bank at approximately ten meter intervals. Depth measurements were added to bed elevations to derive water surface elevations. Discharge during the survey ranged between 22.5 and 22.7 cms during the survey of the east bank and remained steady at 22.5 cms during the survey of the west bank (USGS station 14142500). LiDAR Processing The goal ofLiDAR processing for this project was to classify LiDAR point data within the active channel as water and output this subset data for further analysis. The LiDAR imagery was first clipped to the active channel using a boundary digitized from the 2006 high resolution orthophotography. LiDAR point data were then reclassified to remove bars, banks, and overhanging vegetation (Figure 11). 26 Figure 11. LiDAR Point Filtering Processing Step. LiDAR processing steps. Top image shows entire LiDAR point cloud clipped to active channel boundary. Lower image shows the final processed LiDAR points representing only those points that reflect off the water surface. All bars and overhanging vegetation have been removed as well. 27 Water points were classified using the ground classification algorithm in Terrascan© (Soininen, 2005) to separate water surface returns from those off of vegetation or other surfaces elevated above the ground. The classification routine uses a proprietary mathematical model to accomplish this task. Once the ground classification was finished, classified points were visually inspected to add or remove false positives and remove in-channel features such as bar islands. A total of 11,593 of 1,854,219 LiDAR points were classified as water. Points classified as water were output as comma delimited x,y,z ASCII text files (XYZ), then converted to a 0.5 meter linearly interpolated ESRI formatted grid using ESRI geoprocessing model script. Calculation of Water Surface Slopes Water surface slopes were calculated using the rise over run dimensionless slope equation where the rise is the vertical difference between upstream and downstream water surface elevations and run is the longitudinal distance between elevation locations. LiDAR data is typically used in grid format. For this reason grid data were used for calculation of water surface slopes. We used linear interpolation to grid the LiDAR point data as this is the standard method used by the LiDAR contractor. In order to compare the LiDAR and field data it was also necessary to interpolate field 28 measurements to create a water surface for the entire stream. The field data-based DEM was created using kriging interpolation within ArcGIS Desktop Spatial Analyst (Figure 12). No quantitative analysis was performed to evaluate the interpolation method of the field-based water surface. The kriging interpolation was chosen because it producex the smoothest water surface based on visual inspection when compared to linear and natural neighbor interpolations, which generated irregular fluctuations that were unrealistic for a water surface. The kriged surface provided a water surface elevation model for comparative analysis with LiDAR. 29 Figure 12. Field DEM Interpolated using Kriging. Field DEM interpolated from field survey points using kriging method found in ArcGIS Spatial Analyst. DEM has been hiIlshaded to show surface characteristics. The very small differences in water surface elevations generate only slight variations in the hillshadeing. To compare LiDAR and field-based water surface slopes, water surface elevations from the LiDAR and field-based DEMS were extracted at the same locations along Reach I. To accomplish this, 37 cross sections were manually constructed at approximately Sm spacings (Figure 13). Cross sections comparisons were used rather than point-to-point comparisons between streamside field and LiDAR data points because the cross sections provide water surface slopes that are more representative of the entire channel. The Sm interval spacing was considered to be a sufficient for fine resolution slope extraction. Because cross section center points were used to calculate the longitudinal distance and because the stream was sinuous, the projection of the cross sections from the center line to the banks led to stream side distances between cross sections that differed from Sm. 30 31 Smooth 125 Meters I 100 I 75 I 50 I 25 I Cross Sections Cross Section Data Roughness Delineation Cross Section Sample Locations _ Rough oI ~ each 1 Figure 13. Reach 1 LiDAR Cross Sections and Sample Point Locations. Reach I LiDAR-derived cross section sample locations and areas of smooth and rough water surface delineations. 37 cross section and 444 sample points lie within Reach 1. 32 Cross sections were extracted using a custom ArcObjects VBA script (Appendix A). This script extracted 1 cell nearest neighbor elevations along the transverse cross sections at 5 meter intervals creating 444 cross section sample locations (Figure 13). Cross section averages were calculated using field-based and LiDAR-based elevation water surface grids. The average cross sectional elevation value for field and LiDAR data were then exported to Excel files, merged with longitudinal distance between cross section, and used to calculate field survey-based and LiDAR-based slopes between cross sections. Reaches 2 and 3, for which only LiDAR data were available, were sampled using the same cross sectional approach used in Reach 1. The data extracted from these reaches were used to characterize how LiDAR-based elevations, slopes and point densities interact with varying water surface roughness. Within Reach 2, 359 cross sections were drawn and elevations were sampled every five meters along each cross section creating 3,456 cross section sample locations (Figure 9). Reach 3 contained 550 cross sections and 3,348 cross section sample locations (Figure 10). Slopes were calculated between each cross section. 33 Evaluating LiDAR Slope Accuracies and Controls The accuracy of elevation data is the major control on slope accuracy, so a comparative analysis was performed using field survey and LiDAR elevations. First, field-based and LiDAR slopes were calculated at distance intervals of five, ten and twenty meters using average cross section elevations to test the sensitivity of the slopes to vertical inaccuracies in the LiDAR data. The field and LiDAR elevations were differenced using the same points used to create average cross section elevations. Differences were plotted in the form of histogram and cumulative frequency plot after transforming them into absolute values. Descriptive statistics were calculated to examine the range, minimum, maximum, and mean offset between data sets. Finally LiDAR and field-based values were compared using regression analysis. This study also examined the effects of water surface roughness on LiDAR elevation measurements, LiDAR point density, and LiDAR derived water surface slopes. Each reach was divided into smooth and rough sections based on visual analysis of the orthophoto data. One-meter resolution slope rasters were created from the LiDAR water surface grids using ArcGIS Spatial Analyst. One meter resolution point density grids were created from LiDAR point data (ArcGIS Spatial Analyst). Using the cross section sample points, values for water surface type, elevation, slope, and point density were extracted within each reach. Point sample data were transferred to tabular format, and average values were generated for each cross section. These tables were used to calculate 34 descriptive statistics associated with water surfaces such as elevation variance, average slope variance, average point density, and average slope. It is assumed in this study that smooth water surfaces are associated with pools and thus ought to have relatively low slopes. Conversely rough water surfaces are assumed to be representative of riffles and rapids, and thus ought to have relatively steeper slopes. Reach 1 contains field data, so slopes from LiDAR and field data were compared with respect to water surface conditions as determined from the aerial photos. 35 CHAPTER V RESULTS Results of this study encompass three analyses. Elevation analysis describes the statistical difference between LiDAR and field-based water surface elevations for Reach 1. Slope analysis compares LiDAR derived and field-based slopes calculated at 5, 10, and 20m longitudinal distances. These analyses aim to quantify both slope accuracy and slope sensitivity. Lastly, water surface analysis examines the relationship between LiDAR measured water surface slopes, point density, and water surface roughness. Comparison of Absolute Elevations from Field and LiDAR Data in Reach 1 The difference between water surface elevations from LiDAR affects the numerator within the rise over run equation, which in tum affects slope. This elevation analysis evaluation quantifies differences between field and LiDAR data. LiDAR-based cross section elevations were differenced from field-based cross section elevations. Difference values were examined through statistical analysis. 36 In terms of absolute elevations relative to sea level, the majority of LiDAR-based water surface elevations were lower than field-based elevations, although the LiDAR elevations were higher in the upper portion ofReach 1. Differences ranged between -0.04 and 0.05m with a mean absolute difference between field and LiDAR elevations of 0.02m (Figure 14 and Table 2). The range of differences is within the expected relative accuracies of LiDAR claimed by the LiDAR provider. Elevations for field and LiDAR data are significantly correlated with an R2 of 0.94 (Figure 15). The negative offset was expected given that discharge at time of LiDAR acquisition was lower than discharge at time of field data acquisition. Discharge during field acquisition ranged between 22.5 and 22.7 cfs, while discharge during LiDAR acquisition was between 20.8 and 21.8cfs. The portion of Reach 1 where LiDAR water surface measurements were higher than field measurements may be related to difference in discharge or change in bed configuration. Overall results showed that LiDAR data and field-based water surface measurements are comparable. 37 Distribution of Elevation Differences Between Field and LiDAR Water Surfaces 10 9 8 7 >. 6 u r:: ell 5 :l C'" ~ 4 u.. 3 2 0+---+ -0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 More Elevation Difference, Field - L1DAR (m) Figure 14. Differences Between LiDAR and Field Based Elevations. Elevation difference statistics between cross sections derived from field and LiDAR elevation data. Positive differences indicate that field-based elevations were higher than LiDAR; negative differences indicate LiDAR elevations were higher. Values on x axis represent minimum difference within range. For example, the 0.01 category includes values ranging from 0.01 to 0.0199. y-1.18x-1.03 .... R2 =0.94 ""..,; I •• ./... ./ .- ./ • ./ • ./. /""I ./iI ../. _._~. -? , 38 Table 2. Results of LiDAR and Field Elevation Comparison. Descriptive and regression statistics for absolute difference lField - LiDARI values between cross section elevations. All units in meters. Sample size is 37. Mean 0.028 Median 0.030 Standard Deviation 0.013 Kurtosis -0.640 Skewness -0.484 Range of difference 0.093 Minimum difference 0.002 Absolute maximum difference 0.047 Confidence Level(95.0%) (m) 0.004 Elevation Comparison of Field and LiDAR Water Surface Elevations 5.72 5.70 ~_ 5.68 g 5.66 :0:; I1l 5.64 > iii 5.62 ell 5.60 () ~ 5.58 ~ 5.56 ~ 5.54 1\1 5.52 ~ IX 5.50 ed (x) and LiDAR-based (y) cross section elevations. 39 Comparison of longitudinal profiles offield and LiDAR water surfaces shows a clear relationship in overall shape (Figure 16), capturing similar trends in longitudinal profiles. Figure 16 shows field and LiDAR profiles become more similar in shape as distance between cross sections increases. In terms of overall shape, the greatest differences occur in the upper 30 m, where LiDAR-based profiles demonstrate a higher slope than do field-based measurements. Because of the five day lag between LiDAR and field measurements in this mobile bed stream, it is impossible to know the degree to which this difference represents error in measurements or real change in the system. 40 5 meter Longitudinal Profile Comparison 20 40 60 80 100 120 140 160 180 5.75 .s 5.70 ~" _ • •• • :. 5 Longitudinal Distance Down Stream (m) A 10 meter Longitudinal Profile Comparison 5.75 5.70 . • [,.10 rreler Field Profile! I: I 5.65 • • , . • • • 10 rreter LiDAR Profile • • • I:: 5.60 • • 0 :;:; • • >Cll 5.55 • • ~ • • w 5.50 • • • • • • • • • 5.45 5.40 0 20 40 60 80 100 120 140 160 180 Longitudinal Distance Down Stream (m) B 20 meter Longitudinal Profile Comparison 5.75 5.70 • ,. 20 Cll 5.55 •• Q) W 5.50 •• • , 5.45 . 5.40 0 20 40 60 80 100 120 140 160 180 Longitudinal Distance Down Stream (m) C Figure 16. Comparison of LiDAR and Field Longitudinal Profiles (5, 10, 20 meters). Longitudinal profiles of a) 5 meter, b) 10 meter, and c) 20 meter cross section elevations. 41 Slope Comparisons Slope in this study is calculated as the dimensionless ratio of rise over run. As noted in the Methods section, slopes were calculated over three different horizontal intervals to test the sensitivity of the LiDAR's internal relative accuracy. Differences in Sm LiDAR and field-based slopes derived from cross sections reveal substantial scatter (Figure l7a), although they clearly covary. Ten meter interval slopes show a stronger relationship (Figure 17b), while slopes based on cross sections spaced 20 m apart have the strongest relationship (Figure l7c). The slope associated with regression of field and LiDAR elevation data is not approximately 1 as one might expect. This is because LiDAR elevations are higher than field elevations at the upstream end of the reach, and lower at the downstream end. 42 5m Slope Comparison -c: ~ -0:: Q) (/l ~ ~.01 Q) C. .2 en 0:: « 0 ::i A -c: ~ 0:: --Q) (/l i2 -0.01 Q) C. 0 en 0:: « 0 ::i B 0.004 = 0.58x - 0.001 R2 = 0.38 ~.008 -0.008 Field Slope (Rise/Run) 10 meter Slope Comparison 0.004 y = 0.63x - 0.001 R2 = 0.51 -0.008 -0.008 Field Slope (Rise/Run) 20 meter Slope Comparison • 0.004 0.002 0.004 C :::l -0:: Q) (/l i2 ~.01 -Q) c. o Ci5 0:: « o~ 0.004 =0.66x - 0.001 R2 = 0.80 ~.008 ~.006 -0.008 Field Slope (Rise/Run) 0.002 0.004 C Figure 17. Regression of Field and LiDAR Based Slopes (5,10,20 meters). Scatter plots showing comparisons between slope values calculated at distance intervals of a) 5 meters, b) 10 meters, and c) 20 meters. 43 Figure 18 shows how the range of differences between LiDAR and field-based water surface slopes decrease as longitudinal distance increases. Five meter slope differences ranged between -0.004 and 0.004 (Figure 18a). Ten meter slope differences ranged between -0.002 and 0.003 (Figure 18b). Twenty meter slope differences ranged between 0 and 0.002 (Figure 18c). 44 Differences of Slope at 5m Between Field and LiDAR 10 » 8 0c Ql 6 :J 0" 4 .Q..l u. 2 0 SIll>< SIl"> SIll\- ~<::J <;:><::J <;:><::J SIl" ~ SIl" SIll\- SIl"> SIll>< ~/l, r;:,<::J ~'::; ~'::; ~'::; ~'::; ~o Slope Difference (Field-LiDAR) A Differences of Slope at 10m Between Field and L1DAR 7 6 ~ 5 lii 4 :J 0" 3 ~ u. 2 1 o +---+--~--;..J SIll>< ~<::J Slope Difference (Field-LiDAR) B Differences of Slope at 20m Between Field and LiDAR 4 ~~I\- ~~" ~ ~~" ~~I\- ~~"> ~~I>< o"/l, <;:>.~. ~.~.~.~. ~ Slope Difference (Field-LiDAR) o +---+--+--+--t- SIll>< SIl"> <;:><::J <;:><::J ~ 3 c Ql :J 2 0" ~ U. C Figure 18. Differences Between LiDAR and Field Based Slopes (5, 10,20 meters). Histogram charts showing difference values between field and LiDAR derived slopes at a) 5 meter slope distances, b) 10 meter slope distances, and c) 20 meter slope distances. 45 The mean difference between slopes decreases from 0.0017 to 0.0007 as slope distance interval is increased. Maximum slope difference and standard deviation of offsets decrease from 0.001 to 0.0005 and 0.0047 to 0.0014 respectively. Regression analysis of these data show a significant relationship for all three comparisons, and adjusted R2 increased from 0.357 to 0.763 with slope distance interval (Table 3). Table 3. Results of LiDAR and Field Slope Comparison (5, 10,20 meters). Descriptive and regression statistics for offsets between field and LiDAR derived slope values (Field minus LiDAR). Slope values are dimensionless rise / run. All data is significant at 0.01. Distance Interval 5m 10m 20m Mean 0.0017 0.0012 0.0007 Standard Deviation 0.0010 0.0007 0.0005 Range of Difference 0.0080 0.0047 0.0024 Minimum difference 0.0000 0.0000 0.0001 Maximum difference 0.0047 0.0026 0.0015 Count 36 16 8 Adjusted R squared 0.36 0.47 0.76 Water surface slope for the entire length of Reach 1 (l59.32m) was compared and yielded a difference of 0.0005. This difference is smaller (by 0.0002) than the difference between 20 meter slope (Table 4). Slope was calculated by differencing the most upstream and downstream cross sections and dividing by total length of reach. Differences between LiDAR and field-based slopes may represent real change due to the five day lag between data sets and difference in discharge. 46 Table 4. Results of Reach 1 Slope Comparison. Comparison of slopes calculated using the farthest upstream and downstream cross section elevation values. Slope values have dimensionless units stemming from rise over run. Upper Lower Reach Elevation (m) Elevation (m) Len2th (m) Slope Field 5.652 5.491 159.32 -0.0010 LiDAR 5.697 5.455 159.32 -0.0015 Surface Roughness Analysis Water surface condition was characterized as smooth or rough based on 2006 aerial photography (Figure 19). Surface roughness was examined to understand its effect on LiDAR data within the active channel, as well as LiDAR's ability to potentially capture difference in water surface turbulence. Table 5 shows statistics with relation to water surface condition for all three reaches. 47 Figure 19. Relationship of Water Surfaces to LiDAR Point Density. 2006 aerial photos were used to delineate rough and smooth water surfaces. Image on left shows a transition between rough water surface (seen as white water) and smooth water surface (seen as upstream pool). Image on right shows LiDAR point density in points per square meter. In all reaches point density, variance of elevations, and water surface slopes were significantly higher in rough surface conditions. These results indicate that LiDAR point density is directly related to the roughness of a water surface and that is capturing the rough water characteristics one would expect in areas where turbulence generates surface waves. 48 Table 5. Water Surface Roughness Results for Reach 1,2, and 3. Water surface statistical output for rough and smooth water surface of Reaches 1, 2, and 3. Results within table represent average values for each Reach. Slope values have dimensionless units from rise over run equation derived from ESRI generated slope grid. Point density values based on points/m2 • Elevation variance in meters. Reach 1 Reach 2 Reach 3 Rou~h water No. of Sample Points 153 1981 1968 Avg Slope -0.013 -0.011 -0.007 Point Density (pts/mL ) 1.195 1.002 1.217 Elevation Variance (m) 0.003 0.018 0.041 Smooth water No. of Sample Points 290 1474 1378 Avg Slope 0.0075 -0.0006 -0.0033 Point Density (pts/mL ) 0.149 0.550 0.480 Elevation Variance (m) 0.001 0.0077 0.024 Within Reach 1, cross section elevations were separated into rough and smooth water conditions and slopes were calculated using field and LiDAR data sets (Table 6). Again, results showed that rough water surfaces have greater slopes than smooth water surfaces. The smooth water surface of Reach 1 yielded a larger discrepancy between field and LiDAR derived slopes compared to rough water surface. This is because small differences between LiDAR and field elevations generate larger proportional error in the rise / run equation when total elevation differences between upstream and downstream are small. 49 Table 6. Results of Reach 1 Water Surface Roughness Comparison. Reach 1 water surface roughness slope analysis. Reach 1 was divided into smooth and rough water surfaces based upon visual characteristics present in aerial photography. Slopes were calculated for each area and compared with field data to examine accuracy. Surface Reach Upper Lower Slope Type Lenl!th (m) Elevation (m) Elevation (m) Slope Difference Field Smooth 83.11 5.652 5.642 -0.0001 N/A LiDAR Smooth 83.11 5.697 5.612 -0.0010 0.0009 Field Rough 71.73 5.635 5.491 -0.0020 N/A LiDAR Rough 71.73 5.592 5.455 -0.0019 -0.0001 Prior to collections of the 2007 data, Reach 3 contained the former Marmot Dam that was dismantled on October 19th , 2007 (Figure 20). The areas at and directly below the dam are rough water surfaces. The super critical flow at the dam yielded a slope of - 0.896 (Table 7). The run below the dam contained low slope values of less than -0.002. Both the dam fall and adjacent run yielded high point densities of greater than 2 points per square meter. 50 Cross Sections o Cross Section Sample Locations L1DAR derived Slope Model Value Higll 178814133 25 50 75 100 125 150 ~.',eters I I I I I I La,·, 0003936 Figure 20. Marmot Dam: Orthophotography and Colorized Slope Model. Mannot Dam at far upstream portion of Reach 3. Image on left shows dam site in 2006 orthophotography. Image on right shows the increase in slope associated with the dam. Marmot Dam was removed Oct. 19th , 2007. Table 7. Subset of Reach 3 Water Surface Roughness Analysis Near Marmot Dam. Subset of Reach 3 immediately surrounding Marmot Dam roughness analysis containing values for Mannot Dam. The roughness results fell within expectations showing increases in slope at the dam fall and high point densities at the dam fall and immediate down stream run. Habitat Type Avg Slope Point Density Point Density Variance Dam Fall -0.896 2.284 1.003 Dam Run -0.001 2.085 5.320 51 CHAPTER VI DISCUSSION The elevation analysis portion of this study shows that LiDAR can provide water surface profiles and slopes that are comparable to field-based data. The differences between LiDAR and field based measurements can be attributed to three potential sources. The first is the relative accuracy of the LiDAR data which has been reported between O.05m and O.06m by the vendor. The second source can be associated with the accuracy of field based measurements which are similar to the relative accuracy of the LiDAR (O.03m-O.05m). Lastly, the discharge differed between field data collection and LiDAR collection by O.02cms. It is possible that much of the O.05m difference observed through most of the Reach 1 profile (Figure 16) could be attributed to the difference in discharge and changes in bed configuration, but without further evidence, the degree of difference due to error or real change cannot be identified. Even if one attributes all the difference to error in LiDAR measurements, the overall correspondence ofLiDAR and field measurement (Figure 15 and 16) indicates that LiDAR-based surveys are useful for many hydrologic applications. 52 In the upper portion of the reach, the profiles display LiDAR elevations that are higher than the field data elevations, whereas the reverse is true at the base of the reach. This could be a function of difference in discharge between datasets, change in bed configuration, or an artifact of low point density. Low density of points forces greater lengths of interpolation between LiDAR points leading to a coarse DEM (Figure 21). Overall, the analysis Reach 1 profile indicates that LiDAR was able to match the fieldbased elevation measurements within ±O.05m. 53 Rough & Smooth Wa~t:e:-r~S~u=rf;:a~c:e:s~rz~~J,;~~ Grid Interpolation in Low Point Density Figure 21. LiDAR Point Density versus Interpolation. Side by side image showing long lines of interpolation associated with smooth water surfaces (right image). Smooth water surfaces tend to have low LiDAR point density. The image on the right shows a hillshade ofthe LiDAR DEM. The DEM has been visualized using a 2 standard deviation stretch to highlight long lines of interpolation. The comparability of LiDAR and field-based slopes showed a significant trend with increasing downstream distances between cross sections. Adjusted R2 values increased from 0.36 to 0.76 and the range of difference between field and LiDAR based slopes decreased from 0.0047 to 0.00 14 as longitudinal distance increased from 5 to 20- 54 m. This suggests that the 0.05m of expected variation of LiDAR derived water surface elevation has less effect on water surface slope accuracy as distance between elevation measurements points increases. Likewise, slopes accuracies along rivers with low gradients will improve as the longitudinal distance between elevation points increases. Overall, data has shown that LiDAR can measure water surface slopes with mean difference relative to field measurements of 0.017, 0.012, and 0.007 at horizontal distances of 5, 10, and 20 meters respectively. Although the discrepancy between field and LiDAR-based slopes is greatest at 5-m intervals, the overall slopes (Fig 17) and longitudinal profiles (Fig 16) even at this distance generally correspond. The use of a 5m interval water surface slope as a basis for comparison is really a worst case example, as water surface slopes are usually measured over longer reach scale distances where the discrepancy between LiDAR and field-based measurements is lower. The continuous channel coverage and accuracies derived from LiDAR represent a new level of accuracy and precision in terms of spatial extent and resolution of water surface slope measurements. Analysis of surface roughness found that rough water surfaces had significantly higher point densities than smooth water surfaces. Rough water surfaces averaged at least 1 point/m2 , while smooth water surfaces averaged less than 1 point/2m2 • Longitudinal profiles of Reach 1 indicate the most accurate water surface measurements occur in areas of higher point density (Fig. 16). Future applications that attempt to use 55 LiDAR to measure water surface slope ought to sample DEM elevations from high point density areas of channel. Water surface analysis also showed trends relating water surface roughness and slope. Rough water surfaces for all three analysis reaches averaged larger average slope values than smooth water surfaces. This is because rough water surfaces are commonly associated with steps, riffles, and rapids. All three of these habitat types are areas have higher slopes than smooth water habitats. Smooth water surfaces are commonly associated with pools or glides, which would be areas of lower slope. Future research should examine the potential for using LiDAR to characterize stream habitats based on in-stream point density and slope. This study is not without its limitations. The field area used to test the accuracy of LiDAR is only representative of a small portion of the Sandy River. Comparisons of field and LiDAR data would be improved by having mid-channel field data. One might also question the use of field based water surface slopes as control for measuring "accuracy". Water surface slope is difficult to measure for reasons stated earlier in this paper. One might make the argument that there is no real way to truly measure LiDAR accuracy of water surface slope, and that LiDAR and field based measurements are simply comparable. In this context, LiDAR holds an advantage over field based measurements given its ability to measure large sections of river in a single day. LiDAR has a distinct advantage over traditional methods of measurement in that measurements are returned from the water surface, and consequently not subject to errors 56 associated with variability of surface turbulence piling up against the measuring device. LiDAR can also capture long stretches of channel within a few seconds reducing the influence of changes in discharge. LiDAR data in general does have its limitations. LiDAR data are only as accurate as the instrumentation and vendor capabilities. LiDAR must be corrected for calibrations and GPS drift to create a reliable data set, and not all LiDAR vendors produce the same level of quality. LiDAR data may be more accurate in some river reaches than others. The study reaches of this study contained well defined open channels, which made identifying LiDAR returns off the water surface possible. Both LiDAR data sets were collected at low flows. Flows that are too low or channels that are too narrow may limit ability to extract water surface elevations because of protruding boulders or dense vegetation that hinders accurate measurements. In some cases vegetation within and adjacent to the channel may interfere with LiDAR's ability to reach the water surface. Researchers should consider flow, channel morphology, and biota when obtaining water surface slopes from LiDAR. 57 CHAPTER VII CONCLUSION This paper examined the ability of LiDAR data to accurately measure water surface slopes. This study has shown that LiDAR data provides sufficiently accurate elevation measurements within the active channel to accurately measure water surface slopes. Measurement of water surface slope with LiDAR provides researchers a tool which is both more efficient and cost effective in comparison with traditional field-based survey methods. Additionally, analysis showed that LiDAR point density is significantly higher in rough surface conditions. Water surface elevations should be gathered from high point density areas as low point density may hinder elevation accuracy. Channel morphology, gradient, flow, and biota should be considered when extracting water surface slopes as these attributes influence water surface measurement. Further study should examine accuracy of LiDAR derived water surface slopes in channel morphologies other than those in this study. Overall, the recognition that LiDAR can accurately measure water surface slopes allows researchers an unprecedented ability to study hydraulic processes for large stretches of river. Common: APPENDIX ARCGIS VBA SCRIPT CODE 58 Public g---.pStrmLayer As ILayer ' stream centerline layer selected by user (for step 1) Public g_StrearnLength As Double ' stream centerline length (for step 1) Public g_InputDistance As Integer 'As Double 'distance entered by user (for step 1) Public g_NumSegments As Integer I number of sample points entered by user (for step 1) Public gyPointLayer As ILayer I point layer created from stream centerline (for step 1) Public g]ntShpF1Name As String I point layer pathname (for step 1) Public gyMouseCursor As IMouseCursor 'mouse cursor Public g_LinearConverson As Double I linear conversion factor Public gyDEMLayer As IRasterLayer I DEM layer (for steps 3 and 4) Public g_DEMConvertUnits As Double I DEM vertical units conversion factor (for steps 3 and 4) Public g_MaxSearchDistance As Double 'maximum search distance (for step 4) Public L NumDirections As Integer I number of directions to search in (for step 4) Public g_SampleDistance As Double 'sample distance (for step 5) Public g_SampleNumber As Double ' total sample points (for step 5) Public g_VegBeginPoint As Boolean I where to start the calucaltion (for step 5) Public g_VegCaclMethod As Boolean 'which method for Vegetation Calculation (for step 5) Public gyContribLayer As ILayer ' contributing point layer (for step 6) Public gyReceivLayer As ILayer 'receiving point layer (for step 6) Public gyOutputLayerName As String I output shapefile (for step 6) Function VerifyField(fLayer As ILayer, fldName As String) As Boolean I verify that topo fields are in the stream centerline point layer Dim pFields As IFields Dim pField As IField Dim pFeatLayer As IFeatureLayer Dim pFeatClass As IFeatureClass Set pFeatLayer = fLayer Set pFeatClass = pFeatLayer.FeatureClass Set pFields = pFeatClass.Fields For i = 0 To pFields.FieldCount - 1 Set pField = pFields.Field(i) 'MsgBox pField.Name IfpField.Name = fldName Then VerifyField = True Exit Function End If Next VerifyField = False End Function Function Ca1cPointLatLong(inPnt As IPoint, inLayer As ILayer) As IPoint , in point layer Dim pFLayer As IFeatureLayer Set pFLayer = inLayer , spatial reference environment Dim pInSpatialRef As ISpatialReference Dim pOutSpatialRef As ISpatialReference Dim pGeoTrans As IGeoTransformation Dim pInGeoDataset As IGeoDataset Set pInGeoDataset = pFLayer Dim pSpatRefFact As ISpatialReferenceFactory , get map units of shapefile spatial reference Dim pPCS As IProjectedCoordinateSystem Set pPCS = pInGeoDataset.SpatialReference 'set spatial reference environment Set pSpatRefFact = New SpatialReferenceEnvironment Set pInSpatialRef= pInGeoDataset.SpatialReference 'MsgBox pInSpatialRef.Name Set pOutSpatialRef= pSpatRefFact.CreateGeographicCoordinateSystem(esriSRGeoCS_WGS1984) Set pGeoTrans = pSpatRefFact.CreateGeoTransformation(esriSRGeoTransformation_NADI983_To_WGS1984_1) Dim pOutGeom As IGeometry2 Set Ca1cPointLatLong = New Point Set CalcPointLatLong.SpatialReference = pInSpatialRef Ca1cPointLatLong.PutCoords inPnt.X, inPnt.Y Set pOutGeom = Ca1cPointLatLong pOutGeom.ProjectEx pOutSpatialRef, esriTransformForward, pGeoTrans, 0, 0, ° 'MsgBox inPnt.X &" "& inPnt.Y & vbCrLf& Ca1cPointLatLong.X &" "& Ca1cPointLatLong.Y End Function Sub OpenGxDialogO Dim pGxdial As IGxDialog Set pGxdial = New GxDialog pGxdial.ButtonCaption = "OK" pGxdial.Title = "Create Stream Centerline Point Shapefile" pGxdial.RememberLocation = True Dim pShapeFileObj As IGxObject Dim pGxFilter As IGxObjectFilter Set pGxFilter = New GxFilterShapefiles 'e.g shp Set pGxdial.ObjectFilter = pGxFilter If pGxdial.DoModaISave(ThisDocument.Parent.hWnd) Then Dim pLocation As IGxFile Dim fn As String 59 Set pLocation = pGxdial.FinalLocation fn = pGxdial.Name End If If Not pLocation Is Nothing Then LPntShpFlName = pLocation.Path & "\" & fn frmlB.tbxShpFileName.Text = g]ntShpFlName frmlB.cmdOK.Enabled = True End If End Sub Function GetAngle(pPolyline As IPolyline, dAlong As Double) As Double Dim pi As Double pi = 4 * Atn(l) Dim dAngle As Double Dim pLine As ILine Set pLine = New Line pPolyline.QueryTangent esriNoExtension, dAlong, False, 1, pLine , convert from radians to degrees dAngle = (180 * pLine.Angle) / pi I adjust angles , ESRI defines 0 degrees as the positive X-axis, increasing counter-clockwise I Ecology references 0 degrees as North, increasing clockwise If dAngle <= 90 Then GetAngle = 90 - dAngle Else GetAngle = 360 - (dAngle - 90) End If End Function Function FeatureExists(strFeatureFileName As String) As Boolean On Error GoTo ErrHandler: Dim pWSF As IWorkspaceFactory Set pWSF = New ShapefileWorkspaceFactory Dim pFeatWS As IFeatureWorksiJace Dim pFeatDS As IFeatureClass Dim strWorkspace As String Dim strFeatDS As String strWorkspace = SplitWorkspaceName(strFeatureFileName) & "\" strFeatDS = SplitFileName(strFeatureFileName) If PWSF.IsWorkspace(strWorkspace) Then Set pFeatWS = pWSF.OpenFromFile(strWorkspace, 0) Set pFeatDS = pFeatWS.OpenFeatureClass(strFeatDS) End If 60 FeatureExists = (Not pFeatDS Is Nothing) Set pWSF =Nothing Set pFeatWS = Nothing Set pFeatDS = Nothing Exit Function ErrHandler: FeatureExists = False End Function 'Returns a Workspace given for example C: \temp\dataset returns C:\temp Function SplitWorkspaceName(sWholeName As String) As String On Error GoTo ERH Dim pos As Integer pos = InStrRev(sWholeName, "\") If pos > 0 Then SplitWorkspaceName = Mid(sWholeName, 1, pos - 1) Else Exit Function End If Exit Function ERH: MsgBox "Workspace Split" & Err.Description End Function 'Returns a filename given for example C:\temp\dataset returns dataset Function SplitFileName(sWholeName As String) As String On Error GoTo ERH Dim pos As Integer Dim sT, sName As String pos = InStrRev(sWholeName, "\") Ifpos > 0 Then sT = Mid(sWholeName, 1, pos - 1) Ifpos = Len(sWholeName) Then Exit Function End If sName = Mid(sWholeName, pos + 1, Len(sWholeName) - Len(sT)) pos = InStr(sName, ".") If pos > 0 Then SplitFileName = Mid(sName, 1, pos - 1) Else SplitFileName = sName End If End If Exit Function ERH: 61 • MsgBox "Workspace Split:" & Err.Description End Function Public Sub BusyMouse(bolBusy As Boolean) 'Subroutine to change mouse cursor If g---'pMouseCursor Is Nothing Then Set g---'pMouseCursor = New MouseCursor End If IfbolBusy Then g---'pMouseCursor.SetCursor 2 Else g---'pMouseCursor.SetCursor 0 End If End Sub Function MakeColor(lRGB As Long) As IRgbColor Set MakeColor =New RgbColor MakeColor.RGB = lRGB End Function Function MakeDecoElement(pMarkerSym As IMarkerSymbol, _ dPos As Double)_ As ISimpleLineDecorationElement Set MakeDecoElement

535

社区成员

发帖
与我相关
我的任务
社区描述
企业开发 Exchange Server
社区管理员
  • 消息协作社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧