server Error in '/' Application. Input string was not in a correct format.

Hubert_Biyo 2017-06-09 07:27:02
我在本地测试是正确的,放到服务器中执行出错,字符串格式哪里错了呢,
private string SaveAnswerPageInfo(Yasn.Wap.Models.PersonalInfoModel model, bool isUpdate)
{
string Msg = ""; //字符串,将答题信息放到该字段中
try
{
Hashtable firstInfo = new Hashtable();
firstInfo = (Hashtable)Session["step1"];

Yasn.Model.WXMember WXModel = new Yasn.Model.WXMember(); //实例化微信成员类
WXModel.WXNo = model.WeiXinOpenId; //微信编码
WXModel.CreateDate = DateTime.Now; //创建日期
string vxmid = Yasn.Logic.EduAccessoryWXMember.AddMemberNew(WXModel, ZhanhuiNo); //得到最大的编号
int vxmenberid = Convert.ToInt32(vxmid);
Utils.WriteLog("进入数据保存方法");
string imageUrl = CreateCode_Choose(WXModel.MemberNo.ToString()); //生成二维码
Model.ZhanHuiGuanLianModel ZHGLModel = new Model.ZhanHuiGuanLianModel(); //实例化展会关联类
string phonevalue = "";
string UserNamevalue = "";
string companyvalue = "";
string UserPositionvalue = "";
string UserEmailvalue = "";

foreach (System.Collections.DictionaryEntry de in firstInfo)
{
if (de.Key.ToString() == "Phone")
{
phonevalue = de.Value.ToString();//得到值
break;//退出foreach遍历
}
}
foreach (System.Collections.DictionaryEntry de in firstInfo)
{
if (de.Key.ToString() == "UserName")
{
UserNamevalue = de.Value.ToString();//得到值
break;//退出foreach遍历
}
}
foreach (System.Collections.DictionaryEntry de in firstInfo)
{
if (de.Key.ToString() == "company")
{
companyvalue = de.Value.ToString();//得到值
break;//退出foreach遍历
}
}
foreach (System.Collections.DictionaryEntry de in firstInfo)
{
if (de.Key.ToString() == "UserPosition")
{
UserPositionvalue = de.Value.ToString();//得到值
break;//退出foreach遍历
}
}
foreach (System.Collections.DictionaryEntry de in firstInfo)
{
if (de.Key.ToString() == "UserEmail")
{
UserEmailvalue = de.Value.ToString();//得到值
break;//退出foreach遍历
}
}
ZHGLModel.zhanhuiid = ZhanhuiNo; //展会编码
ZHGLModel.vxmenberid = vxmenberid; //微信用户id
ZHGLModel.createtime = DateTime.Now;
ZHGLModel.phone = phonevalue;
ZHGLModel.name = UserNamevalue;
ZHGLModel.cjiname = companyvalue;
ZHGLModel.job = UserPositionvalue;
ZHGLModel.email = UserEmailvalue;
ZHGLModel.cardimageurl = imageUrl;
ZHGLModel.card = WXModel.MemberNo.ToString();
ZHGLModel.gift = Convert.ToInt32(model.MyGift);
string joinPur = string.Empty; //参加目的
string educatTrends = string.Empty; //教育趋势
string enjoyProduct = string.Empty; //感兴趣的产品或领域
if (!string.IsNullOrEmpty(model.JoinPurpose1))
{
joinPur = model.JoinPurpose1.Trim().TrimEnd(',');
}
else
{
joinPur = "";
}
if (!string.IsNullOrEmpty(model.EducationalTrends1))
{
educatTrends = model.EducationalTrends1.Trim().TrimEnd(',');
}
else
{
educatTrends = "";
}
if (!string.IsNullOrEmpty(model.EnjoyProduct1))
{
enjoyProduct = model.EnjoyProduct1.Trim().TrimEnd(',');
}
else
{
enjoyProduct = "";
}

Msg = "教育展答题情况 单位性质【" + model.UnitProperty1 + "】 职务:【" + model.UnitJob1 + "】 采购环节身份:【" + model.PurchaseIdentity1 + "】 教育趋势:【" + educatTrends + "】 感兴趣的产品或领域:【" + enjoyProduct + "】 教育投资:【" + model.EducationalInvestment1 + "】";
ZHGLModel.AnswerInfo = Msg;


ZHGLModel.IsZS = 0;
ZHGLModel.YXZhanhui = "";
ZHGLModel.Telephone = model.Telephone;

string result = string.Empty;
if (isUpdate == true)
{
result = Yasn.Logic.EduAccessoryWXMember.UpdateZhanHuiGuanLian(ZHGLModel);
}
else
{
result = Yasn.Logic.EduAccessoryWXMember.AddZhanHuiGuanLian(ZHGLModel);
}
if (result != "")
{
string strpwdvalid = ConfigurationManager.AppSettings["smsSuccessCode"].Trim();
SendSMS.SendSingleSMS(model.UserMobile, string.Format(strpwdvalid, WXModel.MemberNo.ToString()), string.Empty, MessageChannel.JiaoYuSucess);
}
}
catch (Exception ex) {
Utils.WriteLog(ex.ToString());
}

return result;
}
...全文
194 2 打赏 收藏 转发到动态 举报
写回复
用AI写文章
2 条回复
切换为时间正序
请发表友善的回复…
发表回复
Hubert_Biyo 2017-06-12
  • 打赏
  • 举报
回复
引用 1 楼 duanzi_peng 的回复:
最起码自己先Debug一下 看看哪行出错了,贴这么多代码我们也不能调试。
谢谢您,我自己查看出问题了,本地数据库和服务器数据库不一样导致的,
exception92 2017-06-10
  • 打赏
  • 举报
回复
最起码自己先Debug一下 看看哪行出错了,贴这么多代码我们也不能调试。
Computer Networking: A Top-Down Approach, 6th Edition Solutions to Review Questions and Problems Version Date: May 2012 This document contains the solutions to review questions and problems for the 5th edition of Computer Networking: A Top-Down Approach by Jim Kurose and Keith Ross. These solutions are being made available to instructors ONLY. Please do NOT copy or distribute this document to others (even other instructors). Please do not post any solutions on a publicly-available Web site. We’ll be happy to provide a copy (up-to-date) of this solution manual ourselves to anyone who asks. Acknowledgments: Over the years, several students and colleagues have helped us prepare this solutions manual. Special thanks goes to HongGang Zhang, Rakesh Kumar, Prithula Dhungel, and Vijay Annapureddy. Also thanks to all the readers who have made suggestions and corrected errors. All material © copyright 1996-2012 by J.F. Kurose and K.W. Ross. All rights reserved Chapter 1 Review Questions There is no difference. Throughout this text, the words “host” and “end system” are used interchangeably. End systems include PCs, workstations, Web servers, mail servers, PDAs, Internet-connected game consoles, etc. From Wikipedia: Diplomatic protocol is commonly described as a set of international courtesy rules. These well-established and time-honored rules have made it easier for nations and people to live and work together. Part of protocol has always been the acknowledgment of the hierarchical standing of all present. Protocol rules are based on the principles of civility. Standards are important for protocols so that people can create networking systems and products that interoperate. 1. Dial-up modem over telephone line: home; 2. DSL over telephone line: home or small office; 3. Cable to HFC: home; 4. 100 Mbps switched Ethernet: enterprise; 5. Wifi (802.11): home and enterprise: 6. 3G and 4G: wide-area wireless. HFC bandwidth is shared among the users. On the downstream channel, all packets emanate from a single source, namely, the head end. Thus, there are no collisions in the downstream channel. In most American cities, the current possibilities include: dial-up; DSL; cable modem; fiber-to-the-home. 7. Ethernet LANs have transmission rates of 10 Mbps, 100 Mbps, 1 Gbps and 10 Gbps. 8. Today, Ethernet most commonly runs over twisted-pair copper wire. It also can run over fibers optic links. 9. Dial up modems: up to 56 Kbps, bandwidth is dedicated; ADSL: up to 24 Mbps downstream and 2.5 Mbps upstream, bandwidth is dedicated; HFC, rates up to 42.8 Mbps and upstream rates of up to 30.7 Mbps, bandwidth is shared. FTTH: 2-10Mbps upload; 10-20 Mbps download; bandwidth is not shared. 10. There are two popular wireless Internet access technologies today: Wifi (802.11) In a wireless LAN, wireless users transmit/receive packets to/from an base station (i.e., wireless access point) within a radius of few tens of meters. The base station is typically connected to the wired Internet and thus serves to connect wireless users to the wired network. 3G and 4G wide-area wireless access networks. In these systems, packets are transmitted over the same wireless infrastructure used for cellular telephony, with the base station thus being managed by a telecommunications provider. This provides wireless access to users within a radius of tens of kilometers of the base station. 11. At time t0 the sending host begins to transmit. At time t1 = L/R1, the sending host completes transmission and the entire packet is received at the router (no propagation delay). Because the router has the entire packet at time t1, it can begin to transmit the packet to the receiving host at time t1. At time t2 = t1 + L/R2, the router completes transmission and the entire packet is received at the receiving host (again, no propagation delay). Thus, the end-to-end delay is L/R1 + L/R2. 12. A circuit-switched network can guarantee a certain amount of end-to-end bandwidth for the duration of a call. Most packet-switched networks today (including the Internet) cannot make any end-to-end guarantees for bandwidth. FDM requires sophisticated analog hardware to shift signal into appropriate frequency bands. 13. a) 2 users can be supported because each user requires half of the link bandwidth. b) Since each user requires 1Mbps when transmitting, if two or fewer users transmit simultaneously, a maximum of 2Mbps will be required. Since the available bandwidth of the shared link is 2Mbps, there will be no queuing delay before the link. Whereas, if three users transmit simultaneously, the bandwidth required will be 3Mbps which is more than the available bandwidth of the shared link. In this case, there will be queuing delay before the link. c) Probability that a given user is transmitting = 0.2 d) Probability that all three users are transmitting simultaneously = = (0.2)3 = 0.008. Since the queue grows when all the users are transmitting, the fraction of time during which the queue grows (which is equal to the probability that all three users are transmitting simultaneously) is 0.008. 14. If the two ISPs do not peer with each other, then when they send traffic to each other they have to send the traffic through a provider ISP (intermediary), to which they have to pay for carrying the traffic. By peering with each other directly, the two ISPs can reduce their payments to their provider ISPs. An Internet Exchange Points (IXP) (typically in a standalone building with its own switches) is a meeting point where multiple ISPs can connect and/or peer together. An ISP earns its money by charging each of the the ISPs that connect to the IXP a relatively small fee, which may depend on the amount of traffic sent to or received from the IXP. 15. Google's private network connects together all its data centers, big and small. Traffic between the Google data centers passes over its private network rather than over the public Internet. Many of these data centers are located in, or close to, lower tier ISPs. Therefore, when Google delivers content to a user, it often can bypass higher tier ISPs. What motivates content providers to create these networks? First, the content provider has more control over the user experience, since it has to use few intermediary ISPs. Second, it can save money by sending less traffic into provider networks. Third, if ISPs decide to charge more money to highly profitable content providers (in countries where net neutrality doesn't apply), the content providers can avoid these extra payments. 16. The delay components are processing delays, transmission delays, propagation delays, and queuing delays. All of these delays are fixed, except for the queuing delays, which are variable. 17. a) 1000 km, 1 Mbps, 100 bytes b) 100 km, 1 Mbps, 100 bytes 18. 10msec; d/s; no; no 19. a) 500 kbps b) 64 seconds c) 100kbps; 320 seconds 20. End system A breaks the large file into chunks. It adds header to each chunk, thereby generating multiple packets from the file. The header in each packet includes the IP address of the destination (end system B). The packet switch uses the destination IP address in the packet to determine the outgoing link. Asking which road to take is analogous to a packet asking which outgoing link it should be forwarded on, given the packet’s destination address. 21. The maximum emission rate is 500 packets/sec and the maximum transmission rate is 350 packets/sec. The corresponding traffic intensity is 500/350 =1.43 > 1. Loss will eventually occur for each experiment; but the time when loss first occurs will be different from one experiment to the next due to the randomness in the emission process. 22. Five generic tasks are error control, flow control, segmentation and reassembly, multiplexing, and connection setup. Yes, these tasks can be duplicated at different layers. For example, error control is often provided at more than one layer. 23. The five layers in the Internet protocol stack are – from top to bottom – the application layer, the transport layer, the network layer, the link layer, and the physical layer. The principal responsibilities are outlined in Section 1.5.1. 24. Application-layer message: data which an application wants to send and passed onto the transport layer; transport-layer segment: generated by the transport layer and encapsulates application-layer message with transport layer header; network-layer datagram: encapsulates transport-layer segment with a network-layer header; link-layer frame: encapsulates network-layer datagram with a link-layer header. 25. Routers process network, link and physical layers (layers 1 through 3). (This is a little bit of a white lie, as modern routers sometimes act as firewalls or caching components, and process Transport layer as well.) Link layer switches process link and physical layers (layers 1 through2). Hosts process all five layers. 26. a) Virus Requires some form of human interaction to spread. Classic example: E-mail viruses. b) Worms No user replication needed. Worm in infected host scans IP addresses and port numbers, looking for vulnerable processes to infect. 27. Creation of a botnet requires an attacker to find vulnerability in some application or system (e.g. exploiting the buffer overflow vulnerability that might exist in an application). After finding the vulnerability, the attacker needs to scan for hosts that are vulnerable. The target is basically to compromise a series of systems by exploiting that particular vulnerability. Any system that is part of the botnet can automatically scan its environment and propagate by exploiting the vulnerability. An important property of such botnets is that the originator of the botnet can remotely control and issue commands to all the nodes in the botnet. Hence, it becomes possible for the attacker to issue a command to all the nodes, that target a single node (for example, all nodes in the botnet might be commanded by the attacker to send a TCP SYN message to the target, which might result in a TCP SYN flood attack at the target). 28. Trudy can pretend to be Bob to Alice (and vice-versa) and partially or completely modify the message(s) being sent from Bob to Alice. For example, she can easily change the phrase “Alice, I owe you $1000” to “Alice, I owe you $10,000”. Furthermore, Trudy can even drop the packets that are being sent by Bob to Alice (and vise-versa), even if the packets from Bob to Alice are encrypted. Chapter 1 Problems Problem 1 There is no single right answer to this question. Many protocols would do the trick. Here's a simple answer below: Messages from ATM machine to Server Msg name purpose -------- ------- HELO Let server know that there is a card in the ATM machine ATM card transmits user ID to Server PASSWD User enters PIN, which is sent to server BALANCE User requests balance WITHDRAWL User asks to withdraw money BYE user all done Messages from Server to ATM machine (display) Msg name purpose -------- ------- PASSWD Ask user for PIN (password) OK last requested operation (PASSWD, WITHDRAWL) OK ERR last requested operation (PASSWD, WITHDRAWL) in ERROR AMOUNT sent in response to BALANCE request BYE user done, display welcome screen at ATM Correct operation: client server HELO (userid) --------------> (check if valid userid) <------------- PASSWD PASSWD --------------> (check password) <------------- AMOUNT WITHDRAWL --------------> check if enough $ to cover withdrawl (check if valid userid) <------------- PASSWD PASSWD --------------> (check password) <------------- AMOUNT WITHDRAWL --------------> check if enough $ to cover withdrawl <------------- BYE Problem 2 At time N*(L/R) the first packet has reached the destination, the second packet is stored in the last router, the third packet is stored in the next-to-last router, etc. At time N*(L/R) + L/R, the second packet has reached the destination, the third packet is stored in the last router, etc. Continuing with this logic, we see that at time N*(L/R) + (P-1)*(L/R) = (N+P-1)*(L/R) all packets have reached the destination. Problem 3 a) A circuit-switched network would be well suited to the application, because the application involves long sessions with predictable smooth bandwidth requirements. Since the transmission rate is known and not bursty, bandwidth can be reserved for each application session without significant waste. In addition, the overhead costs of setting up and tearing down connections are amortized over the lengthy duration of a typical application session. b) In the worst case, all the applications simultaneously transmit over one or more network links. However, since each link has sufficient bandwidth to handle the sum of all of the applications' data rates, no congestion (very little queuing) will occur. Given such generous link capacities, the network does not need congestion control mechanisms. Problem 4 Between the switch in the upper left and the switch in the upper right we can have 4 connections. Similarly we can have four connections between each of the 3 other pairs of adjacent switches. Thus, this network can support up to 16 connections. We can 4 connections passing through the switch in the upper-right-hand corner and another 4 connections passing through the switch in the lower-left-hand corner, giving a total of 8 connections. Yes. For the connections between A and C, we route two connections through B and two connections through D. For the connections between B and D, we route two connections through A and two connections through C. In this manner, there are at most 4 connections passing through any link. Problem 5 Tollbooths are 75 km apart, and the cars propagate at 100km/hr. A tollbooth services a car at a rate of one car every 12 seconds. a) There are ten cars. It takes 120 seconds, or 2 minutes, for the first tollbooth to service the 10 cars. Each of these cars has a propagation delay of 45 minutes (travel 75 km) before arriving at the second tollbooth. Thus, all the cars are lined up before the second tollbooth after 47 minutes. The whole process repeats itself for traveling between the second and third tollbooths. It also takes 2 minutes for the third tollbooth to service the 10 cars. Thus the total delay is 96 minutes. b) Delay between tollbooths is 8*12 seconds plus 45 minutes, i.e., 46 minutes and 36 seconds. The total delay is twice this amount plus 8*12 seconds, i.e., 94 minutes and 48 seconds. Problem 6 a) seconds. b) seconds. c) seconds. d) The bit is just leaving Host A. e) The first bit is in the link and has not reached Host B. f) The first bit has reached Host B. g) Want km. Problem 7 Consider the first bit in a packet. Before this bit can be transmitted, all of the bits in the packet must be generated. This requires sec=7msec. The time required to transmit the packet is sec= sec. Propagation delay = 10 msec. The delay until decoding is 7msec + sec + 10msec = 17.224msec A similar analysis shows that all bits experience a delay of 17.224 msec. Problem 8 a) 20 users can be supported. b) . c) . d) . We use the central limit theorem to approximate this probability. Let be independent random variables such that . “21 or more users” when is a standard normal r.v. Thus “21 or more users” . Problem 9 10,000 Problem 10 The first end system requires L/R1 to transmit the packet onto the first link; the packet propagates over the first link in d1/s1; the packet switch adds a processing delay of dproc; after receiving the entire packet, the packet switch connecting the first and the second link requires L/R2 to transmit the packet onto the second link; the packet propagates over the second link in d2/s2. Similarly, we can find the delay caused by the second switch and the third link: L/R3, dproc, and d3/s3. Adding these five delays gives dend-end = L/R1 + L/R2 + L/R3 + d1/s1 + d2/s2 + d3/s3+ dproc+ dproc To answer the second question, we simply plug the values into the equation to get 6 + 6 + 6 + 20+16 + 4 + 3 + 3 = 64 msec. Problem 11 Because bits are immediately transmitted, the packet switch does not introduce any delay; in particular, it does not introduce a transmission delay. Thus, dend-end = L/R + d1/s1 + d2/s2+ d3/s3 For the values in Problem 10, we get 6 + 20 + 16 + 4 = 46 msec. Problem 12 The arriving packet must first wait for the link to transmit 4.5 *1,500 bytes = 6,750 bytes or 54,000 bits. Since these bits are transmitted at 2 Mbps, the queuing delay is 27 msec. Generally, the queuing delay is (nL + (L - x))/R. Problem 13 The queuing delay is 0 for the first transmitted packet, L/R for the second transmitted packet, and generally, (n-1)L/R for the nth transmitted packet. Thus, the average delay for the N packets is: (L/R + 2L/R + ....... + (N-1)L/R)/N = L/(RN) * (1 + 2 + ..... + (N-1)) = L/(RN) * N(N-1)/2 = LN(N-1)/(2RN) = (N-1)L/(2R) Note that here we used the well-known fact: 1 + 2 + ....... + N = N(N+1)/2 It takes seconds to transmit the packets. Thus, the buffer is empty when a each batch of packets arrive. Thus, the average delay of a packet across all batches is the average delay within one batch, i.e., (N-1)L/2R. Problem 14 The transmission delay is . The total delay is Let . Total delay = For x=0, the total delay =0; as we increase x, total delay increases, approaching infinity as x approaches 1/a. Problem 15 Total delay . Problem 16 The total number of packets in the system includes those in the buffer and the packet that is being transmitted. So, N=10+1. Because , so (10+1)=a*(queuing delay + transmission delay). That is, 11=a*(0.01+1/100)=a*(0.01+0.01). Thus, a=550 packets/sec. Problem 17 There are nodes (the source host and the routers). Let denote the processing delay at the th node. Let be the transmission rate of the th link and let . Let be the propagation delay across the th link. Then . Let denote the average queuing delay at node . Then . Problem 18 On linux you can use the command traceroute www.targethost.com and in the Windows command prompt you can use tracert www.targethost.com In either case, you will get three delay measurements. For those three measurements you can calculate the mean and standard deviation. Repeat the experiment at different times of the day and comment on any changes. Here is an example solution: Traceroutes between San Diego Super Computer Center and www.poly.edu The average (mean) of the round-trip delays at each of the three hours is 71.18 ms, 71.38 ms and 71.55 ms, respectively. The standard deviations are 0.075 ms, 0.21 ms, 0.05 ms, respectively. In this example, the traceroutes have 12 routers in the path at each of the three hours. No, the paths didn’t change during any of the hours. Traceroute packets passed through four ISP networks from source to destination. Yes, in this experiment the largest delays occurred at peering interfaces between adjacent ISPs. Traceroutes from www.stella-net.net (France) to www.poly.edu (USA). The average round-trip delays at each of the three hours are 87.09 ms, 86.35 ms and 86.48 ms, respectively. The standard deviations are 0.53 ms, 0.18 ms, 0.23 ms, respectively. In this example, there are 11 routers in the path at each of the three hours. No, the paths didn’t change during any of the hours. Traceroute packets passed three ISP networks from source to destination. Yes, in this experiment the largest delays occurred at peering interfaces between adjacent ISPs. Problem 19 An example solution: Traceroutes from two different cities in France to New York City in United States In these traceroutes from two different cities in France to the same destination host in United States, seven links are in common including the transatlantic link. In this example of traceroutes from one city in France and from another city in Germany to the same host in United States, three links are in common including the transatlantic link. Traceroutes to two different cities in China from same host in United States Five links are common in the two traceroutes. The two traceroutes diverge before reaching China Problem 20 Throughput = min{Rs, Rc, R/M} Problem 21 If only use one path, the max throughput is given by: . If use all paths, the max throughput is given by . Problem 22 Probability of successfully receiving a packet is: ps= (1-p)N. The number of transmissions needed to be performed until the packet is successfully received by the client is a geometric random variable with success probability ps. Thus, the average number of transmissions needed is given by: 1/ps . Then, the average number of re-transmissions needed is given by: 1/ps -1. Problem 23 Let’s call the first packet A and call the second packet B. If the bottleneck link is the first link, then packet B is queued at the first link waiting for the transmission of packet A. So the packet inter-arrival time at the destination is simply L/Rs. If the second link is the bottleneck link and both packets are sent back to back, it must be true that the second packet arrives at the input queue of the second link before the second link finishes the transmission of the first packet. That is, L/Rs + L/Rs + dprop = L/Rs + dprop + L/Rc Thus, the minimum value of T is L/Rc  L/Rs . Problem 24 40 terabytes = 40 * 1012 * 8 bits. So, if using the dedicated link, it will take 40 * 1012 * 8 / (100 *106 ) =3200000 seconds = 37 days. But with FedEx overnight delivery, you can guarantee the data arrives in one day, and it should cost less than $100. Problem 25 160,000 bits 160,000 bits The bandwidth-delay product of a link is the maximum number of bits that can be in the link. the width of a bit = length of link / bandwidth-delay product, so 1 bit is 125 meters long, which is longer than a football field s/R Problem 26 s/R=20000km, then R=s/20000km= 2.5*108/(2*107)= 12.5 bps Problem 27 80,000,000 bits 800,000 bits, this is because that the maximum number of bits that will be in the link at any given time = min(bandwidth delay product, packet size) = 800,000 bits. .25 meters Problem 28 ttrans + tprop = 400 msec + 80 msec = 480 msec. 20 * (ttrans + 2 tprop) = 20*(20 msec + 80 msec) = 2 sec. Breaking up a file takes longer to transmit because each data packet and its corresponding acknowledgement packet add their own propagation delays. Problem 29 Recall geostationary satellite is 36,000 kilometers away from earth surface. 150 msec 1,500,000 bits 600,000,000 bits Problem 30 Let’s suppose the passenger and his/her bags correspond to the data unit arriving to the top of the protocol stack. When the passenger checks in, his/her bags are checked, and a tag is attached to the bags and ticket. This is additional information added in the Baggage layer if Figure 1.20 that allows the Baggage layer to implement the service or separating the passengers and baggage on the sending side, and then reuniting them (hopefully!) on the destination side. When a passenger then passes through security and additional stamp is often added to his/her ticket, indicating that the passenger has passed through a security check. This information is used to ensure (e.g., by later checks for the security information) secure transfer of people. Problem 31 Time to send message from source host to first packet switch = With store-and-forward switching, the total time to move message from source host to destination host = Time to send 1st packet from source host to first packet switch = . . Time at which 2nd packet is received at the first switch = time at which 1st packet is received at the second switch = Time at which 1st packet is received at the destination host = . After this, every 5msec one packet will be received; thus time at which last (800th) packet is received = . It can be seen that delay in using message segmentation is significantly less (almost 1/3rd). Without message segmentation, if bit errors are not tolerated, if there is a single bit error, the whole message has to be retransmitted (rather than a single packet). Without message segmentation, huge packets (containing HD videos, for example) are sent into the network. Routers have to accommodate these huge packets. Smaller packets have to queue behind enormous packets and suffer unfair delays. Packets have to be put in sequence at the destination. Message segmentation results in many smaller packets. Since header size is usually the same for all packets regardless of their size, with message segmentation the total amount of header bytes is more. Problem 32 Yes, the delays in the applet correspond to the delays in the Problem 31.The propagation delays affect the overall end-to-end delays both for packet switching and message switching equally. Problem 33 There are F/S packets. Each packet is S=80 bits. Time at which the last packet is received at the first router is sec. At this time, the first F/S-2 packets are at the destination, and the F/S-1 packet is at the second router. The last packet must then be transmitted by the first router and the second router, with each transmission taking sec. Thus delay in sending the whole file is To calculate the value of S which leads to the minimum delay, Problem 34 The circuit-switched telephone networks and the Internet are connected together at "gateways". When a Skype user (connected to the Internet) calls an ordinary telephone, a circuit is established between a gateway and the telephone user over the circuit switched network. The skype user's voice is sent in packets over the Internet to the gateway. At the gateway, the voice signal is reconstructed and then sent over the circuit. In the other direction, the voice signal is sent over the circuit switched network to the gateway. The gateway packetizes the voice signal and sends the voice packets to the Skype user.   Chapter 2 Review Questions The Web: HTTP; file transfer: FTP; remote login: Telnet; e-mail: SMTP; BitTorrent file sharing: BitTorrent protocol Network architecture refers to the organization of the communication process into layers (e.g., the five-layer Internet architecture). Application architecture, on the other hand, is designed by an application developer and dictates the broad structure of the application (e.g., client-server or P2P). The process which initiates the communication is the client; the process that waits to be contacted is the server. No. In a P2P file-sharing application, the peer that is receiving a file is typically the client and the peer that is sending the file is typically the server. The IP address of the destination host and the port number of the socket in the destination process. You would use UDP. With UDP, the transaction can be completed in one roundtrip time (RTT) - the client sends the transaction request into a UDP socket, and the server sends the reply back to the client's UDP socket. With TCP, a minimum of two RTTs are needed - one to set-up the TCP connection, and another for the client to send the request, and for the server to send back the reply. One such example is remote word processing, for example, with Google docs. However, because Google docs runs over the Internet (using TCP), timing guarantees are not provided. a) Reliable data transfer TCP provides a reliable byte-stream between client and server but UDP does not. b) A guarantee that a certain value for throughput will be maintained Neither c) A guarantee that data will be delivered within a specified amount of time Neither d) Confidentiality (via encryption) Neither SSL operates at the application layer. The SSL socket takes unencrypted data from the application layer, encrypts it and then passes it to the TCP socket. If the application developer wants TCP to be enhanced with SSL, she has to include the SSL code in the application. A protocol uses handshaking if the two communicating entities first exchange control packets before sending data to each other. SMTP uses handshaking at the application layer whereas HTTP does not. The applications associated with those protocols require that all application data be received in the correct order and without gaps. TCP provides this service whereas UDP does not. When the user first visits the site, the server creates a unique identification number, creates an entry in its back-end database, and returns this identification number as a cookie number. This cookie number is stored on the user’s host and is managed by the browser. During each subsequent visit (and purchase), the browser sends the cookie number back to the site. Thus the site knows when this user (more precisely, this browser) is visiting the site. Web caching can bring the desired content “closer” to the user, possibly to the same LAN to which the user’s host is connected. Web caching can reduce the delay for all objects, even objects that are not cached, since caching reduces the traffic on links. Telnet is not available in Windows 7 by default. to make it available, go to Control Panel, Programs and Features, Turn Windows Features On or Off, Check Telnet client. To start Telnet, in Windows command prompt, issue the following command > telnet webserverver 80 where "webserver" is some webserver. After issuing the command, you have established a TCP connection between your client telnet program and the web server. Then type in an HTTP GET message. An example is given below: Since the index.html page in this web server was not modified since Fri, 18 May 2007 09:23:34 GMT, and the above commands were issued on Sat, 19 May 2007, the server returned "304 Not Modified". Note that the first 4 lines are the GET message and header lines inputed by the user, and the next 4 lines (starting from HTTP/1.1 304 Not Modified) is the response from the web server. FTP uses two parallel TCP connections, one connection for sending control information (such as a request to transfer a file) and another connection for actually transferring the file. Because the control information is not sent over the same connection that the file is sent over, FTP sends control information out of band. The message is first sent from Alice’s host to her mail server over HTTP. Alice’s mail server then sends the message to Bob’s mail server over SMTP. Bob then transfers the message from his mail server to his host over POP3. 17. Received: from 65.54.246.203 (EHLO bay0-omc3-s3.bay0.hotmail.com) (65.54.246.203) by mta419.mail.mud.yahoo.com with SMTP; Sat, 19 May 2007 16:53:51 -0700 Received: from hotmail.com ([65.55.135.106]) by bay0-omc3-s3.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.2668); Sat, 19 May 2007 16:52:42 -0700 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 19 May 2007 16:52:41 -0700 Message-ID: Received: from 65.55.135.123 by by130fd.bay130.hotmail.msn.com with HTTP; Sat, 19 May 2007 23:52:36 GMT From: "prithula dhungel" To: prithula@yahoo.com Bcc: Subject: Test mail Date: Sat, 19 May 2007 23:52:36 +0000 Mime-Version: 1.0 Content-Type: Text/html; format=flowed Return-Path: prithuladhungel@hotmail.com Figure: A sample mail message header Received: This header field indicates the sequence in which the SMTP servers send and receive the mail message including the respective timestamps. In this example there are 4 “Received:” header lines. This means the mail message passed through 5 different SMTP servers before being delivered to the receiver’s mail box. The last (forth) “Received:” header indicates the mail message flow from the SMTP server of the sender to the second SMTP server in the chain of servers. The sender’s SMTP server is at address 65.55.135.123 and the second SMTP server in the chain is by130fd.bay130.hotmail.msn.com. The third “Received:” header indicates the mail message flow from the second SMTP server in the chain to the third server, and so on. Finally, the first “Received:” header indicates the flow of the mail messages from the forth SMTP server to the last SMTP server (i.e. the receiver’s mail server) in the chain. Message-id: The message has been given this number BAY130-F26D9E35BF59E0D18A819AFB9310@phx.gbl (by bay0-omc3-s3.bay0.hotmail.com. Message-id is a unique string assigned by the mail system when the message is first created. From: This indicates the email address of the sender of the mail. In the given example, the sender is “prithuladhungel@hotmail.com” To: This field indicates the email address of the receiver of the mail. In the example, the receiver is “prithula@yahoo.com” Subject: This gives the subject of the mail (if any specified by the sender). In the example, the subject specified by the sender is “Test mail” Date: The date and time when the mail was sent by the sender. In the example, the sender sent the mail on 19th May 2007, at time 23:52:36 GMT. Mime-version: MIME version used for the mail. In the example, it is 1.0. Content-type: The type of content in the body of the mail message. In the example, it is “text/html”. Return-Path: This specifies the email address to which the mail will be sent if the receiver of this mail wants to reply to the sender. This is also used by the sender’s mail server for bouncing back undeliverable mail messages of mailer-daemon error messages. In the example, the return path is “prithuladhungel@hotmail.com”. With download and delete, after a user retrieves its messages from a POP server, the messages are deleted. This poses a problem for the nomadic user, who may want to access the messages from many different machines (office PC, home PC, etc.). In the download and keep configuration, messages are not deleted after the user retrieves the messages. This can also be inconvenient, as each time the user retrieves the stored messages from a new machine, all of non-deleted messages will be transferred to the new machine (including very old messages). Yes an organization’s mail server and Web server can have the same alias for a host name. The MX record is used to map the mail server’s host name to its IP address. You should be able to see the sender's IP address for a user with an .edu email address. But you will not be able to see the sender's IP address if the user uses a gmail account. It is not necessary that Bob will also provide chunks to Alice. Alice has to be in the top 4 neighbors of Bob for Bob to send out chunks to her; this might not occur even if Alice provides chunks to Bob throughout a 30-second interval. Recall that in BitTorrent, a peer picks a random peer and optimistically unchokes the peer for a short period of time. Therefore, Alice will eventually be optimistically unchoked by one of her neighbors, during which time she will receive chunks from that neighbor. The overlay network in a P2P file sharing system consists of the nodes participating in the file sharing system and the logical links between the nodes. There is a logical link (an “edge” in graph theory terms) from node A to node B if there is a semi-permanent TCP connection between A and B. An overlay network does not include routers. Mesh DHT: The advantage is in order to a route a message to the peer (with ID) that is closest to the key, only one hop is required; the disadvantage is that each peer must track all other peers in the DHT. Circular DHT: the advantage is that each peer needs to track only a few other peers; the disadvantage is that O(N) hops are needed to route a message to the peer that is closest to the key. 25. File Distribution Instant Messaging Video Streaming Distributed Computing With the UDP server, there is no welcoming socket, and all data from different clients enters the server through this one socket. With the TCP server, there is a welcoming socket, and each time a client initiates a connection to the server, a new socket is created. Thus, to support n simultaneous connections, the server would need n+1 sockets. For the TCP application, as soon as the client is executed, it attempts to initiate a TCP connection with the server. If the TCP server is not running, then the client will fail to make a connection. For the UDP application, the client does not initiate connections (or attempt to communicate with the UDP server) immediately upon execution Chapter 2 Problems Problem 1 a) F b) T c) F d) F e) F Problem 2 Access control commands: USER, PASS, ACT, CWD, CDUP, SMNT, REIN, QUIT. Transfer parameter commands: PORT, PASV, TYPE STRU, MODE. Service commands: RETR, STOR, STOU, APPE, ALLO, REST, RNFR, RNTO, ABOR, DELE, RMD, MRD, PWD, LIST, NLST, SITE, SYST, STAT, HELP, NOOP. Problem 3 Application layer protocols: DNS and HTTP Transport layer protocols: UDP for DNS; TCP for HTTP Problem 4 The document request was http://gaia.cs.umass.edu/cs453/index.html. The Host : field indicates the server's name and /cs453/index.html indicates the file name. The browser is running HTTP version 1.1, as indicated just before the first pair. The browser is requesting a persistent connection, as indicated by the Connection: keep-alive. This is a trick question. This information is not contained in an HTTP message anywhere. So there is no way to tell this from looking at the exchange of HTTP messages alone. One would need information from the IP datagrams (that carried the TCP segment that carried the HTTP GET request) to answer this question. Mozilla/5.0. The browser type information is needed by the server to send different versions of the same object to different types of browsers. Problem 5 The status code of 200 and the phrase OK indicate that the server was able to locate the document successfully. The reply was provided on Tuesday, 07 Mar 2008 12:39:45 Greenwich Mean Time. The document index.html was last modified on Saturday 10 Dec 2005 18:27:46 GMT. There are 3874 bytes in the document being returned. The first five bytes of the returned document are : server agreed to a persistent connection, as indicated by the Connection: Keep-Alive field Problem 6 Persistent connections are discussed in section 8 of RFC 2616 (the real goal of this question was to get you to retrieve and read an RFC). Sections 8.1.2 and 8.1.2.1 of the RFC indicate that either the client or the server can indicate to the other that it is going to close the persistent connection. It does so by including the connection-token "close" in the Connection-header field of the http request/reply. HTTP does not provide any encryption services. (From RFC 2616) “Clients that use persistent connections should limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy.” Yes. (From RFC 2616) “A client might have started to send a new request at the same time that the server has decided to close the "idle" connection. From the server's point of view, the connection is being closed while it was idle, but from the client's point of view, a request is in progress.” Problem 7 The total amount of time to get the IP address is . Once the IP address is known, elapses to set up the TCP connection and another elapses to request and receive the small object. The total response time is Problem 8 . . Problem 9 The time to transmit an object of size L over a link or rate R is L/R. The average time is the average size of the object divided by R:  = (850,000 bits)/(15,000,000 bits/sec) = .0567 sec The traffic intensity on the link is given by =(16 requests/sec)(.0567 sec/request) = 0.907. Thus, the average access delay is (.0567 sec)/(1 - .907)  .6 seconds. The total average response time is therefore .6 sec + 3 sec = 3.6 sec. The traffic intensity on the access link is reduced by 60% since the 60% of the requests are satisfied within the institutional network. Thus the average access delay is (.0567 sec)/[1 – (.4)(.907)] = .089 seconds. The response time is approximately zero if the request is satisfied by the cache (which happens with probability .6); the average response time is .089 sec + 3 sec = 3.089 sec for cache misses (which happens 40% of the time). So the average response time is (.6)(0 sec) + (.4)(3.089 sec) = 1.24 seconds. Thus the average response time is reduced from 3.6 sec to 1.24 sec. Problem 10 Note that each downloaded object can be completely put into one data packet. Let Tp denote the one-way propagation delay between the client and the server. First consider parallel downloads using non-persistent connections. Parallel downloads would allow 10 connections to share the 150 bits/sec bandwidth, giving each just 15 bits/sec. Thus, the total time needed to receive all objects is given by: (200/150+Tp + 200/150 +Tp + 200/150+Tp + 100,000/150+ Tp ) + (200/(150/10)+Tp + 200/(150/10) +Tp + 200/(150/10)+Tp + 100,000/(150/10)+ Tp ) = 7377 + 8*Tp (seconds) Now consider a persistent HTTP connection. The total time needed is given by: (200/150+Tp + 200/150 +Tp + 200/150+Tp + 100,000/150+ Tp ) + 10*(200/150+Tp + 100,000/150+ Tp ) =7351 + 24*Tp (seconds) Assuming the speed of light is 300*106 m/sec, then Tp=10/(300*106)=0.03 microsec. Tp is therefore negligible compared with transmission delay. Thus, we see that persistent HTTP is not significantly faster (less than 1 percent) than the non-persistent case with parallel download. Problem 11 Yes, because Bob has more connections, he can get a larger share of the link bandwidth. Yes, Bob still needs to perform parallel downloads; otherwise he will get less bandwidth than the other four users. Problem 12 Server.py from socket import * serverPort=12000 serverSocket=socket(AF_INET,SOCK_STREAM) serverSocket.bind(('',serverPort)) serverSocket.listen(1) connectionSocket, addr = serverSocket.accept() while 1: sentence = connectionSocket.recv(1024) print 'From Server:', sentence, '\n' serverSocket.close() Problem 13 The MAIL FROM: in SMTP is a message from the SMTP client that identifies the sender of the mail message to the SMTP server. The From: on the mail message itself is NOT an SMTP message, but rather is just a line in the body of the mail message. Problem 14 SMTP uses a line containing only a period to mark the end of a message body. HTTP uses “Content-Length header field” to indicate the length of a message body. No, HTTP cannot use the method used by SMTP, because HTTP message could be binary data, whereas in SMTP, the message body must be in 7-bit ASCII format. Problem 15 MTA stands for Mail Transfer Agent. A host sends the message to an MTA. The message then follows a sequence of MTAs to reach the receiver’s mail reader. We see that this spam message follows a chain of MTAs. An honest MTA should report where it receives the message. Notice that in this message, “asusus-4b96 ([58.88.21.177])” does not report from where it received the email. Since we assume only the originator is dishonest, so “asusus-4b96 ([58.88.21.177])” must be the originator. Problem 16 UIDL abbreviates “unique-ID listing”. When a POP3 client issues the UIDL command, the server responds with the unique message ID for all of the messages present in the user's mailbox. This command is useful for “download and keep”. By maintaining a file that lists the messages retrieved during earlier sessions, the client can use the UIDL command to determine which messages on the server have already been seen. Problem 17 a) C: dele 1 C: retr 2 S: (blah blah … S: ………..blah) S: . C: dele 2 C: quit S: +OK POP3 server signing off b) C: retr 2 S: blah blah … S: ………..blah S: . C: quit S: +OK POP3 server signing off C: list S: 1 498 S: 2 912 S: . C: retr 1 S: blah ….. S: ….blah S: . C: retr 2 S: blah blah … S: ………..blah S: . C: quit S: +OK POP3 server signing off Problem 18 For a given input of domain name (such as ccn.com), IP address or network administrator name, the whois database can be used to locate the corresponding registrar, whois server, DNS server, and so on. NS4.YAHOO.COM from www.register.com; NS1.MSFT.NET from ww.register.com Local Domain: www.mindspring.com Web servers : www.mindspring.com 207.69.189.21, 207.69.189.22, 207.69.189.23, 207.69.189.24, 207.69.189.25, 207.69.189.26, 207.69.189.27, 207.69.189.28 Mail Servers : mx1.mindspring.com (207.69.189.217) mx2.mindspring.com (207.69.189.218) mx3.mindspring.com (207.69.189.219) mx4.mindspring.com (207.69.189.220) Name Servers: itchy.earthlink.net (207.69.188.196) scratchy.earthlink.net (207.69.188.197) www.yahoo.com Web Servers: www.yahoo.com (216.109.112.135, 66.94.234.13) Mail Servers: a.mx.mail.yahoo.com (209.191.118.103) b.mx.mail.yahoo.com (66.196.97.250) c.mx.mail.yahoo.com (68.142.237.182, 216.39.53.3) d.mx.mail.yahoo.com (216.39.53.2) e.mx.mail.yahoo.com (216.39.53.1) f.mx.mail.yahoo.com (209.191.88.247, 68.142.202.247) g.mx.mail.yahoo.com (209.191.88.239, 206.190.53.191) Name Servers: ns1.yahoo.com (66.218.71.63) ns2.yahoo.com (68.142.255.16) ns3.yahoo.com (217.12.4.104) ns4.yahoo.com (68.142.196.63) ns5.yahoo.com (216.109.116.17) ns8.yahoo.com (202.165.104.22) ns9.yahoo.com (202.160.176.146) www.hotmail.com Web Servers: www.hotmail.com (64.4.33.7, 64.4.32.7) Mail Servers: mx1.hotmail.com (65.54.245.8, 65.54.244.8, 65.54.244.136) mx2.hotmail.com (65.54.244.40, 65.54.244.168, 65.54.245.40) mx3.hotmail.com (65.54.244.72, 65.54.244.200, 65.54.245.72) mx4.hotmail.com (65.54.244.232, 65.54.245.104, 65.54.244.104) Name Servers: ns1.msft.net (207.68.160.190) ns2.msft.net (65.54.240.126) ns3.msft.net (213.199.161.77) ns4.msft.net (207.46.66.126) ns5.msft.net (65.55.238.126) d) The yahoo web server has multiple IP addresses www.yahoo.com (216.109.112.135, 66.94.234.13) e) The address range for Polytechnic University: 128.238.0.0 – 128.238.255.255 f) An attacker can use the whois database and nslookup tool to determine the IP address ranges, DNS server addresses, etc., for the target institution. By analyzing the source address of attack packets, the victim can use whois to obtain information about domain from which the attack is coming and possibly inform the administrators of the origin domain. Problem 19 The following delegation chain is used for gaia.cs.umass.edu a.root-servers.net E.GTLD-SERVERS.NET ns1.umass.edu(authoritative) First command: dig +norecurse @a.root-servers.net any gaia.cs.umass.edu ;; AUTHORITY SECTION: edu. 172800 IN NS E.GTLD-SERVERS.NET. edu. 172800 IN NS A.GTLD-SERVERS.NET. edu. 172800 IN NS G3.NSTLD.COM. edu. 172800 IN NS D.GTLD-SERVERS.NET. edu. 172800 IN NS H3.NSTLD.COM. edu. 172800 IN NS L3.NSTLD.COM. edu. 172800 IN NS M3.NSTLD.COM. edu. 172800 IN NS C.GTLD-SERVERS.NET. Among all returned edu DNS servers, we send a query to the first one. dig +norecurse @E.GTLD-SERVERS.NET any gaia.cs.umass.edu umass.edu. 172800 IN NS ns1.umass.edu. umass.edu. 172800 IN NS ns2.umass.edu. umass.edu. 172800 IN NS ns3.umass.edu. Among all three returned authoritative DNS servers, we send a query to the first one. dig +norecurse @ns1.umass.edu any gaia.cs.umass.edu gaia.cs.umass.edu. 21600 IN A 128.119.245.12 The answer for google.com could be: a.root-servers.net E.GTLD-SERVERS.NET ns1.google.com(authoritative) Problem 20 We can periodically take a snapshot of the DNS caches in the local DNS servers. The Web server that appears most frequently in the DNS caches is the most popular server. This is because if more users are interested in a Web server, then DNS requests for that server are more frequently sent by users. Thus, that Web server will appear in the DNS caches more frequently. For a complete measurement study, see: Craig E. Wills, Mikhail Mikhailov, Hao Shang “Inferring Relative Popularity of Internet Applications by Actively Querying DNS Caches”, in IMC'03, October 27­29, 2003, Miami Beach, Florida, USA Problem 21 Yes, we can use dig to query that Web site in the local DNS server. For example, “dig cnn.com” will return the query time for finding cnn.com. If cnn.com was just accessed a couple of seconds ago, an entry for cnn.com is cached in the local DNS cache, so the query time is 0 msec. Otherwise, the query time is large. Problem 22 For calculating the minimum distribution time for client-server distribution, we use the following formula: Dcs = max {NF/us, F/dmin} Similarly, for calculating the minimum distribution time for P2P distribution, we use the following formula: Where, F = 15 Gbits = 15 * 1024 Mbits us = 30 Mbps dmin = di = 2 Mbps Note, 300Kbps = 300/1024 Mbps. Client Server N 10 100 1000 u 300 Kbps 7680 51200 512000 700 Kbps 7680 51200 512000 2 Mbps 7680 51200 512000 Peer to Peer N 10 100 1000 u 300 Kbps 7680 25904 47559 700 Kbps 7680 15616 21525 2 Mbps 7680 7680 7680 Problem 23 Consider a distribution scheme in which the server sends the file to each client, in parallel, at a rate of a rate of us/N. Note that this rate is less than each of the client’s download rate, since by assumption us/N ≤ dmin. Thus each client can also receive at rate us/N. Since each client receives at rate us/N, the time for each client to receive the entire file is F/( us/N) = NF/ us. Since all the clients receive the file in NF/ us, the overall distribution time is also NF/ us. Consider a distribution scheme in which the server sends the file to each client, in parallel, at a rate of dmin. Note that the aggregate rate, N dmin, is less than the server’s link rate us, since by assumption us/N ≥ dmin. Since each client receives at rate dmin, the time for each client to receive the entire file is F/ dmin. Since all the clients receive the file in this time, the overall distribution time is also F/ dmin. From Section 2.6 we know that DCS ≥ max {NF/us, F/dmin} (Equation 1) Suppose that us/N ≤ dmin. Then from Equation 1 we have DCS ≥ NF/us . But from (a) we have DCS ≤ NF/us . Combining these two gives: DCS = NF/us when us/N ≤ dmin. (Equation 2) We can similarly show that: DCS =F/dmin when us/N ≥ dmin (Equation 3). Combining Equation 2 and Equation 3 gives the desired result. Problem 24 Define u = u1 + u2 + ….. + uN. By assumption us <= (us + u)/N Equation 1 Divide the file into N parts, with the ith part having size (ui/u)F. The server transmits the ith part to peer i at rate ri = (ui/u)us. Note that r1 + r2 + ….. + rN = us, so that the aggregate server rate does not exceed the link rate of the server. Also have each peer i forward the bits it receives to each of the N-1 peers at rate ri. The aggregate forwarding rate by peer i is (N-1)ri. We have (N-1)ri = (N-1)(usui)/u = (us + u)/N Equation 2 Let ri = ui/(N-1) and rN+1 = (us – u/(N-1))/N In this distribution scheme, the file is broken into N+1 parts. The server sends bits from the ith part to the ith peer (i = 1, …., N) at rate ri. Each peer i forwards the bits arriving at rate ri to each of the other N-1 peers. Additionally, the server sends bits from the (N+1) st part at rate rN+1 to each of the N peers. The peers do not forward the bits from the (N+1)st part. The aggregate send rate of the server is r1+ …. + rN + N rN+1 = u/(N-1) + us – u/(N-1) = us Thus, the server’s send rate does not exceed its link rate. The aggregate send rate of peer i is (N-1)ri = ui Thus, each peer’s send rate does not exceed its link rate. In this distribution scheme, peer i receives bits at an aggregate rate of Thus each peer receives the file in NF/(us+u). (For simplicity, we neglected to specify the size of the file part for i = 1, …., N+1. We now provide that here. Let Δ = (us+u)/N be the distribution time. For i = 1, …, N, the ith file part is Fi = ri Δ bits. The (N+1)st file part is FN+1 = rN+1 Δ bits. It is straightforward to show that F1+ ….. + FN+1 = F.) The solution to this part is similar to that of 17 (c). We know from section 2.6 that Combining this with a) and b) gives the desired result. Problem 25 There are N nodes in the overlay network. There are N(N-1)/2 edges. Problem 26 Yes. His first claim is possible, as long as there are enough peers staying in the swarm for a long enough time. Bob can always receive data through optimistic unchoking by other peers. His second claim is also true. He can run a client on each host, let each client “free-ride,” and combine the collected chunks from the different hosts into a single file. He can even write a small scheduling program to make the different hosts ask for different chunks of the file. This is actually a kind of Sybil attack in P2P networks. Problem 27 Peer 3 learns that peer 5 has just left the system, so Peer 3 asks its first successor (Peer 4) for the identifier of its immediate successor (peer 8). Peer 3 will then make peer 8 its second successor. Problem 28 Peer 6 would first send peer 15 a message, saying “what will be peer 6’s predecessor and successor?” This message gets forwarded through the DHT until it reaches peer 5, who realizes that it will be 6’s predecessor and that its current successor, peer 8, will become 6’s successor. Next, peer 5 sends this predecessor and successor information back to 6. Peer 6 can now join the DHT by making peer 8 its successor and by notifying peer 5 that it should change its immediate successor to 6. Problem 29 For each key, we first calculate the distances (using d(k,p)) between itself and all peers, and then store the key in the peer that is closest to the key (that is, with smallest distance value). Problem 30 Yes, randomly assigning keys to peers does not consider the underlying network at all, so it very likely causes mismatches. Such mismatches may degrade the search performance. For example, consider a logical path p1 (consisting of only two logical links): ABC, where A and B are neighboring peers, and B and C are neighboring peers. Suppose that there is another logical path p2 from A to C (consisting of 3 logical links): ADEC. It might be the case that A and B are very far away physically (and separated by many routers), and B and C are very far away physically (and separated by many routers). But it may be the case that A, D, E, and C are all very close physically (and all separated by few routers). In other words, a shorter logical path may correspond to a much longer physical path. Problem 31 If you run TCPClient first, then the client will attempt to make a TCP connection with a non-existent server process. A TCP connection will not be made. UDPClient doesn't establish a TCP connection with the server. Thus, everything should work fine if you first run UDPClient, then run UDPServer, and then type some input into the keyboard. If you use different port numbers, then the client will attempt to establish a TCP connection with the wrong process or a non-existent process. Errors will occur. Problem 32 In the original program, UDPClient does not specify a port number when it creates the socket. In this case, the code lets the underlying operating system choose a port number. With the additional line, when UDPClient is executed, a UDP socket is created with port number 5432 . UDPServer needs to know the client port number so that it can send packets back to the correct client socket. Glancing at UDPServer, we see that the client port number is not “hard-wired” into the server code; instead, UDPServer determines the client port number by unraveling the datagram it receives from the client. Thus UDP server will work with any client port number, including 5432. UDPServer therefore does not need to be modified. Before: Client socket = x (chosen by OS) Server socket = 9876 After: Client socket = 5432 Problem 33 Yes, you can configure many browsers to open multiple simultaneous connections to a Web site. The advantage is that you will you potentially download the file faster. The disadvantage is that you may be hogging the bandwidth, thereby significantly slowing down the downloads of other users who are sharing the same physical links. Problem 34 For an application such as remote login (telnet and ssh), a byte-stream oriented protocol is very natural since there is no notion of message boundaries in the application. When a user types a character, we simply drop the character into the TCP connection. In other applications, we may be sending a series of messages that have inherent boundaries between them. For example, when one SMTP mail server sends another SMTP mail server several email messages back to back. Since TCP does not have a mechanism to indicate the boundaries, the application must add the indications itself, so that receiving side of the application can distinguish one message from the next. If each message were instead put into a distinct UDP segment, the receiving end would be able to distinguish the various messages without any indications added by the sending side of the application. Problem 35 To create a web server, we need to run web server software on a host. Many vendors sell web server software. However, the most popular web server software today is Apache, which is open source and free. Over the years it has been highly optimized by the open-source community. Problem 36 The key is the infohash, the value is an IP address that currently has the file designated by the infohash.   Chapter 3 Review Questions Call this protocol Simple Transport Protocol (STP). At the sender side, STP accepts from the sending process a chunk of data not exceeding 1196 bytes, a destination host address, and a destination port number. STP adds a four-byte header to each chunk and puts the port number of the destination process in this header. STP then gives the destination host address and the resulting segment to the network layer. The network layer delivers the segment to STP at the destination host. STP then examines the port number in the segment, extracts the data from the segment, and passes the data to the process identified by the port number. The segment now has two header fields: a source port field and destination port field. At the sender side, STP accepts a chunk of data not exceeding 1192 bytes, a destination host address, a source port number, and a destination port number. STP creates a segment which contains the application data, source port number, and destination port number. It then gives the segment and the destination host address to the network layer. After receiving the segment, STP at the receiving host gives the application process the application data and the source port number. No, the transport layer does not have to do anything in the core; the transport layer “lives” in the end systems. For sending a letter, the family member is required to give the delegate the letter itself, the address of the destination house, and the name of the recipient. The delegate clearly writes the recipient’s name on the top of the letter. The delegate then puts the letter in an envelope and writes the address of the destination house on the envelope. The delegate then gives the letter to the planet’s mail service. At the receiving side, the delegate receives the letter from the mail service, takes the letter out of the envelope, and takes note of the recipient name written at the top of the letter. The delegate then gives the letter to the family member with this name. No, the mail service does not have to open the envelope; it only examines the address on the envelope. Source port number y and destination port number x. An application developer may not want its application to use TCP’s congestion control, which can throttle the application’s sending rate at times of congestion. Often, designers of IP telephony and IP videoconference applications choose to run their applications over UDP because they want to avoid TCP’s congestion control. Also, some applications do not need the reliable data transfer provided by TCP. Since most firewalls are configured to block UDP traffic, using TCP for video and voice traffic lets the traffic though the firewalls. Yes. The application developer can put reliable data transfer into the application layer protocol. This would require a significant amount of work and debugging, however. Yes, both segments will be directed to the same socket. For each received segment, at the socket interface, the operating system will provide the process with the IP addresses to determine the origins of the individual segments. For each persistent connection, the Web server creates a separate “connection socket”. Each connection socket is identified with a four-tuple: (source IP address, source port number, destination IP address, destination port number). When host C receives and IP datagram, it examines these four fields in the datagram/segment to determine to which socket it should pass the payload of the TCP segment. Thus, the requests from A and B pass through different sockets. The identifier for both of these sockets has 80 for the destination port; however, the identifiers for these sockets have different values for source IP addresses. Unlike UDP, when the transport layer passes a TCP segment’s payload to the application process, it does not specify the source IP address, as this is implicitly specified by the socket identifier. Sequence numbers are required for a receiver to find out whether an arriving packet contains new data or is a retransmission. To handle losses in the channel. If the ACK for a transmitted packet is not received within the duration of the timer for the packet, the packet (or its ACK or NACK) is assumed to have been lost. Hence, the packet is retransmitted. A timer would still be necessary in the protocol rdt 3.0. If the round trip time is known then the only advantage will be that, the sender knows for sure that either the packet or the ACK (or NACK) for the packet has been lost, as compared to the real scenario, where the ACK (or NACK) might still be on the way to the sender, after the timer expires. However, to detect the loss, for each packet, a timer of constant duration will still be necessary at the sender. The packet loss caused a time out after which all the five packets were retransmitted. Loss of an ACK didn’t trigger any retransmission as Go-Back-N uses cumulative acknowledgements. The sender was unable to send sixth packet as the send window size is fixed to 5. When the packet was lost, the received four packets were buffered the receiver. After the timeout, sender retransmitted the lost packet and receiver delivered the buffered packets to application in correct order. Duplicate ACK was sent by the receiver for the lost ACK. The sender was unable to send sixth packet as the send win
SakEmail components Copyright ?1997 - 2003 Sergio A. Kessler web: http://groups.yahoo.com/group/sakemail/To subscribe to the mailing list of sakemail, just go tohttp://groups.yahoo.com/group/sakemail/History:0.9 - First released version0.9.1b -Fixed when a mail server reply on the connection with more than one line0.9.2b - I forget to return a value in functions retrieveHeader/Message =) and fixed it. Some minor bugs that I don‘t remember fixed.- Added MIME-compliant base64 support (not for use by now). Added examples.0.9.2.1b- Fixed a bug when send a mail and the first line disappear (thanks to Arun)- Now, you could do MySMTP.MsgTo := ‘a@doma.com; b@domb.com;c@domc.com‘; the spaces before/after semicolon doesn‘t matter (I hope ;)).0.9.3b- Many changes, I added a SakMsg component that make send binary attachments a snap. But have one problem, if you send as attach a file > 20 Kb, it doesn‘t work (I don‘t know why, maybe a problem of sockets). Developed with a version 2.0b of WSockets and D3.0.9.3.1b- Changed the POP.login to a function that return the number of new msgs.- Added the event OnRetrieveProgress on the SakPOP, and fixed the example, sorry =)- Minor changes to the code.1.0- Developed with WSockets 1.2 POP.Login now return a boolean depending id the user is authorized, and POP.Init return the number of new msgs.1.01- Fixed a bug with a bounced mail.1.02- Minor bugs fixed (some variants of boundary)14/10/971.1.0- Warning: WSockets1.2 have some bugs that result in bad attachments. So I decided to use the sockets of Delphi 3 founded in D3 c/s D3.01 pro and D3.01 c/s. Now all seems to work fine and much more smooth. And of course the interface of SakEmail hasn‘t changed.26/10/971.2.0- Added the Reply-To field to TSakMsg comp. Now you must use ‘,‘ when you want to send the msg. to multiple recipients, i.e.: ‘a@doma.com, b@domb.com,c@domc.com‘ This change is done for better compatibility with other emails clients.- Better formatting of the field Date of TSakMsg. Some changes to the code.17/11/971.2.1- Now, all searches are made in case-insensitive, it could prevent some unexpected responses (no one reported, but...). Some changes to the code (again).20/11/971.2.2- Some bugs fixed. (Thanks to Serge Wagener from .lu)24/11/971.2.3- Added the field ‘MIME-Version: 1.0‘. It seems that is necessary :)25/11/971.3.0- Added compatibility with SCO and VAX servers. Fixed a minor bug with the boundary.- Change the generator of the message id.- Added the field MessageId and InReplyTo to the TSakMsg component.- Added the field In-Reply-To that is added to the message generated when it is <> ‘‘.30/11/971.3.1- Almost rewrote the parsing code. Now is more easy for you if you want hack/modify the code.- Better treatment of emails with html inside.15/12/971.4- Added support for UUCoded attachments.- Added a small delay when sending the email, seems that some servers can‘t deglut the info too fast, causing problems with sockets buffers and leading to crash the client machine, I don‘t know if is a Borland bug or Microsoft bug. (thanks to Don Higgins).19/12/971.4.1- Fixed a bug that send double ‘<‘ and ‘>‘ (ie. <>) when the full user name is used. Check the new SMTP demo. Thanks to Serge Wagener for locate this bug, track it down and send me the fix.2/2/981.5.0- Added the Canceled property to TSakPOP and to TSakSMTP. Due to this addition now RetrieveAllMessages is a function that return the number of msgs. retrieved and SendMessage is a boolean function (maybe someone has pressed the cancel btn).- Fixed a bug when the subject field is too large.9/2/981.5.1- Fixed a bug with a message within a message (recursive msgs).18/2/981.5.2- Fixed a bug what happens when after the field ‘To:‘ appear a blank line(Thanks to Osvaldo Fillia). Fixed a bug when sending email to more than two address (the separator is still ‘,‘).9/3/981.6.0- Sometimes the filenames of an attachment contain invalid chars making very dificult to open a TSaveDialog (you have noted this ?), now SakEmail deletes the invalid chars.- Applied a patch from Matjaz Bravc, that resolve the problem of localized dates, letting you choose (in design time) if you want localized dates (NOT recommended) or standards dates (english) via the LocalizedDates boolean property in the TSakSMTP comp. Thanks also to Serge Dosyukov for sending me a fix.- Also I applied another patch of Gregor Duchalski that cure a bug with PChar when this unit is used under NT. - It seems that some machines need more delay when sendig a msg (see previous posting 19/12/97), thanks to Matjaz Bravc.- I discover a bug in the transparency code, it is fixed now. Did you see the benefits of Open Source Software ? :)26/3/981.6.1- Added a FUNCFileName private variable to manage the complete path of the attached file. I receive problems reports with this, it work now ?.- Reduced the line sleep to 30 (tell me if this value doesn‘t work for you).27/4/981.7.0- Fixed a memory leak, thanks to Don Higgins.- Moved the string esErrorInFormatOfMsg to a property of SakPOP.- Because some people need to use IP addresses instead of Host names, I‘ve added a new property IPAddress to SakPOP and SakSMTP. If both are filled, then the Host name will be used, thanks to Roger F. Reghin for reporting this. The side effect for this is that YOUR app must check if the host is a host name or a IP address, in my app I remove the periods and try to convert the result to a float (long integers don‘t work, but float accept chars ‘e‘) if it doesn‘t work I assume that is a host name (someone has a better and simple idea ?).- Added the property FileStream to the class TAtachedFile and the procedure SaveToStream, this was done by Brian Sheperd- The address separator (in the TO: field) is ‘,‘ and ‘;‘ now (before it was ‘,‘ only).1.7.1- Roger F. Reghin has sended me a pair of nice patches that resolve in a good behavior when the destination address is something like "Roger Reghin" and some servers says that they couldn‘t relay that mail, etc. Also Roger has made the IPAddress property obsolete (do not use it, use Host instead), SakEmail will resolve the host properly no matter if it is a host name or a IP address. So in the next version I will remove the IPAddress property. Thank you, Roger.1.8.0- Well, it seems that I made a mistake, I investigated the previous behavior and it is a fault of the SMTP (RFC 821), so I fixed it.- The IPAddress property has been removed, use Host. Goeran Strehl (asem) has sended me a patch that fix a memory leak and one problem with the object inspector and the Text property of a SakMsg. Dmitry Bondarenko say that some servers do not send the msg size after the RETR command, so he fix that issuing a LIST n command first (work nicely).- Added the property CC (Carbon Copy) to the SakMsg object.1.8.1- Added the property ReturnPath to the SakMsg comp. Minor changes to the scanning code for the filename of attachments.1.8.2- Fixed a bug with the filename of attachments (thanks to Taufer Pavel Ing.).- Added the function IsIPAddress from hou yg (the actual code don‘t work if the server is 265.net :) Fixed a minor bug with html pages like attachments. Some fucking email server return a bounded message declaring the boundary like ‘boundary = ‘ and not ‘boundary=‘ wich is clear in the RFC, fixed.1.8.3- A obscure bug was found by HuangYeJun from china, in the RetrieveHeaders function if the retrieved text was larger than 1024 bytes and the crlf.crlf fall in the middle of two chunks, the function is blocked. I don‘t use this function, btw.1.8.3.1- Just cleaned up a bit the FindUUAtachs function. Not bug or enhancements release. Serge Wagener put me to work >:|1.8.4- Dmitry Bondarenko (again) has found a bug in wich I do not respect the RFC, wich say that replys from the SMTP server could be multi-line, and the previous version just manage as far as two lines. He also send me a nice patch, so the bug is fixed.- Craig Manley added a ExtraHeaders property, please, use with care, it‘s just not valid to put inside it whatever thing.- The CC header was not being added to the headers that were being sent, so Craig fixed it.- Warning: I‘ve put try/except in the TSakPOP.Connect and TSMTP.Connect function around the line FSocket.Open, so you will need to write something like: myPOP.Connect; if POPError then ... in your code, the old way was: try myPOP.Connect; except ..... end; If you are strongly opossed to this change, drop me a line and tell me why (I‘m in doubts).1.8.5- Greg Nixon added the priority property. The default priority for each msg created will be prNormal, so you don‘t need to change your code any bit.1.8.6- Ulf Sturegren has added D4 compatibility, not many changes to the source (one letter), but he found the error.- Hou yg has sent to me a revisited IsIPAddress function, so I put the newer function in, infortunely my reply to him doesn‘t want to go.1.8.7- Ok, I discovered a weird bug, some old emailers (navigator 2) does not format the message in multipart mode if people send an attach, without writing any text and with no MIME settings. Fixed. This could be serious, I recommend upgrading.1.8.8- A small fix with the CC field. Some stupid mail servers put tabs in some fields (CC:, TO:) when they want to make a new line, the correct is to put at least a space in the beginning of the line, added a little code to "fix" that.1.8.9- Some ‘moderns‘ pop3 servers doesn‘t support the LAST command, so I‘ve added a little code to cope with this and added a boolean property ServerSupportLastCmd. See TSakPOP.Init for more details. Reported by Jan Najvarek.1.9.0- Kaufman Alex has added two properties to the SakMsg object, the ContentType and the Headers property, that should be self explaining (I modified a little the code he sended me, btw).1.9.1- I rewrote and greatly simplified the code that deal with the multiple address in the TO: field and remove some possible bugs in it.1.9.2- Alex discovered and fix a bug when a file attached is not enclosed between quotes, resulting in the filename without the first and last character.1.9.3- Better detection of the boundary in multipart messages. Fixed a bug when the attached file is empty.1.9.4- Chris G黱ther send me *lots* of memory leaks fixes, very good job, Chris. - Some weird PGP messages are now processed well.- Yang Qiandong from china fixed a compiler hint and a warning.- Modified TSakSMTP.FReceiveTextFromSocket as suggested by Greg Nixon.- Dmitry Bondarenko send me a patch that fixes some issues with the LAST command (that some servers don‘t implement) and other patch that fixes a problem when servers add spare words in the tail of the answer.- Some minor changes suggested by Matthew Vincent.- Support for _big_ attachments files (me).- Make the code more modular and simple (still is not very modular).1.10.0- Move some stuff to a sak_util unit.- Support for quoted-printable msgs, thanks to Chris G黱ther.- Fix the BCC field.- New property sakMsg.ContentTransferEncoding.2.0.0- Major reestructure of the files and the source code.- Simplifyied sakPOP3.pas a _lot_- Support encapsulated messages (message/rfc822).- Nested multipart messages are processed fine.- Attachs with quoted-printable are processed fine.- Many bugs fixes.2.0.1- A fiasco, sorry.2.0.2- Fixed a bug in the sak_CleanUpAddress.- Do the rigth job if the ContentType is ‘plain/text‘ and the encoding is base64.- Redone sak_ExtractAddress and sak_ExtractAlias.- New ‘Sender‘ property in SakMsg (normally not used, so do not use it, unless you know what you are doing) ‘Thanks‘ to Alex Kaufman for this.2.0.3- A *severe* bug with multiple addresses was fixed.2.0.4- Fixed bogus Message-number (Message-id is the correct) Thanks to Peter Honan- Added SizeInBytes property to the SakMsg component. (petition of Alex Kaufman)- Fixed a minor bug in TSakPOP.RetrieveHeaders. Fix from Alex.- Added RetrieveMessageOnlyHeaders and - RetrieveAllMessagesOnlyHeaders.2.0.5- Fix when the mail server reply is like (two cr).- Fix function IsIpAddress.- Both fixes by Alessandro Rossi.2.0.6- Fix a bug in the sak_Base64Decode function when the data to decode is null (I found it in the hard way).- Andy Charalambous make it sure you can send more than one email without disconnecting and connecting again.- And Chris ‘Memory Hunter‘ G黱ther killed some memory leaks (again).2.2.0- the f* sleep line that was bothering us for years is gone, gone, gone. Thanks to Syed Ahmed.- a getUIDL method of SakPOP. Thanks to Alex Kaufman.- a UIDL property on SakMsg. (me)- a SakPOP.GetUIDLsOnRetrieve boolean property (default false) (me)- change some ‘Exception.Create()‘ to ‘raise Exception.Create()‘ Thanks to Anton Saburov.- change SakPOP.Init from function to procedure (me)- new SakPOP.NewMsgsCount property (me)- changed SakPOP.Password to SakPOP.UserPassword (me)- changed SakPOP.ErrorInFormatOfMsg to SakPOP.StrErrorInFormatOfMsg- OnLookup event on SakPOP and SakSMTP. Thanks to Syed Ahmed.- OnConnecting event on SakPOP and SakSMTP (me).- OnReceiveTextFromSocket event on SakPOP and SakSMTP (me). (mostly for debug)- OnSendTextToSocket event on SakPOP and SakSMTP (me). (mostly for debug)- Headers are retrieved without the mail body (ugly bug, fix from Alex Kaufman)2.4.0- I‘ve revamped TSakMsg, many funcionality from SakPOP was moved to SakMsg, where it belongs.- Now SakMsg has a RawMail property wich you may find useful, now you can do: SakMsg1.RawMail.LoadFromFile(‘(uidl).mail‘); SakMsg1.ParseMsg; or SakMsg1.RawMail.LoadFromStream( myStream); SakMsg1.ParseMsg; or SakMsg1.RawMail.SaveToFile( ‘(uidl).mail‘); etc, etc...- Added a property TSakMsg.ClearRawMailAfterParse for memory saving.- the return of the f* sleep line (it causes freezes on winsock 1.1 systems like win95, win98 has winsock 2 so there is no problem if you remove the line)- lost of the DecodeProgess events :( (sorry, I don‘t know how to fit this events on the new SakMsg)2.6.0- the sleep() line is dead, it will never come back. Sending an email is a pleasure now.- SakMsg has a TextEncoding (8Bit, Base64) property, I think this will be useful to people with others charset than iso-8859-1- the base64 routines have been rewritten, they are more OO and faster (they are now in SakMIME.pas).- cosmetic changes all over the place.2.6.1- simplifyed ParseMsg2 a lot, it work better now.- speed up the search for uucoded attachs (the previous search was very dumb)- fixed bug Msg.SizeInBytes always 0- added a couple of Application.ProcessMessages to make the app more responsive.2.6.2- moved some functions from sak_utils to SakMIME.- make const parameters all over the place.- fix the bug that introduces a final crlf in quoted-printable attachs.- fix a division by zero if attached file is 0 bytes long, fixed by Peter Kollanyi.2.6.3- fix a rare bug when the header of a email (more probably a encapsulated one) has first line/s in blank. Easy and innocuous bug.2.6.4- fix the bug that insert the attachs of type text/* on the body of the email.- change the Smtp.SendMessage for Smtp.SendTheMessage to avoid a BCBuilder problem. Both problems reported by Andreas Franzen. SendMessage is still there, but it‘s now deprecated, I will remove it in the future.2.6.5- moved the ParseMsg activation from SakPOP to SakMsg (where it belong), this means that after setting the RawMail property of SakMsg, this does a ParseMsg automatically. before: SakMsg1.RawMail := ... SakMsg1.ParseMsg; now: SakMsg1.RawMail := ... hope I‘m not breaking too much code out there ... :)- some changes in the way attachments are processed (now the html part is separated correctly and images within the html are recognized)- RetrieveMessage() and RetrieveMessageOnlyHeaders() now take an additional parameter, a TSakMsg var, so people can change some parameters before parsing, see the source in SakPOP3.pas (the old way is still supported, but they will be removed in the future)- bug fixes that I do not remember.3.0.0- moved code around.- removed deprecated functions (I told you about this)- new SakAttFile unit.- Base64Encode( AttFile), Base64Decode( AttFile), UUDecode( AttFile) has been moved to the TAtachedFile object, so you can do AttFile.Base64Encode, AttFile.Base64Decode, etc- SakSMTP have lost EncodeStart, EncodeProgess and EncodeEnd events as a consequence of the previous change.- SakPOP.Canceled and SakSMTP.Canceled properties have been made read-only and SakPOP.Cancel and SakSMTP.Cancel procedures (or methods) have been added.- add a SakMsg.FillRawMail method that will fill the RawMail property with a rfc822 message based on the properties of SakMsg.- changed SakSMTP.Quit & SakPOP.Quit to Disconnect- deleted TAttachedFile.FileStream (redundant), use BodyBin- removed the function sak_getTempFileName (as it should no be trusted) use function sak_GetTempPath- the new SakIMAP component !, this make a pleasure to work with incoming emails (as you can have folders, etc). Note: the IMAP component has only been tested with the Uni. of Washington server, but it should work with any *STANDARD COMPLIANT* server. Anyways, the code of this component is very simple, so if you have problems, a look in the source code can enligthen you.3.0.1- fixed a brown paper type of bug.3.0.2- support the case where attachs do not come from files (Lars Karlslund)- minor bugfix in UUDecode function (Lars Karlslund)- if the SakMsg.Username is empty, do a VRFY command at the smtp server to try to get the full user name (sergio)- function TSakIMAP.GetFolderList (Peter Nagel)- function TSakIMAP.GetHierarchyDelim (Peter Nagel)- frustrated intent (ie. commented out) to remove memory leaks in POP, SMTP & IMAP destroy functions (Ronald Moesbergen)3.0.3- actually create (and free) the FolderList in sakIMAP (Neculau Andrei)- try to send the FQDM to the HELO command in SMTP (sergio)- commented out the VRFY command in SakSMTP, and cut the from address in the From field (in SakMsg), so if the username is empty, the SMTP server rewrite the from address in a complete way, with username & full address (sergio)- fix a minor bug in TBase64DecodingStream.Write function (Lars Karlslund)3.4.0- many, many improvements to the IMAP component by Peter Honan (I applied the patch with minor modifications, mainly to respect delphi coding standard, taking out the overloading, the selectFolder function was overcomplicated, etc)- FAQ updated (me)3.4.1- minimize the chance for two temporal messages stored on disk to collide (can be hit in previous versions if you run multiple instances of retrieveMessage at the same time)- FAQ updated.3.4.2- a new sak_CleanUpAddresses() implementation, by Knut Baardsen- better handling for temporal messages, suggested by Andrew- many improvements (including ACL -Access Control List) to the IMAP component by James Chaplin3.4.3- reverted to the old sak_CleanUpAddresses() implementation Knut‘s one is almost rigth, but don‘t let us use addresses without domains- add Headers.Clear before filling headers, by "Antonio Carlos Ribeiro Faria" 3.5.0- add TSakMsg.LoadFromTextFile from Oak Chantosa- big jumbo mambo patch from James Chaplin first patch: 1) Operation timeout - OperationTimeout timeout for non-responding receive operations. 2) Forced abend - ForceAbend method that will disconnect and reset state. 3) Optional folder lists - AvFolderList and AvSUBFolderList provide alternatives to FolderList and SUBFolderList that ensure the lists do not contain inacessible folders ( flagged by the server ). 4) Folder name fix - Provided a function to "fix" folder names before submission. Currently it fixes names containing spaces. second patch: 1) Capability - Ask for server capabilities/extensions. 2) Noop - Basic noop command - updates message counts as well - preferred alternative to status. 3) Status - Explicit status command - generally useful for status of a non-selected mailbox. 4) Fetch - Retrieve message data. 5) FetchBody - Retrieve the body of the message. 6) ExamineFolder - A read-only select command. 7) CloseSelectedFolder - Close the currently selected folder. 8) Idle - RFC2177 extension - not implemented on very many servers. 9) Search - Search based on RFC2066 criteria. 10) UIDSearch - Search based on RFC2066 criteria - results are in UID form. 11) UIDStoreFlags - Store message flags based on UID. 12) UIDFetch - Fetch message data by UID. 13) UIDCopyMessageToFolder - Copy a message by UID. 14) Authenticate - Basic framework. Only plain authentication extension implemented. 15) CloseOnError - A new property that allows the user to turn off the default behaviour of disconnecting from the server when an IMAP error is received 16) Namespace - RFC2342 Namespace query command. 17) ListFullHierarchy - Property which allows a switch between "*" ( default ) or "%" as the wilcard for default folder/list methods. 18) List - Explicit list command in case it is needed. third patch: 1) fix problem with imapd 2001a, reported by Holger Mauermann. 2) remove all warnings.3.5.1- revert change to the base64 encoding routine.3.5.2- changes from James Chaplin: 1) TSakIMAP will now properly process non-numeric UIDs for messages ( there was a sak_StrWord2Int transform being used before - which always produced a 0 value for non-numeric UIDs ). 2) TSakIMAP.RetrieveMessageExt ( private method ) was modified to provide a retrieval by either MsgID or UID. 3) TSakIMAP.RetrieveMessageByUID was modified to use the slightly more efficient TSakIMAP.RetrieveMessageExt(UID) method specified in 2) above. I also made an update to the SakMIME.pas unit. The changes that were implemented are: 1) sak_Base64Encode - a basic Base64 encoder. String input and string output with the option for CRLF splitting. 2) sak_Base64Decode - a basic Base64 decoder. String input and string output with a control for CRLF interpretation. 3) sak_Base64Verify - a very basic Base64 string verifier.3.5.3- robustify and code cleanups by Paul Vernon.3.5.4- access violation fix by Paul Vernon.3.5.5- go back to good old trusty 3.5.23.5.6- this time, all the cleanup & fixes from Paul Vernon seems to work well.3.6.0- Paul Vernon latest minor fixes- added basic SMTP authentication, by Delfi and Antonio Carlos Ribeiro Faria3.6.1- fix a mayor bug when sending to many addresses (by sergio)3.7.0- add full support for html mails, by Paul Vernon. (The TAttachedFile now has an extra boolean property called embedded. This property lets you use the syntax in your HTML mails)- fix a weird typo for BCC fields- add Content-ID, by alejandro Castro- fix "_" characters in subject, regression fix.- cleanups all around, by Paul Vernon.- SMTP example updated to cope with html emails.**warning** from this version, the html part of mails will not be stored as attachments by default, if you want this behavior, you just do something like: aSakMsg := TSakMsg.Create( self); aSakMsg.HTMLAsAttachment := true; ...3.7.1- fix TSakMsg.PopulateList (Jalin)3.7.2 (codenamed "melissa")- fixed a bug when the Populatelist procedure got re-written in sakMsg. It wasn‘t populating the SendTo field if there was only one e-mail address... (Paul Vernon)3.7.3 - Congratulations to Sergio on the addition to his family. This release was made by Paul Vernon who has temporarily taken over the release functions for the SakMail components whilst Sergio spends time AFK!- The 3.7.2 bug fix added blank entries to the address lists. The PopulateList procedure has been re-written again to hopefully cope with any type of e-mail address formatting.- The SMTP example noted in 3.7.0 actually shipped with this release!3.7.4- Bugfix for detecting UUEncoded mails correctly. Previous versions processed MIME mails with the value ‘begin xyz‘ if it appeared at the beginning of a line as a UUEncoded mail when they should not have.- POP and SMTP connect procedures are now functions. Existing code is unaffected. However, you can now use the following code if (sakPOP.Connect) then begin end;- POP gracefully quits if it receives an error now by calling Disconnect correctly.3.7.5- Further code to improve identification of UUEncoded mails. Essentially looking for the end as well as the beginning to ensure that it is correct.- Code optimisation of certain UUEncoded mail id functions.- Fix to ensure that the body of a mail that is UUEncoded is not lost.- MIME-Version string introduced into TsakMsg component to help with UUEncoded mail identification.- SizeInBytes property altered to read private variable using a function. If the private variable is 0, the function reads the length of the FRawMail.Text property.- Fix to make sure that the filename is not overwritten by a blank value when parsing mail-headers.3.7.6- Fixed list index out of bounds error.- Added POP3 RSET call TSakPOP.Reset.3.7.7- Altered SizeInBytes and Octets values to return server-side size when d/l headers only and use actual size once the entire message is downloaded.- Fixed a bug in GetBasicHeaders where To and CC fields could be mishandled if the mail headers were formed in a particular way.4.0.0 beta- All methods are now wrapped in classes. sak_util is now included for backwards compatibility only.- Several changes to make sakMail thread safe including the introduction of Mutexes which are cross process safe. Critical sections were an option however, although mutexes are a little slower, they are much more effective when you aren‘t sure how the code is going to be deployed...- Made several changes to the way connections are tracked, now making better use of the underlying Delphi components own properties and functions.- Several bug fixes included from solutions posted on mailing lists. Including change to datetime function to respect local time separator. There are more including one that Adem re-raised.- Removed almost all pointers as per Adems suggestion. Makes for neater code.- Hopefully backwards compatibility is kept. This is one of the objectives of the excersice although, internally, the components no longer use any of the non-object based methods. Also some of the non-object based methods actually have been re-written to create an object use the instance of the original method and then destroy the object again. This introduces a minor overhead however, because the objects are discreet, the trade is for much better memory usage and greater thread safety.- Introduced an include file to define compiler directives. Currently there are two directives. One defines whether to use the VCL or not, the other defines whether or not to use the FastStrings components. - With the intoduction of the Include file, this allows the development of code that is optional for users. One of these such changed is the use of the FastStrings base64 decoder. If you install the FastStrings components and turn on the compiler directive, you should have no functional changes however, the base64 decoder routines should have a much higher performance rating. Tests clock in at over 2000% faster attachment decoding on a P4 1.8GHz machine. (1.2Mb file 1686mS native sak Base64 Decoder, 79mS using FastStrings!)- This version is being released as a beta as the changes are pretty drastic. If the code is deemed to be stable and backwards compatible then it will be re-released as v4.0.1 with no changes.4.0.1 beta- Fixed an issue where Range Checking highlighted that the Attachment b64 decode routine raise a Range Error if the line that was to be decoded was empty. i.e. ‘‘.- Introduced a compiler directive to turn off range checking in the sakMIME procedure TBase64DecodingStream.Write to make sure that it runs correctly as Range Checking causes issues in this function.4.0.2 beta- Changed MailDateToDateTime function to the one provided by DengZhaoHui with a few modifications as even though it has better date processing than the original it caused EConvertErrors with some non-rfc dates.- Added the compiler directive to allow the inclusion of MD5 components from the DCPCrypt suite of encryption components. This allows the components to do APOP and SMTP AUTH functions as specified in RFCs 2095, 2104, 2449 and 2554. {UseDCP} ***** NOTE: These functions are experimental as although they are RFC compliant, they have not been tested against a secure mail server yet... *****- Using EurekaLog during load testing of the POP mail component, found and fixed several AV‘s in sakMSG, sakMIME and sakPOP. Mainly simple mistakes that required re-ordering of code or more checks before trying to manipulate data.- Altered the sockets code to be more stable with some servers. The previous implementation was totally incompatible with SendMail NT v3.0.2.- Fix added to compensate for incorrect operation of Connected property in some versions of Delphi.- Altered GetMultiLineFieldBody as per Adems suggestion. Also took some of Adems code and added it to GetFieldValueFromLine as the escape characters can appear in single line headers as well as multi-line ones.- TClientSocket is deprecated in Delphi 7. This may be the next large change in the sakEmail components. - Updated distribution to include more RFC‘s regarding the message format, POP and IMAP and hashing functions for CRAM mechanisms.- Fixed the handling of redirected mails as created by Eudora.- Force PopulateList to clear the list before populating it again.- Created a Delphi 6 package file.4.0.3 - Fixed AUTHSMTP buffer initialisation error. (Dmitry G. Kozhinov and Gabi Slonto)- Improved identification of servers that do not support the UIDL command. A small overhead is intorduced on servers that do support the command and have several mails to download but the feature allows better interaction with those servers that do not support UIDL.- Priority is now reported correctly when an e-mail is being decoded rather than only being used when sending an e-mail.4.0.4- Fixed an issue with a malformed header in a mail sent from MS Word through an Exchange server- Added a couple of try...finally blocks to the sakIMAP component.- Altered the sakIMAP components connected function to mirror the more accurate sakPOP method.- Consolidated all compiler directives into sakDef.inc- Added versioning compiler directives to allow the compilation of sakemail under Delphi 4.- General tidying of code. 4.0.5- Created a Delphi 7 package- Added properties to the IMAP component to allow read access to the LocalAddr and LocalHost socket properties.- Bugfix to sakMsg PopulateList function where a comma separated list did not contain any spaces- Access violation in sakPOP component due to incorrect use of free,freeandnil and compiler directives4.0.6- Added several features to the IMAP components.- Tidied up SMTP authentication routines (Improved use of MD5 for authentication using DCP components)- Included capability to send messages without an SMTP server (using Indy DNS components for MX lookups)- Bugfix in message parsing to stop a recursion loop due to a malformed mail.4.0.7- Memory leaks found by Amos and Paul regarding the sakMsg and sakPOP units respectively.- Bug fixes to attachment save code including stripping out invalid .. sequences from filenames- Improved the GetConnectedState method to check against the RemoteHost value on the Socket.- Updated POP example to be more responsive when downloading mail. Fixed a memory leak.Don‘t forget to subscribe to the mailing list (see the web pages at http://groups.yahoo.com/group/sakemail/)
******************************************* ************ WPTOOLS 6 History ************ ********** http://www.wpcubed.com ********* **** Copyright (C) 2012 J. Ziersch and **** **** WPCubed GmbH, Munich, Germany ******** ******************************************* * - = bug fix * + = new feature * * = changed, expanded feature ******************************************* * IMPORTANT: Please edit file WPINC.INC to * activate WPREPORTER, WPSPELL, wPDF, TBX * and GraphicEx / PNGImage support! * Delphi 2009 and later as inbuilt PNGImage ******************************************* * THE MANUAL (PDF) AND REFERENCE (HLP,CHM) * ARE PROVIDED AS SEPERATE DOWNLOADS! ******************************************* ~~~~~~~~~~ Important Notes ~~~~~~~~~~~~ - With Delphi XE2 or XE3 it is possible to compile WPTools into a 64 bit applications. You need WPTools 6 PRO or WPTools 6 Premium for this. - Localization and Inch/CM selection see http://www.wpcubed.com/manuals/wp5man/index.html?localization.htm Also see demo\tasks\localization. There are the required XML files. - if flag wpDontAddExternalFontLeading is active in property FormatOptionsEx the text will be formatted more like WPTools4/MS-Word. You can alternatively set global variable WPDoNotAddExternalFontLeading := TRUE and select the Printer WYSIWYG mode: TWPRichText1.HeaderFooter.UpdateReformatMode(true) - The FormatOption wpDisableAutosizeTables can be required for good display of tables. wpNoMinimumCellPadding for narrow layout, too. - If you plan to use DBWPRichText please check http://www.wpcubed.com/forum/viewtopic.php?p=3456 - The reader/writer receive their options through "FormatStrings" please see list at http://www.wpcubed.com/manuals/formatstrings.htm - WPTools is configured using the file WPINC.INC, here WPREPORTER is activated and the optional WPShared, WPSPell and wPDF can be selected to be compiled into main WPTools package. - also see FAQ: http://www.wpcubed.com/forum/viewforum.php?f=15 - and support forum: http://www.wpcubed.com/forum/ - WPTools 6 PRO and PREMIUM will be compiled as Version 5 if $define "WP6" is not set ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We recommend to use TMS components to create modern user interfaces, toolbars and ribbons. 19.2.2013 - WPTools 6.29.1 - fix in rtf writing code to solve problem with merged cells - fix possible rangecheck error - fix problem with TextObject.LoadFromFile and Delphi XE3 * RTF reader now handles UNC file links which use "\\" in the path * the cursor was not painted if DoubleBuffered was set to true for the parent of the editor + WPTools Premium: Saves and loads \column * improved theming of TWPToolbar and TWPToolPanel + new event: OnPaintDesktopBackground. It can be used to draw the parent of the editor, for example if it is a TMS panel or pager control. Example: procedure TForm1.WPRichText1PaintDesktopBackground(Sender: TObject; Canvas: TCanvas; R: TRect); begin // This would paint the TWPRichText, too - but TWPRichText is locked for repaint during this event AdvOfficePager1.PaintTo(Canvas, -WPRichText1.Left, -WPRichText1.Top); end; - HighlightTextColor can now also be used if 2Pass Painting is used 21.12.2012 - WPTools 6.29 - images in RTF label were not painted when label was moved + added support for XE3 to WPTools STD edition * stream RTFvariables were not loaded from WPT format. They are loaded now. 9.11.2012 - WPTools 6.28 - Update to RTF reader to load landscape flag for sections better - when page mirror was used, after a page break the text indentation was sometimes wrong - hyphenation code was broken - workaround for word files which have space characters in table definitions 16.10.2012 - WPTools 6.27'' * some additions to the PRO edition for XE3 26.9.2012 - WPTools 6.27' * The PRO Version now supports Delphi XE3 3.8.2012 - WPTools 6.27 - fix for wrong display of tables with header and footer rows. Sometimes both were painted without any data. + to load old Hiedit templates as RTF code use the formatstring -HiEditFields. This will create merge fields for ALL fields. - NL sign was not shown right after CTRL+ENTER was pressed (requires ShowNL) - fix for rangecheck exception with paintpages array - fix for footer and page mirror - doubleclick word selection now stops at NL - Workaround for Windows Spooler problems - some images would get lost - sections use footer and header of previous section, not general - ASetBorder did change all border types 12.3.2012 - WPTools 6.25.4 * allows changing of column width in redonly editors. Can be switchoed off in EditOptions or set compiler define TOTALREADONLY + wpDisableSelectAll in EditOptionsEx2 * changed reformat/repaint after Undo/Redo - pro and premium: Due to a problem with precompiler cursoir selection did not work correctly 8.3.2012 - WPTools 6.25.3' - borders for paragraphs with multiple lines were not drawn correctly - change in DBWPRich.pas to use LoadFromString instead of Text - fix possible range check error - change in WPTbar.pas to use different default for BevelOuter - change in WPIOHTML to use default charattr of paragraph is a paragraph is empty 9.2.2012 - WPTools 6.25.2 * new 2-pass painting triggers CustomPaint event only on second loop (when the text is painted) * changed protection of empty paragraph to WPTools 5 way + inside of the OnPrepareImageForSaving event it is now possible to set Writer.CurrentImageName to the name of the file which should be saved. This is only useful if ObjRef = nil and so no ObjRef.Filename can be set. + TParagraph.GetSubText now has optional parameter to disable the object reference char codes #1, #2 and #3 * HTML writer ignores #13 codes when writing the text. * additional savety code in HTML writer * HTML reader sets image width and height to contents, (if possible) * HTML reader: changes UTF8 handling for UTF8Sequence=1 * HTML reader does not stop on \0 anymore * HTML writer writes img tag also for empty images IF a name is provided in event PrepareImageforSaving * HTML reader does not add space at end anymore - the UNICODE reader uses attribute of current paragraph. This is important for consistent behaviour between ANSI and UNICODE version - when writing HTML background color is not set to white when shading is 0. 0 is treated as default value. - image align center and right now works in HTML - fix an endless loop when image was too large - improvement of table border drawing - improvement for right align in table cells + numbering will be used when wpFormatAsWebpage was set in AsWebpage 10.1.2012 - WPTools 6.25 + HTML reader reads cell heighs - RTF writer writes background color easier to understand by Word * improved XML reader/writer (unit WPIOXml1) * improved word wise cursor movement when fields are used + new "paint attributes" mode. Use WPRichText1.BrushStart to select this mode. - dashes were not painted using the current font color - some stability improvements 7.11.2011 - WPTools 6.22 + procedure TParagraph.CellSelectionAdd; + procedure CellSelectionRemove; + EditOptionsEx2: wpCellMultiSelect - allows multiselection in tables when CTRL is pressed + improved XML import/export (unit WPIOXML1.PAS) - some smaller bugs fixed 3.11.2011 - WPTools 6.21.2 - fix problem with TWPToolButton - improved HTML writer to write parameters in "" - improved display of arabic text 24.10.2011 - WPTools 6.21.1 - fix problem when painting insertpoints after tab stops. They were painted two times. - fix in XML and HTML writer. Close tags when extracting text from fields 19.10.2011 - WPTools 6.21 + CodeLocate can now also accessed in non-visual TWPRTFDataCollection and not just TWPCustomRtfEdit + CodeSelect can now also accessed in non-visual TWPRTFDataCollection and not just TWPCustomRtfEdit + TWPRTFDataCollection.CodeLocatePair( FormatName : String; var spar, epar : TParagraph; var spos, epos, : Integer ) :Integer; - solves problem with integrated Bin64 decoder + DeleteField now has optional "Contents" parameter to delete a certain field with contents + the text writer now understand the option -softlinebreaks to create a \n at the end of every line. In fact all soft line breaks will be handled like the #10 code. + CodeLoadSaveEmbeddedText - load or save text in fields, bookmarks, hyperlinks + the regular save and load methods (LoadFromFile, SaveToFile) can now access text wrapped by paired objects. specify the fieldname in the format string, i.e. "f:name=RTF" to save or load the contents of the field "name". + There is an overloaded LoadFromString which expects a WideString as parameter (Delphi 6+) - The Setup named the Delphi 2009 files "Delphi 2005" due to a typo. (Delphi 2005 units are not included anymore) - fix probable range check error in WPRTEDEFS 13.10.2011 - WPTools 6.20 + completely new setup procedure. The PRO and Premium releases don't include object files which makes them much smaller. * compiled new WPTools 6 Reference (CHM file) * Delphi XE2: several small changes to improve theming support * Delphi XE2: several small changes to provide compatibilty to 64bit compiler (requires WPTools PRO) + new demo developed with Delphi XE2, showcases actions, splitscreen, simulated MDI and property dialogs (Demos\XE2\WPWord) + wpDeleteAtEOFRemovesSpaceAfter in EditOptionsEx2 - TextObjectsGetList did not work * WPRuler uses Delphi XE2 VCL Theme plus * added defaults to properties TWPRichText - change in RTF reader to let section inherit the default layout, not the current page layout - fix of problem with table borders when also PageMirror was used. * change to DeleteMarkedChar. It now has additional parameter DeleteEmptyParAndTables : Boolean * change in unit WPWordConv to handle RTF as DOC files if they do not start with "{\rtf" * updated border dialog TWPParagraphBorderDlgEx * updated border drawing code - now supports dotted lines with wider lines. * modified method DeleteColumn * modified WPT reading code to repair table width which were negative + improved image rendering code for transparent (PNG) images. They will be drawn transparently also when scaled and also in high resolution rendering mode. + new code to draw dotted lines which also supports wider lines + new function WPReplaceTokens (unit WPUtils.PAS). It is a ReplaceTokens function to be used on a TWPRTFDataCollection, not TWPRichText - WPPremium - fix problem when there were too columns * MergeText now restores before Merge Cursor position and selection (except for cell selection) * resizing a table column does not move the cursor to the nearby cell anymore * different frame line when resizing columns and rows + InsertColumn now also works if wpAllowSplitOfCombinedCellsOnly was used in EditOptionsEx + new event OnPaintTextFrameRect let you paint background and borders for text frames, i.e. the text body or, with WPTools Premium, custom frames. + WPPREMIUM: In OnMeasureTextPage it is possible to set columns for certain pages. Using PageInfo.colflags=1 it is possible to activate a line between the columns It is also possible to add custom frames using PageInfo.rtfpage.AddFrameRect. + new ViewOptionEx: wpHideParBordersBeforAndAfterPageBreaks + improved paint routine now avoids clipping of characters which were overlapping their bounding box, such as italic letters or "f". The improvement is especially visible when selecting text or using character background colors + WPPREMIUM: it is now possible to print a line between columns using wpShowColumnCenterLine in ViewOptionsEx + With WPTools Premium it is now possible to print a line between certain colums - use par.ASet(WPAT_COLFLAGS, WPCOLUM_LINE); + paragraph styles can now contain border definion for paragraphs + TWPTextObjList now has a IndexOfSource function and a Source[] string array to access the objects * revised code to draw double borders - always draws twou lines on screen even when zoomed * improved saving of numbering attributes with styles * style dialog can now apply number level even if style does not have numbering yet. * revised wpNoEditOutsideTable - was not checked for keyboard input * fix problem with - - - - - at end of line - fix problem with spell-as-you go after hyperlinks - fix problem with page numbers in sections when tables were spanning pages - fix problem with right aligned negative numbers in merge fields * automatic text attribute was not inherited to tables inserted in fields * images with mode "under text " can now be also clicked at positiones where there is no text. - WPLngCtr now defines DONT_OVERRIDE_COM, that fixes the IDE problem with DevExpress Toolbar controls 18.7.2011 - WPTools 6.16.2 * ObjectMode wpobjPositionAtCenter can be used to aligh character based images to the center line. (Other RTF reader will not understand this feature) * some changes to prepare 64bit compatibility (requires WPTools 6 PRO) 7.7.2011 - WPTools 6.16.1 * change to avoid flickering when doing auto scroll * message strings are now loaded from resource strings - modified DBCS support for RTF reader * outerborder action now works for sinle cells and paragraphs 15.6.2011 - WPTools 6.16 + Many improvements to new Border Dialog. It is now possible to apply borders and colors to odd and even rows only. * RTF reader defines symbol IGNORE_AUTOWIDTH_TBL. This disable the Word2010 auto width columns which are not read correctly. Reason: Word 2010 seem to always add an empty column to the end. + using WPRichText1.Caret.Blink:=true it is possible to activate the blinking caret (0.5sec interval). * updated actions to apply inner and outer borders to selected cells * update to ExSymbolDialog, Tab and Table dialog - improved WPTOOLS reader to read and apply outline properties to current outline setting (unless wpLoadDoesNotOverride is used) - wpsec_ResetPageNumber flag is now saved in WPTOOLS format * tripple click in margin selects paragraph + double click in margin selects current cell + tripple click in margin selects current row - change for PaintEngine and TWPRichTextLabel to not shrink tables which are small enough to fit the page This solves a problem with dissapearing text in WPRichTextLabel - fix selection problem when several images were linked to same paragraph - when moving images the Z order will not be reset - HTML Writer: A style with name "DIV" will be added to the style sheet to save the default font - HTML Writer: BaseFont tag will now be written with font size (requires -writebasefont option) - improved display of character background color for fields and other special code - Saving a style sheet did not work with Unicode Compiler 8.5.2011 - WPTools 6.15 * updated border painting * updated Inner/Outer Border Action * new object sizing routine lets the user make the size larger than the page - update in WPTools reader to overwrite outline styles when loading tzhe same group 9.3.2011 - WPTools 6.14.6 * change in format routine to fix problem when a nested table cell caused a page break. 14.12.2010 - WPTools 6.14 * fix for SetAsString code (Unicode Delphi) * several fixes and updates in editor * WPTools Premium: $define DONT_AUTOENTER_TEXTBOXES in WPINC.INC to switch of the behavior, that when editing a textbox the user can click on any other and edit that. 22.9.2010 - WPTools 6.13.3 * WPCtrMemo.PAS now defines TEXT_IS_UNICODE fro Delphi 2009 and later. Now the property Text and SelText reads and writes unicode strigs * change in RTF reader to read ANSI characters in the range 128..255 * Tables and rows can now be hidden. (TParagraph.Hidden) * The Lines property now supports unicode strings (Delphi 2009 and later) + HTML reader and writer now use the entitly ­ as soft hyphen 27.8.2010 - WPTools 6.13.2a + new ViewOptionEx wpUnderlineWebLinks. If active links like http://www.wptools.de will be drawn using the attributes for hyperlinks The HyperlinkCursor will be selected and the hyperlink event will be triggered * other fixes in RTF engine - Memo._OverrideCharset was not set to -1 27.7.2010 - WPTools 6.13.1 + WPRichText1.Memo.ColorGridLines can be used to change the color of the grid lines (ViewOptions) 23.7.2010 - WPTools 6.13 * several improvements of editor * improvement to RTF writer to when writing table cells * improved right aligned text * fixed problem with line heights of lines which are empty except for new line character 22.6.2010 - WPTools 6.12.1b - fix for problem when pressing Accent + Backspace - fix for WPReporter to assign default attribute to destination - fix for WPReporter to move images when converting templates 18.6.2010 - WPTools 6.12.1a * the new border dialog now reads the current border attributes from table cells, tables or selections - Premium: fix problem when loading columns which started on first line - fix in wpfUseKerning mode (FormatOptionsEx2) - it did not work as expected with some texts (We recommend to use wpfUseKerning - it produces better print quality on screen) * Pasting of HTML now works better - fix for right tabs when also borders were used 13.6.2010 - WPTools 6.12 + all new, powerful yet intuitive Border dialog. The border dialog can modify a range of selected cells, columns, rows, tables and also only modify certain properties while leaving the others unchanged. You need to activate the compiler symbol NEWBORDER to use it by default + new method: procedure SetBorderProperties(Inner, Outer: TWPTextStyle; ApplyMode : TWPParagraphBordApply; Mode : TWPUpdateBorderProperties = [wpSetBorderFlags, wpSetBorderType, wpSetBorderWidth, wpSetBorderColor, wpSetParColor, wpSetParShading]); This method is mainly used by the new border dialog. + new flags in ViewOptionsEx: wpShowCurrentCellAsSelected, // Displays current cell to be selected. Disables current selection wpShowCurrentRowAsSelected, // Displays current table row to be selected. Disables current selection wpShowCurrentTableAsSelected // Displays current table to be selected. Disables current selection + new property YOffsetNormal to defined an upper border for normal and wordwrap view. * feature Header.MarginMirror changed to work like MS Word new flag: wpMarginMirrorBookPrint in FormatOptionsEx2 to enable the previous logic * workaround for MouseWheel UP beeing triggered too often (10 mms check) + ClipboardOption wpcoDontPasteHTML to disable HTML pasting completely (avoid problems with firefox) + it is now possible to load base64 embedded JPEGs from HTML * it is now possible to change width of tables which exceed right margin - fix bug in HTML writer for lists in table cells - fix in RTF writer to write character colors also for text which is using a character style - fix: numbering was not always updated - fix: better use fonts in certain RTF files written by MS Word + Update to WPLanguageControl to make Localization easier to use. Only< this code is not required: procedure TForm1.WPLanguageControl1Loaded(Sender: TObject); begin WPLangInterface := TWPLocalizationInterface.Create(WPLanguageControl1); WPLocalizeAllStrings; end; 6.5.2010 - WPTools 6.11.2 * improvement to border rendering * improvement to XML unit WPIOXML1 (Premium) 5.5.2010 - WPTools 6.11.1 + ConvertTableToText now supports option to also handle soft line breaks - fix problems with underlines at end of line - fix problem when loading hyperlinks in RTF - fix problem when saving attributes to XML (WPTools Premium) - fix problem with text rendering - fix problem with tables which habe header row and pages with different header margin 19.4.2010 - WPTools 6.11 + EditOptionEx: wpRowMultiSelect + new event: OnInternPaintPar + new event: RTFDataCollection AfterApplyUndoObject 14.4.2010 - WPTools 6.10.6 * fields are now passed as unicode strings to PDF exporter * Delphi 2010/2009 import has been improved to load unicode values which are stroed in fields. 5.4.2010 - WPTools 6.10.5 + flag: wpDontExtendSelectionToRightMargin. Do not extend selection to end of line + wpInvertActiveRow in ViewOptionsEx - some fixes for Delphi 2009 and Delphi 2010 - fix for section support - WPTools premium: Fix for images in text boxes - workwaround to load RTF which use emfblib for pngblib 28.2.2010 - WPTools 6.10 - improve word left/right movement to skip hidden text - improve http load of images - improve support for numbering - improve saving of character style attributes 11.2.2010 - WPTools 6.09.1 + LoadFromString now has a "WithClear" parameter - fix in RTF reader to better load files which do not define codepage - fix in paint routine to solve a rare lockup - fix in format routine to improve section support 6.2.2010 - WPTools 6.09.1 - fix of problem in save routine when footnotes were used (WPTools premium) - fix in HTML writer - Image optiions now have a Rotation property which allows 90, 180 and 270 setting. - HTML reader and writer now support different colors for left,right,top, bottom lines 1.2.2010 - WPTools 6.09 - HTML writer will write 8 hard spaces for TAB stops at the start of a paragraph - HTML writer will write page information only if -PageInfo was used in format string - fix problem with left aligned text and image wrap around (wrong alignment) - fix problem with sometimes duplicated images in PDF export - fix problem with black rectangle in first line under Windows 7, 64 bit - Improvment to RTF reader to ignore section properties which are not followed by \sect 14.12.2009 - WPTools 6.08 - graphics are resized to fill text area - fixed problems in numbering - fixed problem with one word paragraphs in justified paragraphs - other improvements in editor 27.10.2009 - WPTools 6.07 - fixed one leaking TList per TWPRichText * improved layout of most important dialogs * improved extended insert symbol dialog - fix in RTF reader to load sections and header+footer written by Word 2003 - don't add unwanted cell padding when loading table cells - fix in WPTools reader to read custom number styles - if paragraph styles use number styles the indent defined in the style has priority over numberstyle - LabelDef now also works for one row and one column - better handling of mousewheel event - fix for tabs in tables + GIF animation (requires GifImage) library (not threaded) to use it You need to set ViewOption wpUseOwnDoubleBuffer and call the method RefreshAniImages using a timer object. - fix problem when sections were used with LabeDef.Active = true * change in HTML writer to close tags before paragraph end 4.10.2009 - WPTools 6.06 - fix problem with Delphi 2010 support (language control) - fix problem with PDF export to reduce PDF size - improve support for IME * improve AsWebPage format mode. Now WordWrap propery is supported. - fix searching text upwards with "Whole Word" selected + RTF writer now can use format option "-writehighlight" to writh \highlight instead of \cb 14.9.2009 - WPTools 6.05.9 + added Delphi 2010 Support - problem when pasting from "The Bat" - fix problem in Delphi 2009 (Assign Method) 3.8.2009 - WPTools 6.05.8' * when using "Delete All" in the tabstop dialog, all tabs will be cleared added to manual: Tabstop Category - fixed problem when deleting text in a paragraph. The alignment was cleared unexpectedly. - fix problem with installer, WPMangeHeaderFooter.DFM was not included - fix for IPara in mail merge field objects - improved handling of hoover effect for hyperlinks - improved text rendering for wPDF output (CID Mode) - add correct WPManHeadFoot.dfm 23.7.2009 - WPTools 6.05.7 + WRITE_PROP_WPTOOLSOBJ $define in WPIOWriteRTF. Avoid problems when saving RTF and opting in Word In case of special objects, such as SPAN codes, \*\wpfldinst is beeing written what is ignored by WOrd + Dialog HeaderFooter can optionally create and manage header&footer for the current section + new KeepN Handling. This is by default activated in FormatOptionsEx2 + new wpfHideParagraphWithHiddenText in FormatOptionsEx2. Now paragraphs will be hidden if empty or only contain hidden text. + new format option -zap1 will remove the every first byte to convert a two byte stream into singly byte -zap2 will remove every second byte. Usie this option when loading data from unicode data sets - bugfix for table loading in RTF 15.7.2009 - WPTools 6.05 + TParagraph.Trim method to remove white spaces at start and end + Vertical Scrolling by pressing the middle mouse button now works. + improved auto thumbnail mode * enhancement to HTML reader / writer to handle embedded SPAN objects + new method: ApplySPANStyles(and_remove : Boolean=false; ignore_charattr : Boolean = false); can be used to apply SPAN styles to the text which it embeds + The function InputSpanObjects( Attributes : TWPAbstractCharAttrInterface ) : TWPTextObj; can be used to wrap the selected text into SPAN objects + method LoadCSSheet can be used to load paragraph styles in CSS format from a string - fix problem with Wordrwap and centered text + new even OnTextObjectMovePosition (move event) - OnTextObjectMove is still used for resize (unchanged) 28.6.2009 - WPTools 6.04 + WPTools Premium: Column Balancing * many improvements in RTF reader. Word documents are now understood better * Improvement in check for protected text (ppMergedText) + new ViewOptionsEx property - auto hyperlinks were not working + TWPComboBox has an event OnUpdateItems which will be triggered after the items had been automatically assigned. 24.6.2009 - WPTools 6.03.6 * thinner page borders in thumbnail mode. ViewOptionsEx: wpAutoThumbnailMode will show pagenumbers only when in thubmbnail mode (= wpShowPageNRinGap in ViewOptions) + property ColorDesktop and DeskGradientHorizontal to render the background with a gradient fill * fix for protected text handling (CR after a field) * fix for text alignment near a movable image - EditOption AutoDetectHyperlinks was not working anymore * WPReporter: SuperMerge.Stack.PageBreakAFterGroup := true was ot working when footers were used 1.6.2009 - WPTools 6.03.5 - fix problem with display of character attributes when attributes were inherited from paragraph styles - fix problems with selection deletion in single column, single row tables - improvement of RTF writer when writing sections 11.5.2009 - WPTools 6.03.3 - improved report band dialog, new ShowOptions property - fix in RTF reader to load header/footer - change in HTML writer to save SPAN instead of FONT tag - several fixes in editor * WPTools Premium: better column support. Fixed column height now splits correctly on 2 pages. 28.4.2009 - WPTools 6.03.2 - fix problem with justified text in PDF 21.4.2009 - WPTools 6.03.1 - fix problem with images when used in Delphi 2009 - better support for header/footer in RTF files created by word. (Ignore bogus header/footer) - soome stability fixes 25.3.2009 - WPTools 6.03 + improved text rendering - optimation for character distances on screen to provide better display + improvement on ShowMergeFieldNames to improve cursor movement and drag and drop + automatic disable dragging of fields inside of fields + improved merge field selection. TextObject.SelectedObject now returns the mergefield if it was completely selected + change in HTML saving code to save src in after width adn height (for outlook) * various bugfixes 17.1.2009 - WPTools 6.02 - WPPREMIUM: Text after Columns initialized with WPAT_COLUMNS_Y is now allowed + TWPToolBar FontName drop down now lists fonts used by document first - fix for tables which use a fixed row height and are splitted on different pages + improvements necessary for Delphi 2009 - the Locaization demo now works + EditOptionEx wpDontPreserveObjectsAgainstDeletion - fix problem in ImageObject LoadFromStream when GraphicEx is used - fix problem with Delphi 2009 when loading WPReporter templates - fix problem with HTML reader with paragraph style of first paragraph + GetFontUnicodeRanges dynamically linked to avoid Win98 problem 26.10.2008 - WPTools 6.01 * updated HTTP Demo, now with "Source View" + DELETE/BACKSPC at start of line removes right/center alignment + loads background images for paragraphs, tables and styles * improvement to text protection (empty lines) - improvements to HTML and CSS reader - improved HTML format routine - improved MIME loading - now supports binary data despite Synapse does not) + MIME reader capturesHTML body for SourceVIew * DataProvider now uses MergeText('',true) instead of MergeText + boolean wphttp_Disable to disconnect HTTP temporarily * several changes to improve compatibility with Delphi 2009 17.10.2008 - WPTools 6.00.1 - several changes to fix problems which occured with use of Delphi 2009 * update to WPIO_MIME to also load binary encoded GIFS and JPEGS from EML files 16.10.2008 - WPTools 6.00 * new installer + VCL demo * fixed problem with TWPComboBox and default attributes * updated "Tasks\Mini" demo project - fix in default actions 3.10.2008 - WPTools 6 Beta c - remove not required unit reference form WPRTEReport * update to manual 2.10.2008 - WPTools 6 Beta b - fix in Installation script 30.9.2008 - WPTools 6 Beta a 1) Application-Server-Mode WPTools 6 introduces a feature which is called "Application-Server-Mode". This is only available when you have the PREMIUM version. This mode is activated when true is assigned to the global boolean variable WPAppServerMode. When this mode is activated the editor does not use the double buffered output anymore. While this can cause some flickering the network traffic is reduced when the application runs on an application server, such as Citrix. Please note, effective with WPTools 6, software which was written for application servers (such as clinic software) may only be distributed when a TEAM or SITE license of WPTools was acquired.END 2) Integrated Label Printing When activated the integrated label printing shows multiple labels on one virtual sheet of paper. The cursor can move from label to label freely. The user can so edit the labels, add new or delete unwanted labels before the complete sheet is printed. This is a very unique and versatile feature. 3) Additions to the PDF export with wPDF V3 Create embedded data objects Create edit / memo fields Create a check box field Also the creation of PDF tags was enhanced. So now hints to paragraph styles will be also exported. 4) Additional Control over Clipboard Actions a) properties to select the format b) added security 5) Added Section API + WPRichText1.ActiveSection + WPRichText1.InputSection + TWPPagePropDlg has new method ExecuteEx. Use it to change current page size or Section WPPagePropDlg1.ExecuteEx(WPRichText1.ActiveSection); 6) Load over HTTP connections (requires Synapse) 7) Load and save MIME encoded HTML with embedded images (requires Synapse) 8) Integrated XML syntax highlighting (non destructive) 9) WPReporter: Token to Template conversion with syntax highlighting 10) Additions to paragraph API 11) Special mode to format HTML documents
Table of Contents Header Files The #define Guard Header File Dependencies Inline Functions The -inl.h Files Function Parameter Ordering Names and Order of Includes Scoping Namespaces Nested Classes Nonmember, Static Member, and Global Functions Local Variables Static and Global Variables Classes Doing Work in Constructors Default Constructors Explicit Constructors Copy Constructors Structs vs. Classes Inheritance Multiple Inheritance Interfaces Operator Overloading Access Control Declaration Order Write Short Functions Google-Specific Magic Smart Pointers cpplint Other C++ Features Reference Arguments Function Overloading Default Arguments Variable-Length Arrays and alloca() Friends Exceptions Run-Time Type Information (RTTI) Casting Streams Preincrement and Predecrement Use of const Integer Types 64-bit Portability Preprocessor Macros 0 and NULL sizeof Boost C++0x Naming General Naming Rules File Names Type Names Variable Names Constant Names Function Names Namespace Names Enumerator Names Macro Names Exceptions to Naming Rules Comments Comment Style File Comments Class Comments Function Comments Variable Comments Implementation Comments Punctuation, Spelling and Grammar TODO Comments Deprecation Comments Formatting Line Length Non-ASCII Characters Spaces vs. Tabs Function Declarations and Definitions Function Calls Conditionals Loops and Switch Statements Pointer and Reference Expressions Boolean Expressions Return Values Variable and Array Initialization Preprocessor Directives Class Format Constructor Initializer Lists Namespace Formatting Horizontal Whitespace Vertical Whitespace Exceptions to the Rules Existing Non-conformant Code Windows Code Important Note Displaying Hidden Details in this Guide link ▶This style guide contains many details that are initially hidden from view. They are marked by the triangle icon, which you see here on your left. Click it now. You should see "Hooray" appear below. Hooray! Now you know you can expand points to get more details. Alternatively, there's an "expand all" at the top of this document. Background C++ is the main development language used by many of Google's open-source projects. As every C++ programmer knows, the language has many powerful features, but this power brings with it complexity, which in turn can make code more bug-prone and harder to read and maintain. The goal of this guide is to manage this complexity by describing in detail the dos and don'ts of writing C++ code. These rules exist to keep the code base manageable while still allowing coders to use C++ language features productively. Style, also known as readability, is what we call the conventions that govern our C++ code. The term Style is a bit of a misnomer, since these conventions cover far more than just source file formatting. One way in which we keep the code base manageable is by enforcing consistency. It is very important that any programmer be able to look at another's code and quickly understand it. Maintaining a uniform style and following conventions means that we can more easily use "pattern-matching" to infer what various symbols are and what invariants are true about them. Creating common, required idioms and patterns makes code much easier to understand. In some cases there might be good arguments for changing certain style rules, but we nonetheless keep things as they are in order to preserve consistency. Another issue this guide addresses is that of C++ feature bloat. C++ is a huge language with many advanced features. In some cases we constrain, or even ban, use of certain features. We do this to keep code simple and to avoid the various common errors and problems that these features can cause. This guide lists these features and explains why their use is restricted. Open-source projects developed by Google conform to the requirements in this guide. Note that this guide is not a C++ tutorial: we assume that the reader is familiar with the language. Header Files In general, every .cc file should have an associated .h file. There are some common exceptions, such as unittests and small .cc files containing just a main() function. Correct use of header files can make a huge difference to the readability, size and performance of your code. The following rules will guide you through the various pitfalls of using header files. The #define Guard link ▶All header files should have #define guards to prevent multiple inclusion. The format of the symbol name should be ___H_. To guarantee uniqueness, they should be based on the full path in a project's source tree. For example, the file foo/src/bar/baz.h in project foo should have the following guard: #ifndef FOO_BAR_BAZ_H_ #define FOO_BAR_BAZ_H_ ... #endif // FOO_BAR_BAZ_H_ Header File Dependencies link ▶Don't use an #include when a forward declaration would suffice. When you include a header file you introduce a dependency that will cause your code to be recompiled whenever the header file changes. If your header file includes other header files, any change to those files will cause any code that includes your header to be recompiled. Therefore, we prefer to minimize includes, particularly includes of header files in other header files. You can significantly minimize the number of header files you need to include in your own header files by using forward declarations. For example, if your header file uses the File class in ways that do not require access to the declaration of the File class, your header file can just forward declare class File; instead of having to #include "file/base/file.h". How can we use a class Foo in a header file without access to its definition? We can declare data members of type Foo* or Foo&. We can declare (but not define) functions with arguments, and/or return values, of type Foo. (One exception is if an argument Foo or const Foo& has a non-explicit, one-argument constructor, in which case we need the full definition to support automatic type conversion.) We can declare static data members of type Foo. This is because static data members are defined outside the class definition. On the other hand, you must include the header file for Foo if your class subclasses Foo or has a data member of type Foo. Sometimes it makes sense to have pointer (or better, scoped_ptr) members instead of object members. However, this complicates code readability and imposes a performance penalty, so avoid doing this transformation if the only purpose is to minimize includes in header files. Of course, .cc files typically do require the definitions of the classes they use, and usually have to include several header files. Note: If you use a symbol Foo in your source file, you should bring in a definition for Foo yourself, either via an #include or via a forward declaration. Do not depend on the symbol being brought in transitively via headers not directly included. One exception is if Foo is used in myfile.cc, it's ok to #include (or forward-declare) Foo in myfile.h, instead of myfile.cc. Inline Functions link ▶Define functions inline only when they are small, say, 10 lines or less. Definition: You can declare functions in a way that allows the compiler to expand them inline rather than calling them through the usual function call mechanism. Pros: Inlining a function can generate more efficient object code, as long as the inlined function is small. Feel free to inline accessors and mutators, and other short, performance-critical functions. Cons: Overuse of inlining can actually make programs slower. Depending on a function's size, inlining it can cause the code size to increase or decrease. Inlining a very small accessor function will usually decrease code size while inlining a very large function can dramatically increase code size. On modern processors smaller code usually runs faster due to better use of the instruction cache. Decision: A decent rule of thumb is to not inline a function if it is more than 10 lines long. Beware of destructors, which are often longer than they appear because of implicit member- and base-destructor calls! Another useful rule of thumb: it's typically not cost effective to inline functions with loops or switch statements (unless, in the common case, the loop or switch statement is never executed). It is important to know that functions are not always inlined even if they are declared as such; for example, virtual and recursive functions are not normally inlined. Usually recursive functions should not be inline. The main reason for making a virtual function inline is to place its definition in the class, either for convenience or to document its behavior, e.g., for accessors and mutators. The -inl.h Files link ▶You may use file names with a -inl.h suffix to define complex inline functions when needed. The definition of an inline function needs to be in a header file, so that the compiler has the definition available for inlining at the call sites. However, implementation code properly belongs in .cc files, and we do not like to have much actual code in .h files unless there is a readability or performance advantage. If an inline function definition is short, with very little, if any, logic in it, you should put the code in your .h file. For example, accessors and mutators should certainly be inside a class definition. More complex inline functions may also be put in a .h file for the convenience of the implementer and callers, though if this makes the .h file too unwieldy you can instead put that code in a separate -inl.h file. This separates the implementation from the class definition, while still allowing the implementation to be included where necessary. Another use of -inl.h files is for definitions of function templates. This can be used to keep your template definitions easy to read. Do not forget that a -inl.h file requires a #define guard just like any other header file. Function Parameter Ordering link ▶When defining a function, parameter order is: inputs, then outputs. Parameters to C/C++ functions are either input to the function, output from the function, or both. Input parameters are usually values or const references, while output and input/output parameters will be non-const pointers. When ordering function parameters, put all input-only parameters before any output parameters. In particular, do not add new parameters to the end of the function just because they are new; place new input-only parameters before the output parameters. This is not a hard-and-fast rule. Parameters that are both input and output (often classes/structs) muddy the waters, and, as always, consistency with related functions may require you to bend the rule. Names and Order of Includes link ▶Use standard order for readability and to avoid hidden dependencies: C library, C++ library, other libraries' .h, your project's .h. All of a project's header files should be listed as descentants of the project's source directory without use of UNIX directory shortcuts . (the current directory) or .. (the parent directory). For example, google-awesome-project/src/base/logging.h should be included as #include "base/logging.h" In dir/foo.cc, whose main purpose is to implement or test the stuff in dir2/foo2.h, order your includes as follows: dir2/foo2.h (preferred location — see details below). C system files. C++ system files. Other libraries' .h files. Your project's .h files. The preferred ordering reduces hidden dependencies. We want every header file to be compilable on its own. The easiest way to achieve this is to make sure that every one of them is the first .h file #included in some .cc. dir/foo.cc and dir2/foo2.h are often in the same directory (e.g. base/basictypes_test.cc and base/basictypes.h), but can be in different directories too. Within each section it is nice to order the includes alphabetically. For example, the includes in google-awesome-project/src/foo/internal/fooserver.cc might look like this: #include "foo/public/fooserver.h" // Preferred location. #include #include #include #include #include "base/basictypes.h" #include "base/commandlineflags.h" #include "foo/public/bar.h" Scoping Namespaces link ▶Unnamed namespaces in .cc files are encouraged. With named namespaces, choose the name based on the project, and possibly its path. Do not use a using-directive. Definition: Namespaces subdivide the global scope into distinct, named scopes, and so are useful for preventing name collisions in the global scope. Pros: Namespaces provide a (hierarchical) axis of naming, in addition to the (also hierarchical) name axis provided by classes. For example, if two different projects have a class Foo in the global scope, these symbols may collide at compile time or at runtime. If each project places their code in a namespace, project1::Foo and project2::Foo are now distinct symbols that do not collide. Cons: Namespaces can be confusing, because they provide an additional (hierarchical) axis of naming, in addition to the (also hierarchical) name axis provided by classes. Use of unnamed spaces in header files can easily cause violations of the C++ One Definition Rule (ODR). Decision: Use namespaces according to the policy described below. Unnamed Namespaces Unnamed namespaces are allowed and even encouraged in .cc files, to avoid runtime naming conflicts: namespace { // This is in a .cc file. // The content of a namespace is not indented enum { kUnused, kEOF, kError }; // Commonly used tokens. bool AtEof() { return pos_ == kEOF; } // Uses our namespace's EOF. } // namespace However, file-scope declarations that are associated with a particular class may be declared in that class as types, static data members or static member functions rather than as members of an unnamed namespace. Terminate the unnamed namespace as shown, with a comment // namespace. Do not use unnamed namespaces in .h files. Named Namespaces Named namespaces should be used as follows: Namespaces wrap the entire source file after includes, gflags definitions/declarations, and forward declarations of classes from other namespaces: // In the .h file namespace mynamespace { // All declarations are within the namespace scope. // Notice the lack of indentation. class MyClass { public: ... void Foo(); }; } // namespace mynamespace // In the .cc file namespace mynamespace { // Definition of functions is within scope of the namespace. void MyClass::Foo() { ... } } // namespace mynamespace The typical .cc file might have more complex detail, including the need to reference classes in other namespaces. #include "a.h" DEFINE_bool(someflag, false, "dummy flag"); class C; // Forward declaration of class C in the global namespace. namespace a { class A; } // Forward declaration of a::A. namespace b { ...code for b... // Code goes against the left margin. } // namespace b Do not declare anything in namespace std, not even forward declarations of standard library classes. Declaring entities in namespace std is undefined behavior, i.e., not portable. To declare entities from the standard library, include the appropriate header file. You may not use a using-directive to make all names from a namespace available. // Forbidden -- This pollutes the namespace. using namespace foo; You may use a using-declaration anywhere in a .cc file, and in functions, methods or classes in .h files. // OK in .cc files. // Must be in a function, method or class in .h files. using ::foo::bar; Namespace aliases are allowed anywhere in a .cc file, anywhere inside the named namespace that wraps an entire .h file, and in functions and methods. // Shorten access to some commonly used names in .cc files. namespace fbz = ::foo::bar::baz; // Shorten access to some commonly used names (in a .h file). namespace librarian { // The following alias is available to all files including // this header (in namespace librarian): // alias names should therefore be chosen consistently // within a project. namespace pd_s = ::pipeline_diagnostics::sidetable; inline void my_inline_function() { // namespace alias local to a function (or method). namespace fbz = ::foo::bar::baz; ... } } // namespace librarian Note that an alias in a .h file is visible to everyone #including that file, so public headers (those available outside a project) and headers transitively #included by them, should avoid defining aliases, as part of the general goal of keeping public APIs as small as possible. Nested Classes link ▶Although you may use public nested classes when they are part of an interface, consider a namespace to keep declarations out of the global scope. Definition: A class can define another class within it; this is also called a member class. class Foo { private: // Bar is a member class, nested within Foo. class Bar { ... }; }; Pros: This is useful when the nested (or member) class is only used by the enclosing class; making it a member puts it in the enclosing class scope rather than polluting the outer scope with the class name. Nested classes can be forward declared within the enclosing class and then defined in the .cc file to avoid including the nested class definition in the enclosing class declaration, since the nested class definition is usually only relevant to the implementation. Cons: Nested classes can be forward-declared only within the definition of the enclosing class. Thus, any header file manipulating a Foo::Bar* pointer will have to include the full class declaration for Foo. Decision: Do not make nested classes public unless they are actually part of the interface, e.g., a class that holds a set of options for some method. Nonmember, Static Member, and Global Functions link ▶Prefer nonmember functions within a namespace or static member functions to global functions; use completely global functions rarely. Pros: Nonmember and static member functions can be useful in some situations. Putting nonmember functions in a namespace avoids polluting the global namespace. Cons: Nonmember and static member functions may make more sense as members of a new class, especially if they access external resources or have significant dependencies. Decision: Sometimes it is useful, or even necessary, to define a function not bound to a class instance. Such a function can be either a static member or a nonmember function. Nonmember functions should not depend on external variables, and should nearly always exist in a namespace. Rather than creating classes only to group static member functions which do not share static data, use namespaces instead. Functions defined in the same compilation unit as production classes may introduce unnecessary coupling and link-time dependencies when directly called from other compilation units; static member functions are particularly susceptible to this. Consider extracting a new class, or placing the functions in a namespace possibly in a separate library. If you must define a nonmember function and it is only needed in its .cc file, use an unnamed namespace or static linkage (eg static int Foo() {...}) to limit its scope. Local Variables link ▶Place a function's variables in the narrowest scope possible, and initialize variables in the declaration. C++ allows you to declare variables anywhere in a function. We encourage you to declare them in as local a scope as possible, and as close to the first use as possible. This makes it easier for the reader to find the declaration and see what type the variable is and what it was initialized to. In particular, initialization should be used instead of declaration and assignment, e.g. int i; i = f(); // Bad -- initialization separate from declaration. int j = g(); // Good -- declaration has initialization. Note that gcc implements for (int i = 0; i < 10; ++i) correctly (the scope of i is only the scope of the for loop), so you can then reuse i in another for loop in the same scope. It also correctly scopes declarations in if and while statements, e.g. while (const char* p = strchr(str, '/')) str = p + 1; There is one caveat: if the variable is an object, its constructor is invoked every time it enters scope and is created, and its destructor is invoked every time it goes out of scope. // Inefficient implementation: for (int i = 0; i < 1000000; ++i) { Foo f; // My ctor and dtor get called 1000000 times each. f.DoSomething(i); } It may be more efficient to declare such a variable used in a loop outside that loop: Foo f; // My ctor and dtor get called once each. for (int i = 0; i < 1000000; ++i) { f.DoSomething(i); } Static and Global Variables link ▶Static or global variables of class type are forbidden: they cause hard-to-find bugs due to indeterminate order of construction and destruction. Objects with static storage duration, including global variables, static variables, static class member variables, and function static variables, must be Plain Old Data (POD): only ints, chars, floats, or pointers, or arrays/structs of POD. The order in which class constructors and initializers for static variables are called is only partially specified in C++ and can even change from build to build, which can cause bugs that are difficult to find. Therefore in addition to banning globals of class type, we do not allow static POD variables to be initialized with the result of a function, unless that function (such as getenv(), or getpid()) does not itself depend on any other globals. Likewise, the order in which destructors are called is defined to be the reverse of the order in which the constructors were called. Since constructor order is indeterminate, so is destructor order. For example, at program-end time a static variable might have been destroyed, but code still running -- perhaps in another thread -- tries to access it and fails. Or the destructor for a static 'string' variable might be run prior to the destructor for another variable that contains a reference to that string. As a result we only allow static variables to contain POD data. This rule completely disallows vector (use C arrays instead), or string (use const char []). If you need a static or global variable of a class type, consider initializing a pointer (which will never be freed), from either your main() function or from pthread_once(). Note that this must be a raw pointer, not a "smart" pointer, since the smart pointer's destructor will have the order-of-destructor issue that we are trying to avoid. Classes Classes are the fundamental unit of code in C++. Naturally, we use them extensively. This section lists the main dos and don'ts you should follow when writing a class. Doing Work in Constructors link ▶In general, constructors should merely set member variables to their initial values. Any complex initialization should go in an explicit Init() method. Definition: It is possible to perform initialization in the body of the constructor. Pros: Convenience in typing. No need to worry about whether the class has been initialized or not. Cons: The problems with doing work in constructors are: There is no easy way for constructors to signal errors, short of using exceptions (which are forbidden). If the work fails, we now have an object whose initialization code failed, so it may be an indeterminate state. If the work calls virtual functions, these calls will not get dispatched to the subclass implementations. Future modification to your class can quietly introduce this problem even if your class is not currently subclassed, causing much confusion. If someone creates a global variable of this type (which is against the rules, but still), the constructor code will be called before main(), possibly breaking some implicit assumptions in the constructor code. For instance, gflags will not yet have been initialized. Decision: If your object requires non-trivial initialization, consider having an explicit Init() method. In particular, constructors should not call virtual functions, attempt to raise errors, access potentially uninitialized global variables, etc. Default Constructors link ▶You must define a default constructor if your class defines member variables and has no other constructors. Otherwise the compiler will do it for you, badly. Definition: The default constructor is called when we new a class object with no arguments. It is always called when calling new[] (for arrays). Pros: Initializing structures by default, to hold "impossible" values, makes debugging much easier. Cons: Extra work for you, the code writer. Decision: If your class defines member variables and has no other constructors you must define a default constructor (one that takes no arguments). It should preferably initialize the object in such a way that its internal state is consistent and valid. The reason for this is that if you have no other constructors and do not define a default constructor, the compiler will generate one for you. This compiler generated constructor may not initialize your object sensibly. If your class inherits from an existing class but you add no new member variables, you are not required to have a default constructor. Explicit Constructors link ▶Use the C++ keyword explicit for constructors with one argument. Definition: Normally, if a constructor takes one argument, it can be used as a conversion. For instance, if you define Foo::Foo(string name) and then pass a string to a function that expects a Foo, the constructor will be called to convert the string into a Foo and will pass the Foo to your function for you. This can be convenient but is also a source of trouble when things get converted and new objects created without you meaning them to. Declaring a constructor explicit prevents it from being invoked implicitly as a conversion. Pros: Avoids undesirable conversions. Cons: None. Decision: We require all single argument constructors to be explicit. Always put explicit in front of one-argument constructors in the class definition: explicit Foo(string name); The exception is copy constructors, which, in the rare cases when we allow them, should probably not be explicit. Classes that are intended to be transparent wrappers around other classes are also exceptions. Such exceptions should be clearly marked with comments. Copy Constructors link ▶Provide a copy constructor and assignment operator only when necessary. Otherwise, disable them with DISALLOW_COPY_AND_ASSIGN. Definition: The copy constructor and assignment operator are used to create copies of objects. The copy constructor is implicitly invoked by the compiler in some situations, e.g. passing objects by value. Pros: Copy constructors make it easy to copy objects. STL containers require that all contents be copyable and assignable. Copy constructors can be more efficient than CopyFrom()-style workarounds because they combine construction with copying, the compiler can elide them in some contexts, and they make it easier to avoid heap allocation. Cons: Implicit copying of objects in C++ is a rich source of bugs and of performance problems. It also reduces readability, as it becomes hard to track which objects are being passed around by value as opposed to by reference, and therefore where changes to an object are reflected. Decision: Few classes need to be copyable. Most should have neither a copy constructor nor an assignment operator. In many situations, a pointer or reference will work just as well as a copied value, with better performance. For example, you can pass function parameters by reference or pointer instead of by value, and you can store pointers rather than objects in an STL container. If your class needs to be copyable, prefer providing a copy method, such as CopyFrom() or Clone(), rather than a copy constructor, because such methods cannot be invoked implicitly. If a copy method is insufficient in your situation (e.g. for performance reasons, or because your class needs to be stored by value in an STL container), provide both a copy constructor and assignment operator. If your class does not need a copy constructor or assignment operator, you must explicitly disable them. To do so, add dummy declarations for the copy constructor and assignment operator in the private: section of your class, but do not provide any corresponding definition (so that any attempt to use them results in a link error). For convenience, a DISALLOW_COPY_AND_ASSIGN macro can be used: // A macro to disallow the copy constructor and operator= functions // This should be used in the private: declarations for a class #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ TypeName(const TypeName&); \ void operator=(const TypeName&) Then, in class Foo: class Foo { public: Foo(int f); ~Foo(); private: DISALLOW_COPY_AND_ASSIGN(Foo); }; Structs vs. Classes link ▶Use a struct only for passive objects that carry data; everything else is a class. The struct and class keywords behave almost identically in C++. We add our own semantic meanings to each keyword, so you should use the appropriate keyword for the data-type you're defining. structs should be used for passive objects that carry data, and may have associated constants, but lack any functionality other than access/setting the data members. The accessing/setting of fields is done by directly accessing the fields rather than through method invocations. Methods should not provide behavior but should only be used to set up the data members, e.g., constructor, destructor, Initialize(), Reset(), Validate(). If more functionality is required, a class is more appropriate. If in doubt, make it a class. For consistency with STL, you can use struct instead of class for functors and traits. Note that member variables in structs and classes have different naming rules. Inheritance link ▶Composition is often more appropriate than inheritance. When using inheritance, make it public. Definition: When a sub-class inherits from a base class, it includes the definitions of all the data and operations that the parent base class defines. In practice, inheritance is used in two major ways in C++: implementation inheritance, in which actual code is inherited by the child, and interface inheritance, in which only method names are inherited. Pros: Implementation inheritance reduces code size by re-using the base class code as it specializes an existing type. Because inheritance is a compile-time declaration, you and the compiler can understand the operation and detect errors. Interface inheritance can be used to programmatically enforce that a class expose a particular API. Again, the compiler can detect errors, in this case, when a class does not define a necessary method of the API. Cons: For implementation inheritance, because the code implementing a sub-class is spread between the base and the sub-class, it can be more difficult to understand an implementation. The sub-class cannot override functions that are not virtual, so the sub-class cannot change implementation. The base class may also define some data members, so that specifies physical layout of the base class. Decision: All inheritance should be public. If you want to do private inheritance, you should be including an instance of the base class as a member instead. Do not overuse implementation inheritance. Composition is often more appropriate. Try to restrict use of inheritance to the "is-a" case: Bar subclasses Foo if it can reasonably be said that Bar "is a kind of" Foo. Make your destructor virtual if necessary. If your class has virtual methods, its destructor should be virtual. Limit the use of protected to those member functions that might need to be accessed from subclasses. Note that data members should be private. When redefining an inherited virtual function, explicitly declare it virtual in the declaration of the derived class. Rationale: If virtual is omitted, the reader has to check all ancestors of the class in question to determine if the function is virtual or not. Multiple Inheritance link ▶Only very rarely is multiple implementation inheritance actually useful. We allow multiple inheritance only when at most one of the base classes has an implementation; all other base classes must be pure interface classes tagged with the Interface suffix. Definition: Multiple inheritance allows a sub-class to have more than one base class. We distinguish between base classes that are pure interfaces and those that have an implementation. Pros: Multiple implementation inheritance may let you re-use even more code than single inheritance (see Inheritance). Cons: Only very rarely is multiple implementation inheritance actually useful. When multiple implementation inheritance seems like the solution, you can usually find a different, more explicit, and cleaner solution. Decision: Multiple inheritance is allowed only when all superclasses, with the possible exception of the first one, are pure interfaces. In order to ensure that they remain pure interfaces, they must end with the Interface suffix. Note: There is an exception to this rule on Windows. Interfaces link ▶Classes that satisfy certain conditions are allowed, but not required, to end with an Interface suffix. Definition: A class is a pure interface if it meets the following requirements: It has only public pure virtual ("= 0") methods and static methods (but see below for destructor). It may not have non-static data members. It need not have any constructors defined. If a constructor is provided, it must take no arguments and it must be protected. If it is a subclass, it may only be derived from classes that satisfy these conditions and are tagged with the Interface suffix. An interface class can never be directly instantiated because of the pure virtual method(s) it declares. To make sure all implementations of the interface can be destroyed correctly, they must also declare a virtual destructor (in an exception to the first rule, this should not be pure). See Stroustrup, The C++ Programming Language, 3rd edition, section 12.4 for details. Pros: Tagging a class with the Interface suffix lets others know that they must not add implemented methods or non static data members. This is particularly important in the case of multiple inheritance. Additionally, the interface concept is already well-understood by Java programmers. Cons: The Interface suffix lengthens the class name, which can make it harder to read and understand. Also, the interface property may be considered an implementation detail that shouldn't be exposed to clients. Decision: A class may end with Interface only if it meets the above requirements. We do not require the converse, however: classes that meet the above requirements are not required to end with Interface. Operator Overloading link ▶Do not overload operators except in rare, special circumstances. Definition: A class can define that operators such as + and / operate on the class as if it were a built-in type. Pros: Can make code appear more intuitive because a class will behave in the same way as built-in types (such as int). Overloaded operators are more playful names for functions that are less-colorfully named, such as Equals() or Add(). For some template functions to work correctly, you may need to define operators. Cons: While operator overloading can make code more intuitive, it has several drawbacks: It can fool our intuition into thinking that expensive operations are cheap, built-in operations. It is much harder to find the call sites for overloaded operators. Searching for Equals() is much easier than searching for relevant invocations of ==. Some operators work on pointers too, making it easy to introduce bugs. Foo + 4 may do one thing, while &Foo + 4 does something totally different. The compiler does not complain for either of these, making this very hard to debug. Overloading also has surprising ramifications. For instance, if a class overloads unary operator&, it cannot safely be forward-declared. Decision: In general, do not overload operators. The assignment operator (operator=), in particular, is insidious and should be avoided. You can define functions like Equals() and CopyFrom() if you need them. Likewise, avoid the dangerous unary operator& at all costs, if there's any possibility the class might be forward-declared. However, there may be rare cases where you need to overload an operator to interoperate with templates or "standard" C++ classes (such as operator<<(ostream&, const T&) for logging). These are acceptable if fully justified, but you should try to avoid these whenever possible. In particular, do not overload operator== or operator< just so that your class can be used as a key in an STL container; instead, you should create equality and comparison functor types when declaring the container. Some of the STL algorithms do require you to overload operator==, and you may do so in these cases, provided you document why. See also Copy Constructors and Function Overloading. Access Control link ▶Make data members private, and provide access to them through accessor functions as needed (for technical reasons, we allow data members of a test fixture class to be protected when using Google Test). Typically a variable would be called foo_ and the accessor function foo(). You may also want a mutator function set_foo(). Exception: static const data members (typically called kFoo) need not be private. The definitions of accessors are usually inlined in the header file. See also Inheritance and Function Names. Declaration Order link ▶Use the specified order of declarations within a class: public: before private:, methods before data members (variables), etc. Your class definition should start with its public: section, followed by its protected: section and then its private: section. If any of these sections are empty, omit them. Within each section, the declarations generally should be in the following order: Typedefs and Enums Constants (static const data members) Constructors Destructor Methods, including static methods Data Members (except static const data members) Friend declarations should always be in the private section, and the DISALLOW_COPY_AND_ASSIGN macro invocation should be at the end of the private: section. It should be the last thing in the class. See Copy Constructors. Method definitions in the corresponding .cc file should be the same as the declaration order, as much as possible. Do not put large method definitions inline in the class definition. Usually, only trivial or performance-critical, and very short, methods may be defined inline. See Inline Functions for more details. Write Short Functions link ▶Prefer small and focused functions. We recognize that long functions are sometimes appropriate, so no hard limit is placed on functions length. If a function exceeds about 40 lines, think about whether it can be broken up without harming the structure of the program. Even if your long function works perfectly now, someone modifying it in a few months may add new behavior. This could result in bugs that are hard to find. Keeping your functions short and simple makes it easier for other people to read and modify your code. You could find long and complicated functions when working with some code. Do not be intimidated by modifying existing code: if working with such a function proves to be difficult, you find that errors are hard to debug, or you want to use a piece of it in several different contexts, consider breaking up the function into smaller and more manageable pieces. Google-Specific Magic There are various tricks and utilities that we use to make C++ code more robust, and various ways we use C++ that may differ from what you see elsewhere. Smart Pointers link ▶If you actually need pointer semantics, scoped_ptr is great. You should only use std::tr1::shared_ptr under very specific conditions, such as when objects need to be held by STL containers. You should never use auto_ptr. "Smart" pointers are objects that act like pointers but have added semantics. When a scoped_ptr is destroyed, for instance, it deletes the object it's pointing to. shared_ptr is the same way, but implements reference-counting so only the last pointer to an object deletes it. Generally speaking, we prefer that we design code with clear object ownership. The clearest object ownership is obtained by using an object directly as a field or local variable, without using pointers at all. On the other extreme, by their very definition, reference counted pointers are owned by nobody. The problem with this design is that it is easy to create circular references or other strange conditions that cause an object to never be deleted. It is also slow to perform atomic operations every time a value is copied or assigned. Although they are not recommended, reference counted pointers are sometimes the simplest and most elegant way to solve a problem. cpplint link ▶Use cpplint.py to detect style errors. cpplint.py is a tool that reads a source file and identifies many style errors. It is not perfect, and has both false positives and false negatives, but it is still a valuable tool. False positives can be ignored by putting // NOLINT at the end of the line. Some projects have instructions on how to run cpplint.py from their project tools. If the project you are contributing to does not, you can download cpplint.py separately. Other C++ Features Reference Arguments link ▶All parameters passed by reference must be labeled const. Definition: In C, if a function needs to modify a variable, the parameter must use a pointer, eg int foo(int *pval). In C++, the function can alternatively declare a reference parameter: int foo(int &val). Pros: Defining a parameter as reference avoids ugly code like (*pval)++. Necessary for some applications like copy constructors. Makes it clear, unlike with pointers, that NULL is not a possible value. Cons: References can be confusing, as they have value syntax but pointer semantics. Decision: Within function parameter lists all references must be const: void Foo(const string &in, string *out); In fact it is a very strong convention in Google code that input arguments are values or const references while output arguments are pointers. Input parameters may be const pointers, but we never allow non-const reference parameters. One case when you might want an input parameter to be a const pointer is if you want to emphasize that the argument is not copied, so it must exist for the lifetime of the object; it is usually best to document this in comments as well. STL adapters such as bind2nd and mem_fun do not permit reference parameters, so you must declare functions with pointer parameters in these cases, too. Function Overloading link ▶Use overloaded functions (including constructors) only if a reader looking at a call site can get a good idea of what is happening without having to first figure out exactly which overload is being called. Definition: You may write a function that takes a const string& and overload it with another that takes const char*. class MyClass { public: void Analyze(const string &text); void Analyze(const char *text, size_t textlen); }; Pros: Overloading can make code more intuitive by allowing an identically-named function to take different arguments. It may be necessary for templatized code, and it can be convenient for Visitors. Cons: If a function is overloaded by the argument types alone, a reader may have to understand C++'s complex matching rules in order to tell what's going on. Also many people are confused by the semantics of inheritance if a derived class overrides only some of the variants of a function. Decision: If you want to overload a function, consider qualifying the name with some information about the arguments, e.g., AppendString(), AppendInt() rather than just Append(). Default Arguments link ▶We do not allow default function parameters, except in a few uncommon situations explained below. Pros: Often you have a function that uses lots of default values, but occasionally you want to override the defaults. Default parameters allow an easy way to do this without having to define many functions for the rare exceptions. Cons: People often figure out how to use an API by looking at existing code that uses it. Default parameters are more difficult to maintain because copy-and-paste from previous code may not reveal all the parameters. Copy-and-pasting of code segments can cause major problems when the default arguments are not appropriate for the new code. Decision: Except as described below, we require all arguments to be explicitly specified, to force programmers to consider the API and the values they are passing for each argument rather than silently accepting defaults they may not be aware of. One specific exception is when default arguments are used to simulate variable-length argument lists. // Support up to 4 params by using a default empty AlphaNum. string StrCat(const AlphaNum &a, const AlphaNum &b = gEmptyAlphaNum, const AlphaNum &c = gEmptyAlphaNum, const AlphaNum &d = gEmptyAlphaNum); Variable-Length Arrays and alloca() link ▶We do not allow variable-length arrays or alloca(). Pros: Variable-length arrays have natural-looking syntax. Both variable-length arrays and alloca() are very efficient. Cons: Variable-length arrays and alloca are not part of Standard C++. More importantly, they allocate a data-dependent amount of stack space that can trigger difficult-to-find memory overwriting bugs: "It ran fine on my machine, but dies mysteriously in production". Decision: Use a safe allocator instead, such as scoped_ptr/scoped_array. Friends link ▶We allow use of friend classes and functions, within reason. Friends should usually be defined in the same file so that the reader does not have to look in another file to find uses of the private members of a class. A common use of friend is to have a FooBuilder class be a friend of Foo so that it can construct the inner state of Foo correctly, without exposing this state to the world. In some cases it may be useful to make a unittest class a friend of the class it tests. Friends extend, but do not break, the encapsulation boundary of a class. In some cases this is better than making a member public when you want to give only one other class access to it. However, most classes should interact with other classes solely through their public members. Exceptions link ▶We do not use C++ exceptions. Pros: Exceptions allow higher levels of an application to decide how to handle "can't happen" failures in deeply nested functions, without the obscuring and error-prone bookkeeping of error codes. Exceptions are used by most other modern languages. Using them in C++ would make it more consistent with Python, Java, and the C++ that others are familiar with. Some third-party C++ libraries use exceptions, and turning them off internally makes it harder to integrate with those libraries. Exceptions are the only way for a constructor to fail. We can simulate this with a factory function or an Init() method, but these require heap allocation or a new "invalid" state, respectively. Exceptions are really handy in testing frameworks. Cons: When you add a throw statement to an existing function, you must examine all of its transitive callers. Either they must make at least the basic exception safety guarantee, or they must never catch the exception and be happy with the program terminating as a result. For instance, if f() calls g() calls h(), and h throws an exception that f catches, g has to be careful or it may not clean up properly. More generally, exceptions make the control flow of programs difficult to evaluate by looking at code: functions may return in places you don't expect. This causes maintainability and debugging difficulties. You can minimize this cost via some rules on how and where exceptions can be used, but at the cost of more that a developer needs to know and understand. Exception safety requires both RAII and different coding practices. Lots of supporting machinery is needed to make writing correct exception-safe code easy. Further, to avoid requiring readers to understand the entire call graph, exception-safe code must isolate logic that writes to persistent state into a "commit" phase. This will have both benefits and costs (perhaps where you're forced to obfuscate code to isolate the commit). Allowing exceptions would force us to always pay those costs even when they're not worth it. Turning on exceptions adds data to each binary produced, increasing compile time (probably slightly) and possibly increasing address space pressure. The availability of exceptions may encourage developers to throw them when they are not appropriate or recover from them when it's not safe to do so. For example, invalid user input should not cause exceptions to be thrown. We would need to make the style guide even longer to document these restrictions! Decision: On their face, the benefits of using exceptions outweigh the costs, especially in new projects. However, for existing code, the introduction of exceptions has implications on all dependent code. If exceptions can be propagated beyond a new project, it also becomes problematic to integrate the new project into existing exception-free code. Because most existing C++ code at Google is not prepared to deal with exceptions, it is comparatively difficult to adopt new code that generates exceptions. Given that Google's existing code is not exception-tolerant, the costs of using exceptions are somewhat greater than the costs in a new project. The conversion process would be slow and error-prone. We don't believe that the available alternatives to exceptions, such as error codes and assertions, introduce a significant burden. Our advice against using exceptions is not predicated on philosophical or moral grounds, but practical ones. Because we'd like to use our open-source projects at Google and it's difficult to do so if those projects use exceptions, we need to advise against exceptions in Google open-source projects as well. Things would probably be different if we had to do it all over again from scratch. There is an exception to this rule (no pun intended) for Windows code. Run-Time Type Information (RTTI) link ▶We do not use Run Time Type Information (RTTI). Definition: RTTI allows a programmer to query the C++ class of an object at run time. Pros: It is useful in some unittests. For example, it is useful in tests of factory classes where the test has to verify that a newly created object has the expected dynamic type. In rare circumstances, it is useful even outside of tests. Cons: A query of type during run-time typically means a design problem. If you need to know the type of an object at runtime, that is often an indication that you should reconsider the design of your class. Decision: Do not use RTTI, except in unittests. If you find yourself in need of writing code that behaves differently based on the class of an object, consider one of the alternatives to querying the type. Virtual methods are the preferred way of executing different code paths depending on a specific subclass type. This puts the work within the object itself. If the work belongs outside the object and instead in some processing code, consider a double-dispatch solution, such as the Visitor design pattern. This allows a facility outside the object itself to determine the type of class using the built-in type system. If you think you truly cannot use those ideas, you may use RTTI. But think twice about it. :-) Then think twice again. Do not hand-implement an RTTI-like workaround. The arguments against RTTI apply just as much to workarounds like class hierarchies with type tags. Casting link ▶Use C++ casts like static_cast(). Do not use other cast formats like int y = (int)x; or int y = int(x);. Definition: C++ introduced a different cast system from C that distinguishes the types of cast operations. Pros: The problem with C casts is the ambiguity of the operation; sometimes you are doing a conversion (e.g., (int)3.5) and sometimes you are doing a cast (e.g., (int)"hello"); C++ casts avoid this. Additionally C++ casts are more visible when searching for them. Cons: The syntax is nasty. Decision: Do not use C-style casts. Instead, use these C++-style casts. Use static_cast as the equivalent of a C-style cast that does value conversion, or when you need to explicitly up-cast a pointer from a class to its superclass. Use const_cast to remove the const qualifier (see const). Use reinterpret_cast to do unsafe conversions of pointer types to and from integer and other pointer types. Use this only if you know what you are doing and you understand the aliasing issues. Do not use dynamic_cast except in test code. If you need to know type information at runtime in this way outside of a unittest, you probably have a design flaw. Streams link ▶Use streams only for logging. Definition: Streams are a replacement for printf() and scanf(). Pros: With streams, you do not need to know the type of the object you are printing. You do not have problems with format strings not matching the argument list. (Though with gcc, you do not have that problem with printf either.) Streams have automatic constructors and destructors that open and close the relevant files. Cons: Streams make it difficult to do functionality like pread(). Some formatting (particularly the common format string idiom %.*s) is difficult if not impossible to do efficiently using streams without using printf-like hacks. Streams do not support operator reordering (the %1s directive), which is helpful for internationalization. Decision: Do not use streams, except where required by a logging interface. Use printf-like routines instead. There are various pros and cons to using streams, but in this case, as in many other cases, consistency trumps the debate. Do not use streams in your code. Extended Discussion There has been debate on this issue, so this explains the reasoning in greater depth. Recall the Only One Way guiding principle: we want to make sure that whenever we do a certain type of I/O, the code looks the same in all those places. Because of this, we do not want to allow users to decide between using streams or using printf plus Read/Write/etc. Instead, we should settle on one or the other. We made an exception for logging because it is a pretty specialized application, and for historical reasons. Proponents of streams have argued that streams are the obvious choice of the two, but the issue is not actually so clear. For every advantage of streams they point out, there is an equivalent disadvantage. The biggest advantage is that you do not need to know the type of the object to be printing. This is a fair point. But, there is a downside: you can easily use the wrong type, and the compiler will not warn you. It is easy to make this kind of mistake without knowing when using streams. cout << this; // Prints the address cout << *this; // Prints the contents The compiler does not generate an error because << has been overloaded. We discourage overloading for just this reason. Some say printf formatting is ugly and hard to read, but streams are often no better. Consider the following two fragments, both with the same typo. Which is easier to discover? cerr << "Error connecting to '" hostname.first << ":" hostname.second << ": " hostname.first, foo->bar()->hostname.second, strerror(errno)); And so on and so forth for any issue you might bring up. (You could argue, "Things would be better with the right wrappers," but if it is true for one scheme, is it not also true for the other? Also, remember the goal is to make the language smaller, not add yet more machinery that someone has to learn.) Either path would yield different advantages and disadvantages, and there is not a clearly superior solution. The simplicity doctrine mandates we settle on one of them though, and the majority decision was on printf + read/write. Preincrement and Predecrement link ▶Use prefix form (++i) of the increment and decrement operators with iterators and other template objects. Definition: When a variable is incremented (++i or i++) or decremented (--i or i--) and the value of the expression is not used, one must decide whether to preincrement (decrement) or postincrement (decrement). Pros: When the return value is ignored, the "pre" form (++i) is never less efficient than the "post" form (i++), and is often more efficient. This is because post-increment (or decrement) requires a copy of i to be made, which is the value of the expression. If i is an iterator or other non-scalar type, copying i could be expensive. Since the two types of increment behave the same when the value is ignored, why not just always pre-increment? Cons: The tradition developed, in C, of using post-increment when the expression value is not used, especially in for loops. Some find post-increment easier to read, since the "subject" (i) precedes the "verb" (++), just like in English. Decision: For simple scalar (non-object) values there is no reason to prefer one form and we allow either. For iterators and other template types, use pre-increment. Use of const link ▶We strongly recommend that you use const whenever it makes sense to do so. Definition: Declared variables and parameters can be preceded by the keyword const to indicate the variables are not changed (e.g., const int foo). Class functions can have the const qualifier to indicate the function does not change the state of the class member variables (e.g., class Foo { int Bar(char c) const; };). Pros: Easier for people to understand how variables are being used. Allows the compiler to do better type checking, and, conceivably, generate better code. Helps people convince themselves of program correctness because they know the functions they call are limited in how they can modify your variables. Helps people know what functions are safe to use without locks in multi-threaded programs. Cons: const is viral: if you pass a const variable to a function, that function must have const in its prototype (or the variable will need a const_cast). This can be a particular problem when calling library functions. Decision: const variables, data members, methods and arguments add a level of compile-time type checking; it is better to detect errors as soon as possible. Therefore we strongly recommend that you use const whenever it makes sense to do so: If a function does not modify an argument passed by reference or by pointer, that argument should be const. Declare methods to be const whenever possible. Accessors should almost always be const. Other methods should be const if they do not modify any data members, do not call any non-const methods, and do not return a non-const pointer or non-const reference to a data member. Consider making data members const whenever they do not need to be modified after construction. However, do not go crazy with const. Something like const int * const * const x; is likely overkill, even if it accurately describes how const x is. Focus on what's really useful to know: in this case, const int** x is probably sufficient. The mutable keyword is allowed but is unsafe when used with threads, so thread safety should be carefully considered first. Where to put the const Some people favor the form int const *foo to const int* foo. They argue that this is more readable because it's more consistent: it keeps the rule that const always follows the object it's describing. However, this consistency argument doesn't apply in this case, because the "don't go crazy" dictum eliminates most of the uses you'd have to be consistent with. Putting the const first is arguably more readable, since it follows English in putting the "adjective" (const) before the "noun" (int). That said, while we encourage putting const first, we do not require it. But be consistent with the code around you! Integer Types link ▶Of the built-in C++ integer types, the only one used is int. If a program needs a variable of a different size, use a precise-width integer type from , such as int16_t. Definition: C++ does not specify the sizes of its integer types. Typically people assume that short is 16 bits, int is 32 bits, long is 32 bits and long long is 64 bits. Pros: Uniformity of declaration. Cons: The sizes of integral types in C++ can vary based on compiler and architecture. Decision: defines types like int16_t, uint32_t, int64_t, etc. You should always use those in preference to short, unsigned long long and the like, when you need a guarantee on the size of an integer. Of the C integer types, only int should be used. When appropriate, you are welcome to use standard types like size_t and ptrdiff_t. We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t. For integers we know can be "big", use int64_t. You should not use the unsigned integer types such as uint32_t, unless the quantity you are representing is really a bit pattern rather than a number, or unless you need defined twos-complement overflow. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this. On Unsigned Integers Some people, including some textbook authors, recommend using unsigned types to represent numbers that are never negative. This is intended as a form of self-documentation. However, in C, the advantages of such documentation are outweighed by the real bugs it can introduce. Consider: for (unsigned int i = foo.Length()-1; i >= 0; --i) ... This code will never terminate! Sometimes gcc will notice this bug and warn you, but often it will not. Equally bad bugs can occur when comparing signed and unsigned variables. Basically, C's type-promotion scheme causes unsigned types to behave differently than one might expect. So, document that a variable is non-negative using assertions. Don't use an unsigned type. 64-bit Portability link ▶Code should be 64-bit and 32-bit friendly. Bear in mind problems of printing, comparisons, and structure alignment. printf() specifiers for some types are not cleanly portable between 32-bit and 64-bit systems. C99 defines some portable format specifiers. Unfortunately, MSVC 7.1 does not understand some of these specifiers and the standard is missing a few, so we have to define our own ugly versions in some cases (in the style of the standard include file inttypes.h): // printf macros for size_t, in the style of inttypes.h #ifdef _LP64 #define __PRIS_PREFIX "z" #else #define __PRIS_PREFIX #endif // Use these macros after a % in a printf format string // to get correct 32/64 bit behavior, like this: // size_t size = records.size(); // printf("%"PRIuS"\n", size); #define PRIdS __PRIS_PREFIX "d" #define PRIxS __PRIS_PREFIX "x" #define PRIuS __PRIS_PREFIX "u" #define PRIXS __PRIS_PREFIX "X" #define PRIoS __PRIS_PREFIX "o" Type DO NOT use DO use Notes void * (or any pointer) %lx %p int64_t %qd, %lld %"PRId64" uint64_t %qu, %llu, %llx %"PRIu64", %"PRIx64" size_t %u %"PRIuS", %"PRIxS" C99 specifies %zu ptrdiff_t %d %"PRIdS" C99 specifies %zd Note that the PRI* macros expand to independent strings which are concatenated by the compiler. Hence if you are using a non-constant format string, you need to insert the value of the macro into the format, rather than the name. It is still possible, as usual, to include length specifiers, etc., after the % when using the PRI* macros. So, e.g. printf("x = %30"PRIuS"\n", x) would expand on 32-bit Linux to printf("x = %30" "u" "\n", x), which the compiler will treat as printf("x = %30u\n", x). Remember that sizeof(void *) != sizeof(int). Use intptr_t if you want a pointer-sized integer. You may need to be careful with structure alignments, particularly for structures being stored on disk. Any class/structure with a int64_t/uint64_t member will by default end up being 8-byte aligned on a 64-bit system. If you have such structures being shared on disk between 32-bit and 64-bit code, you will need to ensure that they are packed the same on both architectures. Most compilers offer a way to alter structure alignment. For gcc, you can use __attribute__((packed)). MSVC offers #pragma pack() and __declspec(align()). Use the LL or ULL suffixes a
Contents Overview 1 Lesson 1: Concepts – Locks and Lock Manager 3 Lesson 2: Concepts – Batch and Transaction 31 Lesson 3: Concepts – Locks and Applications 51 Lesson 4: Information Collection and Analysis 63 Lesson 5: Concepts – Formulating and Implementing Resolution 81 Module 4: Troubleshooting Locking and Blocking Overview At the end of this module, you will be able to:  Discuss how lock manager uses lock mode, lock resources, and lock compatibility to achieve transaction isolation.  Describe the various transaction types and how transactions differ from batches.  Describe how to troubleshoot blocking and locking issues.  Analyze the output of blocking scripts and Microsoft® SQL Server™ Profiler to troubleshoot locking and blocking issues.  Formulate hypothesis to resolve locking and blocking issues. Lesson 1: Concepts – Locks and Lock Manager This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Describe locking architecture used by SQL Server.  Identify the various lock modes used by SQL Server.  Discuss lock compatibility and concurrent access.  Identify different types of lock resources.  Discuss dynamic locking and lock escalation.  Differentiate locks, latches, and other SQL Server internal “locking” mechanism such as spinlocks and other synchronization objects. Recommended Reading  Chapter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  SOX000821700049 – SQL 7.0 How to interpret lock resource Ids  SOX000925700237 – TITLE: Lock escalation in SQL 7.0  SOX001109700040 – INF: Queries with PREFETCH in the plan hold lock until the end of transaction Locking Concepts Delivery Tip Prior to delivering this material, test the class to see if they fully understand the different isolation levels. If the class is not confident in their understanding, review appendix A04_Locking and its accompanying PowerPoint® file. Transactions in SQL Server provide the ACID properties: Atomicity A transaction either commits or aborts. If a transaction commits, all of its effects remain. If it aborts, all of its effects are undone. It is an “all or nothing” operation. Consistency An application should maintain the consistency of a database. For example, if you defer constraint checking, it is your responsibility to ensure that the database is consistent. Isolation Concurrent transactions are isolated from the updates of other incomplete transactions. These updates do not constitute a consistent state. This property is often called serializability. For example, a second transaction traversing the doubly linked list mentioned above would see the list before or after the insert, but it will see only complete changes. Durability After a transaction commits, its effects will persist even if there are system failures. Consistency and isolation are the most important in describing SQL Server’s locking model. It is up to the application to define what consistency means, and isolation in some form is needed to achieve consistent results. SQL Server uses locking to achieve isolation. Definition of Dependency: A set of transactions can run concurrently if their outputs are disjoint from the union of one another’s input and output sets. For example, if T1 writes some object that is in T2’s input or output set, there is a dependency between T1 and T2. Bad Dependencies These include lost updates, dirty reads, non-repeatable reads, and phantoms. ANSI SQL Isolation Levels An isolation level determines the degree to which data is isolated for use by one process and guarded against interference from other processes. Prior to SQL Server 7.0, REPEATABLE READ and SERIALIZABLE isolation levels were synonymous. There was no way to prevent non-repeatable reads while not preventing phantoms. By default, SQL Server 2000 operates at an isolation level of READ COMMITTED. To make use of either more or less strict isolation levels in applications, locking can be customized for an entire session by setting the isolation level of the session with the SET TRANSACTION ISOLATION LEVEL statement. To determine the transaction isolation level currently set, use the DBCC USEROPTIONS statement, for example: USE pubs GO SET TRANSACTION ISOLATION LEVEL REPEATABLE READ GO DBCC USEROPTIONS GO Multigranular Locking Multigranular Locking In our example, if one transaction (T1) holds an exclusive lock at the table level, and another transaction (T2) holds an exclusive lock at the row level, each of the transactions believe they have exclusive access to the resource. In this scenario, since T1 believes it locks the entire table, it might inadvertently make changes to the same row that T2 thought it has locked exclusively. In a multigranular locking environment, there must be a way to effectively overcome this scenario. Intent lock is the answer to this problem. Intent Lock Intent Lock is the term used to mean placing a marker in a higher-level lock queue. The type of intent lock can also be called the multigranular lock mode. An intent lock indicates that SQL Server wants to acquire a shared (S) lock or exclusive (X) lock on some of the resources lower down in the hierarchy. For example, a shared intent lock placed at the table level means that a transaction intends on placing shared (S) locks on pages or rows within that table. Setting an intent lock at the table level prevents another transaction from subsequently acquiring an exclusive (X) lock on the table containing that page. Intent locks improve performance because SQL Server examines intent locks only at the table level to determine whether a transaction can safely acquire a lock on that table. This removes the requirement to examine every row or page lock on the table to determine whether a transaction can lock the entire table. Lock Mode The code shown in the slide represents how the lock mode is stored internally. You can see these codes by querying the master.dbo.spt_values table: SELECT * FROM master.dbo.spt_values WHERE type = N'L' However, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. For example, value of req_mode = 3 represents the Shared lock mode rather than the Schema Modification lock mode. Lock Compatibility These locks can apply at any coarser level of granularity. If a row is locked, SQL Server will apply intent locks at both the page and the table level. If a page is locked, SQL Server will apply an intent lock at the table level. SIX locks imply that we have shared access to a resource and we have also placed X locks at a lower level in the hierarchy. SQL Server never asks for SIX locks directly, they are always the result of a conversion. For example, suppose a transaction scanned a page using an S lock and then subsequently decided to perform a row level update. The row would obtain an X lock, but now the page would require an IX lock. The resultant mode on the page would be SIX. Another type of table lock is a schema stability lock (Sch-S) and is compatible with all table locks except the schema modification lock (Sch-M). The schema modification lock (Sch-M) is incompatible with all table locks. Locking Resources Delivery Tip Note the differences between Key and Key Range locks. Key Range locks will be covered in a couple of slides. SQL Server can lock these resources: Item Description DB A database. File A database file Index An entire index of a table. Table An entire table, including all data and indexes. Extent A contiguous group of data pages or index pages. Page An 8-KB data page or index page. Key Row lock within an index. Key-range A key-range. Used to lock ranges between records in a table to prevent phantom insertions or deletions into a set of records. Ensures serializable transactions. RID A Row Identifier. Used to individually lock a single row within a table. Application A lock resource defined by an application. The lock manager knows nothing about the resource format. It simply compares the 'strings' representing the lock resources to determine whether it has found a match. If a match is found, it knows that resource is already locked. Some of the resources have “sub-resources.” The followings are sub-resources displayed by the sp_lock output: Database Lock Sub-Resources: Full Database Lock (default) [BULK-OP-DB] – Bulk Operation Lock for Database [BULK-OP-LOG] – Bulk Operation Lock for Log Table Lock Sub-Resources: Full Table Lock (default) [UPD-STATS] – Update statistics Lock [COMPILE] – Compile Lock Index Lock sub-Resources: Full Index Lock (default) [INDEX_ID] – Index ID Lock [INDEX_NAME] – Index Name Lock [BULK_ALLOC] – Bulk Allocation Lock [DEFRAG] – Defragmentation Lock For more information, see also… SOX000821700049 SQL 7.0 How to interpret lock resource Ids Lock Resource Block The resource type has the following resource block format: Resource Type (Code) Content DB (2) Data 1: sub-resource; Data 2: 0; Data 3: 0 File (3) Data 1: File ID; Data 2: 0; Data 3: 0 Index (4) Data 1: Object ID; Data 2: sub-resource; Data 3: Index ID Table (5) Data 1: Object ID; Data 2: sub-resource; Data 3: 0. Page (6) Data 1: Page Number; Data 3: 0. Key (7) Data 1: Object ID; Data 2: Index ID; Data 3: Hashed Key Extent (8) Data 1: Extent ID; Data 3: 0. RID (9) Data 1: RID; Data 3: 0. Application (10) Data 1: Application resource name The rsc_bin column of master..syslockinfo contains the resource block in hexadecimal format. For an example of how to decode value from this column using the information above, let us assume we have the following value: 0x000705001F83D775010002014F0BEC4E With byte swapping within each field, this can be decoded as: Byte 0: Flag – 0x00 Byte 1: Resource Type – 0x07 (Key) Byte 2-3: DBID – 0x0005 Byte 4-7: ObjectID – 0x 75D7831F (1977058079) Byte 8-9: IndexID – 0x0001 Byte 10-16: Hash Key value – 0x 02014F0BEC4E For more information about how to decode this value, see also… Inside SQL Server 2000, pages 803 and 806. Key Range Locking Key Range Locking To support SERIALIZABLE transaction semantics, SQL Server needs to lock sets of rows specified by a predicate, such as WHERE salary BETWEEN 30000 AND 50000 SQL Server needs to lock data that does not exist! If no rows satisfy the WHERE condition the first time the range is scanned, no rows should be returned on any subsequent scans. Key range locks are similar to row locks on index keys (whether clustered or not). The locks are placed on individual keys rather than at the node level. The hash value consists of all the key components and the locator. So, for a nonclustered index over a heap, where columns c1 and c2 where indexed, the hash would contain contributions from c1, c2 and the RID. A key range lock applied to a particular key means that all keys between the value locked and the next value would be locked for all data modification. Key range locks can lock a slightly larger range than that implied by the WHERE clause. Suppose the following select was executed in a transaction with isolation level SERIALIZABLE: SELECT * FROM members WHERE first_name between ‘Al’ and ‘Carl’ If 'Al', 'Bob', and 'Dave' are index keys in the table, the first two of these would acquire key range locks. Although this would prevent anyone from inserting either 'Alex' or 'Ben', it would also prevent someone from inserting 'Dan', which is not within the range of the WHERE clause. Prior to SQL Server 7.0, page locking was used to prevent phantoms by locking the entire set of pages on which the phantom would exist. This can be too conservative. Key Range locking lets SQL Server lock only a much more restrictive area of the table. Impact Key-range locking ensures that these scenarios are SERIALIZABLE:  Range scan query  Singleton fetch of nonexistent row  Delete operation  Insert operation However, the following conditions must be satisfied before key-range locking can occur:  The transaction-isolation level must be set to SERIALIZABLE.  The operation performed on the data must use an index range access. Range locking is activated only when query processing (such as the optimizer) chooses an index path to access the data. Key Range Lock Mode Again, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. Dynamic Locking When modifying individual rows, SQL Server typically would take row locks to maximize concurrency (for example, OLTP, order-entry application). When scanning larger volumes of data, it would be more appropriate to take page or table locks to minimize the cost of acquiring locks (for example, DSS, data warehouse, reporting). Locking Decision The decision about which unit to lock is made dynamically, taking many factors into account, including other activity on the system. For example, if there are multiple transactions currently accessing a table, SQL Server will tend to favor row locking more so than it otherwise would. It may mean the difference between scanning the table now and paying a bit more in locking cost, or having to wait to acquire a more coarse lock. A preliminary locking decision is made during query optimization, but that decision can be adjusted when the query is actually executed. Lock Escalation When the lock count for the transaction exceeds and is a multiple of ESCALATION_THRESHOLD (1250), the Lock Manager attempts to escalate. For example, when a transaction acquired 1250 locks, lock manager will try to escalate. The number of locks held may continue to increase after the escalation attempt (for example, because new tables are accessed, or the previous lock escalation attempts failed due to incompatible locks held by another spid). If the lock count for this transaction reaches 2500 (1250 * 2), Lock Manager will attempt escalation again. The Lock Manager looks at the lock memory it is using and if it is more than 40 percent of SQL Server’s allocated buffer pool memory, it tries to find a scan (SDES) where no escalation has already been performed. It then repeats the search operation until all scans have been escalated or until the memory used drops under the MEMORY_LOAD_ESCALATION_THRESHOLD (40%) value. If lock escalation is not possible or fails to significantly reduce lock memory footprint, SQL Server can continue to acquire locks until the total lock memory reaches 60 percent of the buffer pool (MAX_LOCK_RESOURCE_MEMORY_PERCENTAGE=60). Lock escalation may be also done when a single scan (SDES) holds more than LOCK_ESCALATION_THRESHOLD (765) locks. There is no lock escalation on temporary tables or system tables. Trace Flag 1211 disables lock escalation. Important Do not relay this to the customer without careful consideration. Lock escalation is a necessary feature, not something to be avoided completely. Trace flags are global and disabling lock escalation could lead to out of memory situations, extremely poor performing queries, or other problems. Lock escalation tracing can be seen using the Profiler or with the general locking trace flag, -T1200. However, Trace Flag 1200 shows all lock activity so it should not be usable on a production system. For more information, see also… SOX000925700237 “TITLE: SQL 7.0 Lock escalation in SQL 7.0” Lock Timeout Application Lock Timeout An application can set lock timeout for a session with the SET option: SET LOCK_TIMEOUT N where N is a number of milliseconds. A value of -1 means that there will be no timeout, which is equivalent to the version 6.5 behavior. A value of 0 means that there will be no waiting; if a process finds a resource locked, it will generate error message 1222 and continue with the next statement. The current value of LOCK_TIMEOUT is stored in the global variable @@lock_timeout. Note After a lock timeout any transaction containing the statement, is rolled back or canceled by SQL Server 2000 (bug#352640 was filed). This behavior is different from that of SQL Server 7.0. With SQL Server 7.0, the application must have an error handler that can trap error 1222 and if an application does not trap the error, it can proceed unaware that an individual statement within a transaction has been canceled, and errors can occur because statements later in the transaction may depend on the statement that was never executed. Bug#352640 is fixed in hotfix build 8.00.266 whereby a lock timeout will only Internal Lock Timeout At time, internal operations within SQL Server will attempt to acquire locks via lock manager. Typically, these lock requests are issued with “no waiting.” For example, the ghost record processing might try to clean up rows on a particular page, and before it can do that, it needs to lock the page. Thus, the ghost record manager will request a page lock with no wait so that if it cannot lock the page, it will just move on to other pages; it can always come back to this page later. If you look at SQL Profiler Lock: Timeout events, internal lock timeout typically have a duration value of zero. Lock Duration Lock Mode and Transaction Isolation Level For REPEATABLE READ transaction isolation level, update locks are held until data is read and processed, unless promoted to exclusive locks. "Data is processed" means that we have decided whether the row in question matched the search criteria; if not then the update lock is released, otherwise, we get an exclusive lock and make the modification. Consider the following query: use northwind go dbcc traceon(3604, 1200, 1211) -- turn on lock tracing -- and disable escalation go set transaction isolation level repeatable read begin tran update dbo.[order details] set discount = convert (real, discount) where discount = 0.0 exec sp_lock Update locks are promoted to exclusive locks when there is a match; otherwise, the update lock is released. The sp_lock output verifies that the SPID does not hold any update locks or shared locks at the end of the query. Lock escalation is turned off so that exclusive table lock is not held at the end. Warning Do not use trace flag 1200 in a production environment because it produces a lot of output and slows down the server. Trace flag 1211 should not be used unless you have done extensive study to make sure it helps with performance. These trace flags are used here for illustration and learning purposes only. Lock Ownership Most of the locking discussion in this lesson relates to locks owned by “transactions.” In addition to transaction, cursor and session can be owners of locks and they both affect how long locks are held. For every row that is fetched, when SCROLL_LOCKS option is used, regardless of the state of a transaction, a cursor lock is held until the next row is fetched or when the cursor is closed. Locks owned by session are outside the scope of a transaction. The duration of these locks are bounded by the connection and the process will continue to hold these locks until the process disconnects. A typical lock owned by session is the database (DB) lock. Locking – Read Committed Scan Under read committed isolation level, when database pages are scanned, shared locks are held when the page is read and processed. The shared locks are released “behind” the scan and allow other transactions to update rows. It is important to note that the shared lock currently acquired will not be released until shared lock for the next page is successfully acquired (this is commonly know as “crabbing”). If the same pages are scanned again, rows may be modified or deleted by other transactions. Locking – Repeatable Read Scan Under repeatable read isolation level, when database pages are scanned, shared locks are held when the page is read and processed. SQL Server continues to hold these shared locks, thus preventing other transactions to update rows. If the same pages are scanned again, previously scanned rows will not change but new rows may be added by other transactions. Locking – Serializable Read Scan Under serializable read isolation level, when database pages are scanned, shared locks are held not only on rows but also on scanned key range. SQL Server continues to hold these shared locks until the end of transaction. Because key range locks are held, not only will this prevent other transactions from modifying the rows, no new rows can be inserted. Prefetch and Isolation Level Prefetch and Locking Behavior The prefetch feature is available for use with SQL Server 7.0 and SQL Server 2000. When searching for data using a nonclustered index, the index is searched for a particular value. When that value is found, the index points to the disk address. The traditional approach would be to immediately issue an I/O for that row, given the disk address. The result is one synchronous I/O per row and, at most, one disk at a time working to evaluate the query. This does not take advantage of striped disk sets. The prefetch feature takes a different approach. It continues looking for more record pointers in the nonclustered index. When it has collected a number of them, it provides the storage engine with prefetch hints. These hints tell the storage engine that the query processor will need these particular records soon. The storage engine can now issue several I/Os simultaneously, taking advantage of striped disk sets to execute multiple operations simultaneously. For example, if the engine is scanning a nonclustered index to determine which rows qualify but will eventually need to visit the data page as well to access columns that are not in the index, it may decide to submit asynchronous page read requests for a group of qualifying rows. The prefetch data pages are then revisited later to avoid waiting for each individual page read to complete in a serial fashion. This data access path requires that a lock be held between the prefetch request and the row lookup to stabilize the row on the page so it is not to be moved by a page split or clustered key update. For our example, the isolation level of the query is escalated to REPEATABLE READ, overriding the transaction isolation level. With SQL Server 7.0 and SQL Server 2000, portions of a transaction can execute at a different transaction isolation level than the entire transaction itself. This is implemented as lock classes. Lock classes are used to control lock lifetime when portions of a transaction need to execute at a stricter isolation level than the underlying transaction. Unfortunately, in SQL Server 7.0 and SQL Server 2000, the lock class is created at the topmost operator of the query and hence released only at the end of the query. Currently there is no support to release the lock (lock class) after the row has been discarded or fetched by the filter or join operator. This is because isolation level can be set at the query level via a lock class, but no lower. Because of this, locks acquired during the query will not be released until the query completes. If prefetch is occurring you may see a single SPID that holds hundreds of Shared KEY or PAG locks even though the connection’s isolation level is READ COMMITTED. Isolation level can be determined from DBCC PSS output. For details about this behavior see “SOX001109700040 INF: Queries with PREFETCH in the plan hold lock until the end of transaction”. Other Locking Mechanism Lock manager does not manage latches and spinlocks. Latches Latches are internal mechanisms used to protect pages while doing operations such as placing a row physically on a page, compressing space on a page, or retrieving rows from a page. Latches can roughly be divided into I/O latches and non-I/O latches. If you see a high number of non-I/O related latches, SQL Server is usually doing a large number of hash or sort operations in tempdb. You can monitor latch activities via DBCC SQLPERF(‘WAITSTATS’) command. Spinlock A spinlock is an internal data structure that is used to protect vital information that is shared within SQL Server. On a multi-processor machine, when SQL Server tries to access a particular resource protected by a spinlock, it must first acquire the spinlock. If it fails, it executes a loop that will check to see if the lock is available and if not, decrements a counter. If the counter reaches zero, it yields the processor to another thread and goes into a “sleep” (wait) state for a pre-determined amount of time. When it wakes, hopefully, the lock is free and available. If not, the loop starts again and it is terminated only when the lock is acquired. The reason for implementing a spinlock is that it is probably less costly to “spin” for a short time rather than yielding the processor. Yielding the processor will force an expensive context switch where:  The old thread’s state must be saved  The new thread’s state must be reloaded  The data stored in the L1 and L2 cache are useless to the processor On a single-processor computer, the loop is not useful because no other thread can be running and thus, no one can release the spinlock for the currently executing thread to acquire. In this situation, the thread yields the processor immediately. Lesson 2: Concepts – Batch and Transaction This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Review batch processing and error checking.  Review explicit, implicit and autocommit transactions and transaction nesting level.  Discuss how commit and rollback transaction done in stored procedure and trigger affects transaction nesting level.  Discuss various transaction isolation level and their impact on locking.  Discuss the difference between aborting a statement, a transaction, and a batch.  Describe how @@error, @@transcount, and @@rowcount can be used for error checking and handling. Recommended Reading  Charter 12 “Transactions and Triggers”, Inside SQL Server 2000 by Kalen Delaney Batch Definition SQL Profiler Statements and Batches To help further your understanding of what is a batch and what is a statement, you can use SQL Profiler to study the definition of batch and statement.  Try This: Using SQL Profiler to Analyze Batch 1. Log on to a server with Query Analyzer 2. Startup the SQL Profiler against the same server 3. Start a trace using the “StandardSQLProfiler” template 4. Execute the following using Query Analyzer: SELECT @@VERSION SELECT @@SPID The ‘SQL:BatchCompleted’ event is captured by the trace. It shows both the statements as a single batch. 5. Now execute the following using Query Analyzer {call sp_who()} What shows up? The ‘RPC:Completed’ with the sp_who information. RPC is simply another entry point to the SQL Server to call stored procedures with native data types. This allows one to avoid parsing. The ‘RPC:Completed’ event should be considered the same as a batch for the purposes of this discussion. Stop the current trace and start a new trace using the “SQLProfilerTSQL_SPs” template. Issue the same command as outlines in step 5 above. Looking at the output, not only can you see the batch markers but each statement as executed within the batch. Autocommit, Explicit, and Implicit Transaction Autocommit Transaction Mode (Default) Autocommit mode is the default transaction management mode of SQL Server. Every Transact-SQL statement, whether it is a standalone statement or part of a batch, is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back. A SQL Server connection operates in autocommit mode whenever this default mode has not been overridden by either explicit or implicit transactions. Autocommit mode is also the default mode for ADO, OLE DB, ODBC, and DB-Library. A SQL Server connection operates in autocommit mode until a BEGIN TRANSACTION statement starts an explicit transaction, or implicit transaction mode is set on. When the explicit transaction is committed or rolled back, or when implicit transaction mode is turned off, SQL Server returns to autocommit mode. Explicit Transaction Mode An explicit transaction is a transaction that starts with a BEGIN TRANSACTION statement. An explicit transaction can contain one or more statements and must be terminated by either a COMMIT TRANSACTION or a ROLLBACK TRANSACTION statement. Implicit Transaction Mode SQL Server can automatically or, more precisely, implicitly start a transaction for you if a SET IMPLICIT_TRANSACTIONS ON statement is run or if the implicit transaction option is turned on globally by running sp_configure ‘user options’ 2. (Actually, the bit mask 0x2 must be turned on for the user option so you might have to perform an ‘OR’ operation with the existing user option value.) See SQL Server 2000 Books Online on how to turn on implicit transaction under ODBC and OLE DB (acdata.chm::/ac_8_md_06_2g6r.htm). Transaction Nesting Explicit transactions can be nested. Committing inner transactions is ignored by SQL Server other than to decrements @@TRANCOUNT. The transaction is either committed or rolled back based on the action taken at the end of the outermost transaction. If the outer transaction is committed, the inner nested transactions are also committed. If the outer transaction is rolled back, then all inner transactions are also rolled back, regardless of whether the inner transactions were individually committed. Each call to COMMIT TRANSACTION applies to the last executed BEGIN TRANSACTION. If the BEGIN TRANSACTION statements are nested, then a COMMIT statement applies only to the last nested transaction, which is the innermost transaction. Even if a COMMIT TRANSACTION transaction_name statement within a nested transaction refers to the transaction name of the outer transaction, the commit applies only to the innermost transaction. If a ROLLBACK TRANSACTION statement without a transaction_name parameter is executed at any level of a set of nested transaction, it rolls back all the nested transactions, including the outermost transaction. The @@TRANCOUNT function records the current transaction nesting level. Each BEGIN TRANSACTION statement increments @@TRANCOUNT by one. Each COMMIT TRANSACTION statement decrements @@TRANCOUNT by one. A ROLLBACK TRANSACTION statement that does not have a transaction name rolls back all nested transactions and decrements @@TRANCOUNT to 0. A ROLLBACK TRANSACTION that uses the transaction name of the outermost transaction in a set of nested transactions rolls back all the nested transactions and decrements @@TRANCOUNT to 0. When you are unsure if you are already in a transaction, SELECT @@TRANCOUNT to determine whether it is 1 or more. If @@TRANCOUNT is 0 you are not in a transaction. You can also find the transaction nesting level by checking the sysprocess.open_tran column. See SQL Server 2000 Books Online topic “Nesting Transactions” (acdata.chm::/ac_8_md_06_66nq.htm) for more information. Statement, Transaction, and Batch Abort One batch can have many statements and one transaction can have multiple statements, also. One transaction can span multiple batches and one batch can have multiple transactions. Statement Abort Currently executing statement is aborted. This can be a bit confusing when you start talking about statements in a trigger or stored procedure. Let us look closely at the following trigger: CREATE TRIGGER TRG8134 ON TBL8134 AFTER INSERT AS BEGIN SELECT 1/0 SELECT 'Next command in trigger' END To fire the INSERT trigger, the batch could be as simple as ‘INSERT INTO TBL8134 VALUES(1)’. However, the trigger contains two statements that must be executed as part of the batch to satisfy the clients insert request. When the ‘SELECT 1/0’ causes the divide by zero error, a statement abort is issued for the ‘SELECT 1/0’ statement. Batch and Transaction Abort On SQL Server 2000 (and SQL Server 7.0) whenever a non-informational error is encountered in a trigger, the statement abort is promoted to a batch and transactional abort. Thus, in the example the statement abort for ‘select 1/0’ promotion results in an entire batch abort. No further statements in the trigger or batch will be executed and a rollback is issued. On SQL Server 6.5, the statement aborts immediately and results in a transaction abort. However, the rest of the statements within the trigger are executed. This trigger could return ‘Next command in trigger’ as a result set. Once the trigger completes the batch abort promotion takes effect. Conversely, submitting a similar set of statements in a standalone batch can result in different behavior. SELECT 1/0 SELECT 'Next command in batch' Not considering the set option possibilities, a divide by zero error generally results in a statement abort. Since it is not in a trigger, the promotion to a batch abort is avoided and subsequent SELECT statement can execute. The programmer should add an “if @@ERROR” check immediately after the ‘select 1/0’ to T-SQL execution to control the flow correctly. Aborting and Set Options ARITHABORT If SET ARITHABORT is ON, these error conditions cause the query or batch to terminate. If the errors occur in a transaction, the transaction is rolled back. If SET ARITHABORT is OFF and one of these errors occurs, a warning message is displayed, and NULL is assigned to the result of the arithmetic operation. When an INSERT, DELETE, or UPDATE statement encounters an arithmetic error (overflow, divide-by-zero, or a domain error) during expression evaluation when SET ARITHABORT is OFF, SQL Server inserts or updates a NULL value. If the target column is not nullable, the insert or update action fails and the user receives an error. XACT_ABORT When SET XACT_ABORT is ON, if a Transact-SQL statement raises a run-time error, the entire transaction is terminated and rolled back. When OFF, only the Transact-SQL statement that raised the error is rolled back and the transaction continues processing. Compile errors, such as syntax errors, are not affected by SET XACT_ABORT. For example: CREATE TABLE t1 (a int PRIMARY KEY) CREATE TABLE t2 (a int REFERENCES t1(a)) GO INSERT INTO t1 VALUES (1) INSERT INTO t1 VALUES (3) INSERT INTO t1 VALUES (4) INSERT INTO t1 VALUES (6) GO SET XACT_ABORT OFF GO BEGIN TRAN INSERT INTO t2 VALUES (1) INSERT INTO t2 VALUES (2) /* Foreign key error */ INSERT INTO t2 VALUES (3) COMMIT TRAN SELECT 'Continue running batch 1...' GO SET XACT_ABORT ON GO BEGIN TRAN INSERT INTO t2 VALUES (4) INSERT INTO t2 VALUES (5) /* Foreign key error */ INSERT INTO t2 VALUES (6) COMMIT TRAN SELECT 'Continue running batch 2...' GO /* Select shows only keys 1 and 3 added. Key 2 insert failed and was rolled back, but XACT_ABORT was OFF and rest of transaction succeeded. Key 5 insert error with XACT_ABORT ON caused all of the second transaction to roll back. Also note that 'Continue running batch 2...' is not Returned to indicate that the batch is aborted. */ SELECT * FROM t2 GO DROP TABLE t2 DROP TABLE t1 GO Compile and Run-time Errors Compile Errors Compile errors are encountered during syntax checks, security checks, and other general operations to prepare the batch for execution. These errors can prevent the optimization of the query and thus lead to immediate abort. The statement is not run and the batch is aborted. The transaction state is generally left untouched. For example, assume there are four statements in a particular batch. If the third statement has a syntax error, none of the statements in the batch is executed. Optimization Errors Optimization errors would include rare situations where the statement encounters a problem when attempting to build an optimal execution plan. Example: “too many tables referenced in the query” error is reported because a “work table” was added to the plan. Runtime Errors Runtime errors are those that are encountered during the execution of the query. Consider the following batch: SELECT * FROM pubs.dbo.titles UPDATE pubs.dbo.authors SET au_lname = au_lname SELECT * FROM foo UPDATE pubs.dbo.authors SET au_lname = au_lname If you run the above statements in a batch, the first two statements will be executed, the third statement will fail because table foo does not exist, and the batch will terminate. Deferred Name Resolution is the feature that allows this batch to start executing before resolving the object foo. This feature allows SQL Server to delay object resolution and place a “placeholder” in the query’s execution. The object referenced by the placeholder is resolved until the query is executed. In our example, the execution of the statement “SELECT * FROM foo” will trigger another compile process to resolve the name again. This time, error message 208 is returned. Error: 208, Level 16, State 1, Line 1 Invalid object name 'foo'. Message 208 can be encountered as a runtime or compile error depending on whether the Deferred Name Resolution feature is available. In SQL Server 6.5 this would be considered a compile error and on SQL Server 2000 (and SQL Server7.0) as a runtime error due to Deferred Name Resolution. In the following example, if a trigger referenced authors2, the error is detected as SQL Server attempts to execute the trigger. However, under SQL Server 6.5 the create trigger statement fails because authors2 does not exist at compile time. When errors are encountered in a trigger, generally, the statement, batch, and transaction are aborted. You should be able to observe this by running the following script in pubs database: Create table tblTest(iID int) go create trigger trgInsert on tblTest for INSERT as begin select * from authors select * from authors2 select * from titles end go begin tran select 'Before' insert into tblTest values(1) select 'After' go select @@TRANCOUNT go When run in a batch, the statement and the batch are aborted but the transaction remains active. The follow script illustrates this: begin tran select 'Before' select * from authors2 select 'After' go select @@TRANCOUNT go One other factor in a compile versus runtime error is implicit data type conversions. If you were to run the following statements on SQL Server 6.5 and SQL Server 2000 (and SQL Server 7.0): create table tblData(dtData datetime) go select 1 insert into tblData values(12/13/99) go On SQL Server 6.5, you get an error before execution of the batch begins so no statements are executed and the batch is aborted. Error: 206, Level 16, State 2, Line 2 Operand type clash: int is incompatible with datetime On SQL Server 2000, you get the default value (1900-01-01 00:00:00.000) inserted into the table. SQL Server 2000 implicit data type conversion treats this as integer division. The integer division of 12/13/99 is 0, so the default date and time value is inserted, no error returned. To correct the problem on either version is to wrap the date string with quotes. See Bug #56118 (sqlbug_70) for more details about this situation. Another example of a runtime error is a 605 message. Error: 605 Attempt to fetch logical page %S_PGID in database '%.*ls' belongs to object '%.*ls', not to object '%.*ls'. A 605 error is always a runtime error. However, depending on the transaction isolation level, (e.g. using the NOLOCK lock hint), established by the SPID the handling of the error can vary. Specifically, a 605 error is considered an ACCESS error. Errors associated with buffer and page access are found in the 600 series of errors. When the error is encountered, the isolation level of the SPID is examined to determine proper handling based on information or fatal error level. Transaction Error Checking Not all errors cause transactions to automatically rollback. Although it is difficult to determine exactly which errors will rollback transactions and which errors will not, the main idea here is that programmers must perform error checking and handle errors appropriately. Error Handling Raiserror Details Raiserror seems to be a source of confusion but is really rather simple. Raiserror with severity levels of 20 or higher will terminate the connection. Of course, when the connection is terminated a full rollback of any open transaction will immediately be instantiated by the SQL Server (except distributed transaction with DTC involved). Severity levels lower than 20 will simply result in the error message being returned to the client. They do not affect the transaction scope of the connection. Consider the following batch: use pubs begin tran update authors set au_lname = 'smith' raiserror ('This is bad', 19, 1) with log select @@trancount With severity set at 19, the 'select @@trancount' will be executed after the raiserror statement and will return a value of 1. If severity is changed to 20, then the select statement will not run and the connection is broken. Important Error handling must occur not only in T-SQL batches and stored procedures, but also in application program code. Transactions and Triggers (1 of 2) Basic behavior assumes the implicit transactions setting is set to OFF. This behavior makes it possible to identify business logic errors in a trigger, raise an error, rollback the action, and add an audit table entry. Logically, the insert to the audit table cannot take place before the ROLLBACK action and you would not want to build in the audit table insert into every applications error handler that violated the business rule of the trigger. For more information, see also… SQL Server 2000 Books Online topic “Rollbacks in stored procedure and triggers“ (acdata.chm::/ac_8_md_06_4qcz.htm) IMPLICIT_TRANSACTIONS ON Behavior The behavior of firing other triggers on the same table can be tricky. Say you added a trigger that checks the CODE field. Read only versions of the rows contain the code ‘RO’ and read/write versions use ‘RW.’ Whenever someone tries to delete a row with a code ‘RO’ the trigger issues the rollback and logs an audit table entry. However, you also have a second trigger that is responsible for cascading delete operations. One client could issue the delete without implicit transactions on and only the current trigger would execute and then terminate the batch. However, a second client with implicit transactions on could issue the same delete and the secondary trigger would fire. You end up with a situation in which the cascading delete operations can take place (are committed) but the initial row remains in the table because of the rollback operation. None of the delete operations should be allowed but because the transaction scope was restarted because of the implicit transactions setting, they did. Transactions and Triggers (2 of 2) It is extremely difficult to determine the execution state of a trigger when using explicit rollback statements in combination with implicit transactions. The RETURN statement is not allowed to return a value. The only way I have found to set the @@ERROR is using a ‘raiserror’ as the last execution statement in the last trigger to execute. If you modify the example, this following RAISERROR statement will set @@ERROR to 50000: CREATE TRIGGER trgTest on tblTest for INSERT AS BEGIN ROLLBACK INSERT INTO tblAudit VALUES (1) RAISERROR('This is bad', 14,1) END However, this value does not carry over to a secondary trigger for the same table. If you raise an error at the end of the first trigger and then look at @@ERROR in the secondary trigger the @@ERROR remains 0. Carrying Forward an Active/Open Transaction It is possible to exit from a trigger and carry forward an open transaction by issuing a BEGIN TRAN or by setting implicit transaction on and doing INSERT, UPDATE, or DELETE. Warning It is never recommended that a trigger call BEGIN TRANSACTION. By doing this you increment the transaction count. Invalid code logic, not calling commit transaction, can lead to a situation where the transaction count remains elevated upon exit of the trigger. Transaction Count The behavior is better explained by understanding how the server works. It does not matter whether you are in a transaction, when a modification takes place the transaction count is incremented. So, in the simplest form, during the processing of an insert the transaction count is 1. On completion of the insert, the server will commit (and thus decrement the transaction count). If the commit identifies the transaction count has returned to 0, the actual commit processing is completed. Issuing a commit when the transaction count is greater than 1 simply decrements the nested transaction counter. Thus, when we enter a trigger, the transaction count is 1. At the completion of the trigger, the transaction count will be 0 due to the commit issued at the end of the modification statement (insert). In our example, if the connection was already in a transaction and called the second INSERT, since implicit transaction is ON, the transaction count in the trigger will be 2 as long as the ROLLBACK is not executed. At the end of the insert, the commit is again issued to decrement the transaction reference count to 1. However, the value does not return to 0 so the transaction remains open/active. Subsequent triggers are only fired if the transaction count at the end of the trigger remains greater than or equal to 1. The key to continuation of secondary triggers and the batch is the transaction count at the end of a trigger execution. If the trigger that performs a rollback has done an explicit begin transaction or uses implicit transactions, subsequent triggers and the batch will continue. If the transaction count is not 1 or greater, subsequent triggers and the batch will not execute. Warning Forcing the transaction count after issuing a rollback is dangerous because you can easily loose track of your transaction nesting level. When performing an explicit rollback in a trigger, you should immediately issue a return statement to maintain consistent behavior between a connection with and without implicit transaction settings. This will force the trigger(s) and batch to terminate immediately. One of the methods of dealing with this issue is to run ‘SET IMPLICIT_TRANSACTIONS OFF’ as the first statement of any trigger. Other methods may entails checking @@TRANCOUNT at the end of the trigger and continue to COMMIT the transaction as long as @@TRANCOUNT is greater than 1. Examples The following examples are based on this table: create table tbl50000Insert (iID int NOT NULL) go Note If more than one trigger is used, to guarantee the trigger firing sequence, the sp_settriggerorder command should be used. This command is omitted in these examples to simplify the complexity of the statements. First Example In the first example, the second trigger was never fired and the batch, starting with the insert statement, was aborted. Thus, the print statement was never issued. print('Trigger issues rollback - cancels batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran select 'End of trigger', @@TRANCOUNT as 'TRANCOUNT' end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(1) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Second Example The next example shows that since a new transaction is started, the second trigger will be fired and the print statement in the batch will be executed. Note that the insert is rolled back. print('Trigger issues rollback - increases tran count to continue batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(2) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Third Example In the third example, the raiserror statement is used to set the @@ERROR value and the BEGIN TRAN statement is used in the trigger to allow the batch to continue to run. print('Trigger issues rollback - uses raiserror to set @@ERROR') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran -- Increase @@trancount to allow -- batch to continue select @@trancount as ‘Trancount’ raiserror('This is from the trigger', 14,1) end go insert into tbl50000Insert values(3) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert go delete from tbl50000Insert Fourth Example For the fourth example, a second trigger is added to illustrate the fact that @@ERROR value set in the first trigger will not be seen in the second trigger nor will it show up in the batch after the second trigger is fired. print('Trigger issues rollback - uses raiserror to set @@ERROR, not seen in second trigger and cleared in batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback begin tran -- Increase @@trancount to -- allow batch to continue select @@TRANCOUNT as 'Trancount' raiserror('This is from the trigger', 14,1) end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' end go insert into tbl50000Insert values(4) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Lesson 3: Concepts – Locks and Applications This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Explain how lock hints are used and their impact.  Discuss the effect on locking when an application uses Microsoft Transaction Server.  Identify the different kinds of deadlocks including distributed deadlock. Recommended Reading  Charter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  Charter 16 “Query Tuning”, Inside SQL Server 2000 by Kalen Delaney Q239753 – Deadlock Situation Not Detected by SQL Server Q288752 – Blocked SPID Not Participating in Deadlock May Incorrectly be Chosen as victim Locking Hints UPDLOCK If update locks are used instead of shared locks while reading a table, the locks are held until the end of the statement or transaction. UPDLOCK has the advantage of allowing you to read data (without blocking other readers) and update it later with the assurance that the data has not changed since you last read it. READPAST READPAST is an optimizer hint for use with SELECT statements. When this hint is used, SQL Server will read past locked rows. For example, assume table T1 contains a single integer column with the values of 1, 2, 3, 4, and 5. If transaction A changes the value of 3 to 8 but has not yet committed, a SELECT * FROM T1 (READPAST) yields values 1, 2, 4, 5. Tip READPAST only applies to transactions operating at READ COMMITTED isolation and only reads past row-level locks. This lock hint can be used to implement a work queue on a SQL Server table. For example, assume there are many external work requests being thrown into a table and they should be serviced in approximate insertion order but they do not have to be completely FIFO. If you have 4 worker threads consuming work items from the queue they could each pick up a record using read past locking and then delete the entry from the queue and commit when they're done. If they fail, they could rollback, leaving the entry on the queue for the next worker thread to pick up. Caution The READPAST hint is not compatible with HOLDLOCK.  Try This: Using Locking Hints 1. Open a Query Window and connect to the pubs database. 2. Execute the following statements (--Conn 1 is optional to help you keep track of each connection): BEGIN TRANSACTION -- Conn 1 UPDATE titles SET price = price * 0.9 WHERE title_id = 'BU1032' 3. Open a second connection and execute the following statements: SELECT @@lock_timeout -- Conn 2 GO SELECT * FROM titles SELECT * FROM authors 4. Open a third connection and execute the following statements: SET LOCK_TIMEOUT 0 -- Conn 3 SELECT * FROM titles SELECT * FROM authors 5. Open a fourth connection and execute the following statement: SELECT * FROM titles (READPAST) -- Conn 4 WHERE title_ID < 'C' SELECT * FROM authors How many records were returned? 3 6. Open a fifth connection and execute the following statement: SELECT * FROM titles (NOLOCK) -- Conn 5 WHERE title_ID 0 the lock manager also checks for deadlocks every time a SPID gets blocked. So a single deadlock will trigger 20 seconds of more immediate deadlock detection, but if no additional deadlocks occur in that 20 seconds, the lock manager no longer checks for deadlocks at each block and detection again only happens every 5 seconds. Although normally not needed, you may use trace flag -T1205 to trace the deadlock detection process. Note Please note the distinction between application lock and other locks’ deadlock detection. For application lock, we do not rollback the transaction of the deadlock victim but simply return a -3 to sp_getapplock, which the application needs to handle itself. Deadlock Resolution How is a deadlock resolved? SQL Server picks one of the connections as a deadlock victim. The victim is chosen based on either which is the least expensive transaction (calculated using the number and size of the log records) to roll back or in which process “SET DEADLOCK_PRIORITY LOW” is specified. The victim’s transaction is rolled back, held locks are released, and SQL Server sends error 1205 to the victim’s client application to notify it that it was chosen as a victim. The other process can then obtain access to the resource it was waiting on and continue. Error 1205: Your transaction (process ID #%d) was deadlocked with another process and has been chosen as the deadlock victim. Rerun your transaction. Symptoms of deadlocking Error 1205 usually is not written to the SQL Server errorlog. Unfortunately, you cannot use sp_altermessage to cause 1205 to be written to the errorlog. If the client application does not capture and display error 1205, some of the symptoms of deadlock occurring are:  Clients complain of mysteriously canceled queries when using certain features of an application.  May be accompanied by excessive blocking. Lock contention increases the chances that a deadlock will occur. Triggers and Deadlock Triggers promote the deadlock priority of the SPID for the life of the trigger execution when the DEADLOCK PRIORITY is not set to low. When a statement in a trigger causes a deadlock to occur, the SPID executing the trigger is given preferential treatment and will not become the victim. Warning Bug 235794 is filed against SQL Server 2000 where a blocked SPID that is not a participant of a deadlock may incorrectly be chosen as a deadlock victim if the SPID is blocked by one of the deadlock participants and the SPID has the least amount of transaction logging. See KB article Q288752: “Blocked Spid Not Participating in Deadlock May Incorrectly be Chosen as victim” for more information. Distributed Deadlock – Scenario 1 Distributed Deadlocks The term distributed deadlock is ambiguous. There are many types of distributed deadlocks. Scenario 1 Client application opens connection A, begins a transaction, acquires some locks, opens connection B, connection B gets blocked by A but the application is designed to not commit A’s transaction until B completes. Note SQL Server has no way of knowing that connection A is somehow dependent on B – they are two distinct connections with two distinct transactions. This situation is discussed in scenario #4 in “Q224453 INF: Understanding and Resolving SQL Server 7.0 Blocking Problems”. Distributed Deadlock – Scenario 2 Scenario 2 Distributed deadlock involving bound connections. Two connections can be bound into a single transaction context with sp_getbindtoken/sp_bindsession or via DTC. Spid 60 enlists in a transaction with spid 61. A third spid 62 is blocked by spid 60, but spid 61 is blocked by spid 62. Because they are doing work in the same transaction, spid 60 cannot commit until spid 61 finishes his work, but spid 61 is blocked by 62 who is blocked by 60. This scenario is described in article “Q239753 - Deadlock Situation Not Detected by SQL Server.” Note SQL Server 6.5 and 7.0 do not detect this deadlock. The SQL Server 2000 deadlock detection algorithm has been enhanced to detect this type of distributed deadlock. The diagram in the slide illustrates this situation. Resources locked by a spid are below that spid (in a box). Arrows indicate blocking and are drawn from the blocked spid to the resource that the spid requires. A circle represents a transaction; spids in the same transaction are shown in the same circle. Distributed Deadlock – Scenario 3 Scenario 3 Distributed deadlock involving linked servers or server-to-server RPC. Spid 60 on Server 1 executes a stored procedure on Server 2 via linked server. This stored procedure does a loopback linked server query against a table on Server 1, and this connection is blocked by a lock held by Spid 60. Note No version of SQL Server is currently designed to detect this distributed deadlock. Lesson 4: Information Collection and Analysis This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Identify specific information needed for troubleshooting issues.  Locate and collect information needed for troubleshooting issues.  Analyze output of DBCC Inputbuffer, DBCC PSS, and DBCC Page commands.  Review information collected from master.dbo.sysprocesses table.  Review information collected from master.dbo.syslockinfo table.  Review output of sp_who, sp_who2, sp_lock.  Analyze Profiler log for query usage pattern.  Review output of trace flags to help troubleshoot deadlocks. Recommended Reading Q244455 - INF: Definition of Sysprocesses Waittype and Lastwaittype Fields Q244456 - INF: Description of DBCC PSS Command for SQL Server 7.0 Q271509 - INF: How to Monitor SQL Server 2000 Blocking Q251004 - How to Monitor SQL Server 7.0 Blocking Q224453 - Understanding and Resolving SQL Server 7.0 Blocking Problem Q282749 – BUG: Deadlock information reported with SQL Server 2000 Profiler Locking and Blocking  Try This: Examine Blocked Processes 1. Open a Query Window and connect to the pubs database. Execute the following statements: BEGIN TRAN -- connection 1 UPDATE titles SET price = price + 1 2. Open another connection and execute the following statement: SELECT * FROM titles-- connection 2 3. Open a third connection and execute sp_who; note the process id (spid) of the blocked process. (Connection 3) 4. In the same connection, execute the following: SELECT spid, cmd, waittype FROM master..sysprocesses WHERE waittype 0 -- connection 3 5. Do not close any of the connections! What was the wait type of the blocked process?  Try This: Look at locks held Assumes all your connections are still open from the previous exercise. • Execute sp_lock -- Connection 3 What locks is the process from the previous example holding? Make sure you run ROLLBACK TRAN in Connection 1 to clean up your transaction. Collecting Information See Module 2 for more about how to gather this information using various tools. Recognizing Blocking Problems How to Recognize Blocking Problems  Users complain about poor performance at a certain time of day, or after a certain number of users connect.  SELECT * FROM sysprocesses or sp_who2 shows non-zero values in the blocked or BlkBy column.  More severe blocking incidents will have long blocking chains or large sysprocesses.waittime values for blocked spids.  Possibl

110,534

社区成员

发帖
与我相关
我的任务
社区描述
.NET技术 C#
社区管理员
  • C#
  • Web++
  • by_封爱
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告

让您成为最强悍的C#开发者

试试用AI创作助手写篇文章吧