ࡱ> q` bjbjqPqP ::p$ $ $ $ H H H 4| H,H,H,h,-| h6z7777:::$hEH L?:^:L?L?$ $ 77K(HHHL?^$ 7H 7HL?HHnfk H ^n76 P_wH,?vnl Dr!s<l' @'@^n^n'H Po:;H< =?:::F:::L?L?L?L?| | %i| | %| | | $ $ $ $ $ $   A Quick Look at Serial ATA (SATA) Disk Performance Tom Barclay Wyman Chong Jim Gray October 2003 Technical Report MSR-TR-2003-70 Microsoft Research Microsoft Corporation One Microsoft Way Redmond, WA 98052 A Quick Look at SATA Disk Performance Tom Barclay, Wyman Chong, Jim Gray {TBarclay, WymanC, Gray  HYPERLINK "mailto:Dslutz}@microsoft.com" }@microsoft.com Microsoft Research, 455 Market St., Suite 1690, San Francisco, CA 94105 http://research.microsoft.com/barc Abstract We have been investigating the use of low-cost, commodity components for multi-terabyte SQL Server databases [SQL]. Dubbed storage bricks, these servers are white box PCs containing the largest ATA drives, value-priced AMD or Intel processors, and inexpensive ECC memory. One issue has been the wiring mess, air flow problems, length restrictions, and connector failures created by seven or more parallel ATA (PATA) ribbon cables and drives in]a tower or 3U rack-mount chassis. Large capacity Serial ATA (SATA) drives have recently become widely available for the PC environment at a reasonable price. In addition to being faster, the SATA connectors seem more reliable, have a more reasonable length restriction (1m) and allow better airflow. We tested two drive brands along with two RAID controllers to evaluate SATA drive performance and reliablility. This paper documents our results so far. Overview We built and operate the TerraServer-USA web site  HYPERLINK "http://terraserver-usa.com" http://terraserver-usa.com [Barclay98, Barclay99, Barclay00, Barclay02]. It has been on the Internet since June 1998, and is one of the world's largest public repositories of high-resolution aerial and topographic data. The TerraServer hosts about 3.3 terabytes of user data growing about 20GB per month as new data arrives from the USGS [NATLMAP03]. The imagery is compressed into Jpeg or GIF format and stored in SQL Server relational databases [SQL]. Since September 2000, the TerraServer-USA hardware environment has been a classic three tier application (DMZ, web server, database server) backed by an enterprise class Storage Area Network (SAN.) 18 terabytes of raw disk storage is connected to the four database servers using fiber-channel. All the servers and storage were supplied by Hewlett-Packard. The system is designed for high availability. The database servers use Microsoft Clustering Service (MSC) with a hot standby server [MSCS03]. The storage environment has triple-mirrored disks, dual redundant controllers and dual-redundant SAN switch fabrics. This configuration has performed very well and has had no service outages other than server failovers that stall part of the database for less than a minute. When built in September 2000, this configuration including compute servers had a list price of $1.6 million. We are building the next generation TerraServer-USA environment using storage bricks. The goal of the project is to meet or exceed the performance and availability achieved by the SAN environment over the last three years at 10% of the capital cost, and a 1/3 the operations cost (hosting charges and operations tasks). ATA drives typically cost 800$/TB (PATA or Parallel ATA) and 1k$/TB (SATA or Serial ATA). Controllers, power, cooling, processors, and sheet metal bring the packaged price up to about 2k$/TB (or 3k$/TB for branded systems). We experimented with a number of hardware configurations and database partitioning schemes. A seven disk 1.75 TB storage brick with one 2.4 GHz processor and 1 GB of RAM can comfortably support the database activity of 1.4 TB of TerraServer data. So, 3 such storage bricks costing about $10,000 could support the TerraServer backend database activity. But, such a design would have no fault-tolerance. The experiments reported in this document verify that two hyper-threaded 2.4 GHz processors with 4 GB of RAM, and 16 disks (2 TB mirrored) can comfortably support the load and tolerate disk failures. We plan to deploy 3 such machines to support the TerraServer load, with an addition 3 mirrors and a spare server (7 nodes in all). This design is highly fault tolerant, and has a very easy recovery and management model. A separate paper will describe that design once the systems are operational. We commissioned Silicon Mechanics [SM03], a white-box PC manufacturer in Seattle Washington, to build two Storage Bricks with the following common attributes: 3u rack mountable chassis Dual 2.4ghz Xeon processors with hyper-threading SuperMicro Motherboard 4 GB of RAM 3ware 8506 SATA disk controllers 8 Western Digital 250gb SATA drives each with 8mb of cache 8 Maxtor 250gb SATA drives with 8mb of cache 2 on-board 10/100/1000 ethernet NIC 5 amps @ 208 volts One Storage Brick was configured with two 8 port SATA RAID controllers manufactured by 3Ware [3WARE]. The second Storage Brick was configured with four 4 port SATA RAID controllers manufactured by 3Ware. At the time the systems were configured, there were no other SATA RAID controllers available that support the 64-bit PCI bus. The experiment was architected to answer four high level questions: Which SATA drive performs best or is more reliable? What are the performance differences between two 4-port controllers and an 8-port controller? What are the performance and manageability differences between hardware and software RAID1 (mirroring)? What percentage of the normal peak weekly TerraServer application load can a storage brick support? Test Environment Test Methods We used two methods to evaluate the performance of SATA disks and controllers stress testing a single volume and then running the storage bricks under the simulated load of the real Teraserver. Stress Test a Single Volume The first method was designed to measure the maximum throughput of the drives and the controllers using synthetic I/O profiles. We used the SQLIO.EXE program originally written by the lead SQL Server developer to asses the performance of a disk sub-system running SQL Server. SQLIO can be configured to and measure read versus write, and sequential versus random performance varying the number of memory buffers, number overlapped (parallel) I/O operations, etc. We configured the SQLIO tests on one volume as follows: Sequential read using 64Kb buffers and 4-deep I/O. Sequential write using 64Kb buffers and 4-deep I/Os. Random read using 8Kb buffers and 1-deep I/O (no overlapped I/O) Random write using 8Kb buffers and 1-dep I/O. Random read using 8Kb buffers and 4-deep I/Os. Random write using 8Kb buffers and 4-deep I/Os. We found that there was no perceptible difference in performance when running a test for 1 minute versus 2 hours. The tests in this report were all run for 5 minutes. The tests are not equally relevant - some tests closely simulate how TerraServer works whereas others dont. The following ranks each tests value to TerraServer. The Random Read with 4 overlapped I/Os simulates how the TerraServer web application operates. The TerraServer web application is read-only and primarily performs single row fetches or single key index probes. The Random Write with 4 overlapped I/Os simulates how the TerraServer load processes operate. The TerraServer loaders operate one row at a time within a single transaction. A probe for tile existence, fetch meta data, possibly fetch image data, insert new row, and delete old row is the profile of the TerraServer load programs. The Sequential Read with 4 overlapped I/Os simulates how databases are backed up. The SQL backup utility sequentially read the database being backed up. SQL database restore sequentially reads an on-line backup whenever a server loses a disk and a database is restored. Software mirroring also does sequential reads during the restore of a mirror. Sequential Write with 4 overlapped I/Os simulates how data is restored to a replaced disk. The files are either copied using file system utilities or restored using the SQL Restore utility. Both methods execute large sequential write operations. Random Read with 1 overlapped I/O measures the ability of one drive to perform random read I/O operations without measuring the ability of the controller. Random Write with 1 overlapped I/O measures the ability of one drive to perform random write I/O operations without measuring the ability of the controller. TerraServer Database Request Playback The second method was designed to place the Storage Bricks under our real-world workload. We wrote a program, StressTestFromWebLog.exe that simulates the database operations performed by the TerraServer web application by reading a TerraServer web log and generating the appropriate requests. StressTestFromWebLog reads the web log files generated on the production web servers. The four production web servers each generate a web log. Each of these logs drives a separate instance of StressTestFromWebLog executed on a machine that has the same or more CPU power than the production web servers. The TerraServer web application runs within the ASP.NET framework and calls SQL stored procedures using ADO.NET. StressTestFromWebLog simulates the ASP.NET environment by using multiple threads to invoke the same SQL stored procedures using ADO.NET as the ASP.NET application does. The query string information in the web log has enough information to be able to discern the SQL stored procedure parameter values for six of the TerraServer SQL stored procedures that represent 97% of the daily database transactions. StressTestFromWebLog can be run in one of two modes fire hose or time synchronized. In fire-hose mode, the program executes SQL stored procedures as fast as possible. Each instance of the program will run up to 20 requests in parallel. If all 20 SQL processing threads are active, then the program waits for a thread to complete before invoking another parallel thread. This is similar to ASP.NET operation under load. In time-synchronized mode, StressTestFromWebLog uses the time stamp in the web log to pace the SQL database requests at the same rate they appear in the web log. We monitored the system performance to measure the performance difference between fire hose and time synchronized mode. Test Results SQLIO Stress Tests All these tests measure the performance of a single mirrored 250 GB SATA volume. We filled the volume with a 250GB file and ran SQLIO against that file. For random IO this is a worst case scenario since reads have no locality they are randomized across the entire disk. The following tables list the Test type, disk Vendor, IOps (I/O Operations per second), MBps (Mega-Bytes per second), and % of Best. The column heading identifies what was going on with the system and controllers. Normal indicates that the system and controllers were only executing the test. Ctlr Rebuild indicates that the RAID controller executing the test was re-building a mirror, but the test was not measuring that mirror. The Vol Rebuild means the controller was rebuilding the mirrored volume being measured. The Vendor column identifies the drive manufacturer (WD for Western Digital.) The Mirror column describes the kind of RAID1 volume: (Software for Windows 2003 mirroring [MSW2k303], 3ware for 3Ware controller mirroring [3Ware], and None for no RAID.) We forgot to measure Maxtor Ctlr Rebuild cases so those values are zero. Table 1: SQLIO Random 8KB 4-Deep Read Volume.VendorMirrorNormal IOpsCtlr Rebuild IOpsVol Rebuild IOpsMaxtorSoftware1460533ware137049None72730WDSoftware146145583ware13814454None75740% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware100%0%36%3ware94%0%34%None96%97%0%WDSoftware100%99%40%3ware95%99%37%None100%99%0%Table 1 shows the Random Read with 4 overlapped I/Os tests. Windows 2003 mirroring slightly outperforms 3Ware hardware mirroring. Mirror rebuilds reduce Random Read performance by about 70% in both the software and hardware mirror case. Table 2 shows Random 8KB Write performance with 4 overlapped I/Os. Similar to the read tests, the write tests Windows 2003 software mirrors perform best under normal circumstances. Repairing a broken mirror cuts controller and volume performance almost in for both software and hardware mirroring even if the volume is not involved in the failure. However, software mirroring outperformed hardware mirroring in most cases. Table 2: SQLIO Random 8KB 4-Deep Write Volume.VendorMirrorNormal IOpsCtlr Rebuild IOpsVol Rebuild IOpsMaxtorSoftware1170603ware117057None1161150WDSoftware12779793ware1208354None1301320% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware92%0%47%3ware92%0%45%None89%88%0%WDSoftware100%62%62%3ware94%65%43%None100%102%0%Table 3: Sequential 64KB 4-Deep Read SATA Volume VendorMirrorNormal MBpsCtlr Rebuild MBpsVol Rebuild MBpsMaxtorSoftware4806.683ware3308None48160WDSoftware500103ware33147.5None50140% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware96%0%13%3ware66%0%16%None96%32%0%WDSoftware100%0%20%3ware66%28%15%None100%28%0%Table 3 shows the Sequential Read performance (MBps) of 4-deep SQLIO tests. Windows 2003 software mirrors match the performance of no mirroring or failed mirroring. At 50 MBps, it is understandable why serial is the first word in the name of the drives. The Sequential Read test identified a problem with the 3Ware mirroring that seems caps the maximum volume read throughput at 32 MB/sec (525 IOps). Table 4 shows the bandwidth of the 64KB 4-deep Sequential Write tests. In all cases, the Western Digital drives out performed the Maxtor drives. Windows 2003 software mirroring outperformed 3Ware hardware mirroring except during rebuild where they are equal. Rebuilding a mirror has substantial performance costs. The 3Ware sequential read performance problem disappeared in the write case. Table 4: Sequential 64KB 4-Deep Write SATA Volume. VendorMirrorNormal MBpsCtlr Rebuild MBpsVol Rebuild MBpsMaxtorSoftware480253ware48010None48160WDSoftware5024243ware506.66.6None50140% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware96%0%50%3ware96%0%20%None96%32%0%WDSoftware100%48%48%3ware100%13%13%None100%28%0%Table 5: Random 8KB 1-Deep Read SATA VolumeVendorMirrorNormal IOpsCtlr Rebuild IOpsVol Rebuild IOpsMaxtorSoftware760303ware78049None73660WDSoftware7249533ware757353None74630% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware97%0%38%3ware100%0%63%None99%89%0%WDSoftware92%63%68%3ware96%94%68%None100%28%0%Table 5 shows 1-deep Random Read performance. Maxtor drives performed about 7% better. Software was slightly slower than hardware mirroring. The test shows the benefit of overlapped I/Os; 4-deep IO has nearly a 100% performance advantage since it can use both disk arms. Like all other tests, mirror rebuild dramatically reduces performance. Table 6 shows the 1-deep 8KB Random Write performance. Win2003 software mirroring out performed the 3Ware hardware mirroring on Western Digital drives but the performance was the same on Maxtor drives. As in all tests, rebuilding mirrors is an expensive operation. Table 6: Random 8KB 1-Deep Write SATA VolumeVendorMirrorNormal IOpsCtlr Rebuild IOpsVol Rebuild IOpsMaxtorSoftware1160413ware117058None1151150WDSoftware1270533ware1206452None1301290% of bestVendorMirrorNormalCtlr RebuildVol RebuildMaxtorSoftware91%0%32%3ware92%0%46%None88%88%0%WDSoftware100%0%42%3ware94%50%41%None100%99%0%StressTestFromWebLog We captured the web logs from the four TerraServer production web servers for Monday August 25, 2003. Mondays are traditionally busier days on average than other days of the week. Each log file was copied to a separate Windows 2003 server on the same network switch as the two Storage Brick servers. The Windows 2003 servers physically mimic the production web servers. Each server runs a copy of the StressTestFromWebLog program. Three of the servers running StressTestFromWebLog and the two Storage Bricks are connected to the same one gigabit network switch. The fourth server (TS-WEB) was connected to the building LAN at 100 Mbps and then to the Gbps network switch. As Table 7 shows, this server could not achieve the same throughput as servers directly attached to the gigabit switch. We executed StressTestFromWebLog in fire-hose mode repeatedly running all four web logs in parallel. The results from each run were very similar with a small standard deviation. Table 7 shows the average database throughput (calls per second) and average database calls handled by each server. The significant point is that two Storage Bricks are able to sustain average 180 database calls per second per storage server and the pair can execute 360 database calls per second. This is approximately four times the peak weekly load of TerraServer-USA.com. (The server throughput averages 20Mbps limit on the data output rate to the Internet.) Table 7: Throughput in fire hose mode. Test ServerAvg Calls per SecAvg Db CallsTS-WEB791,892,377TWEB1941,748,869TWEB3911,696,663TWEB4901,891,869Grand Total891,799,739When the StressTestFromWebLog programs were executed in time-synchronized mode, each simulated web server averaged 22.5 database calls per second. Thus we conclude that two Storage Bricks can handle 4 times the normal TerraServer workload. Table 8 shows the response time of the five most common database-related web pages (each of these pages invokes a single SQL stored procedure). We measured the time to receive the last byte of data from SQL Server. Table 8 shows for each page, the total number of calls, the average execution time (milliseconds), the maximum observed execution time over all tests (Max ms), and the average maximum time over all tests (average of max ms). The Tile fetch and Image meta-data fetch dominate the traffic. The other three methods download, famous [places], and imageinfo are insignificant. Table 8: Response time of each web page (and corresponding database stored procedure, measured in milliseconds.Web PageCallsAvg ms Max msAvg Max mstile1,606,8543916,0947,245image132,900638,1885,417download1,51316012,4844,481imageinfo199421,531252famous10591,781289Totals1,741,5404116,0949,581The average duration is 41 ms but there some calls have multi-second delays. We observed that these long delays occurred at the very beginning of each test. We summarize this is caused by SQL Server waking up, loading its buffer pool, compiling the stored procedures, and handling new connection requests all at once. Drive Reliability We stress tested four Western Digital drives in our office for a month driving them full speed with random 8KB IOs while mounted in an ordinary PC cabinet with no special cooling. This is a more demanding test than placing the drives in a correctly cooled and mechanically stable mounting. The test reported no drive errors. During the entire experiment we dealt with 32 drives. We saw one drive failure. We ascribe that to the burn-in process the drive was returned to the vendor for evaluation. Comparing Controllers 3ware sells the only controller 66 MHz-64bit PCI SATA controller at present. The 66/64 PCI interface has a theoretical bandwidth 528 MBps (=66x8) whereas the 33/32 PCI busses are limited to 132 MBps (33x4). The 3ware controllers cost about 70$/port and so add about 25% to the cost of the disk system if you buy large drives, and up to 100% of the cost of the system if you buy small (80GB) drives. Other companies, notably Highpoint Technologies and Promise offer less expensive 33/32 PCI SATA cards. We evaluated a 345$ 3ware Escalade 8506-4 4-port 66/64 PCI SATA card versus a 71$ Highpoint 1540 33/32 PCI SATA card (branded as a RocketRaid"! card). Both cards have comparable RAID software and management software on Windows. Both are easy to manage and both have email alerting. 3ware has a background raid scrubber  but otherwise the card s management software seems comparable. Figure 1 shows the sequential throughput of an 8-disk 3ware Escalade 8508 controller (costing 580$) versus a 4-disk Highpoint 1540 RocketRaid controller (costing 70$). All tests are 4-deep. Highpoint Sequential reads saturate at 110 MBps and 2 disks. Sequential writes saturate at 79MBps and 2 disks. 3ware saturates at 225 MBps read and 200MBps write. The combination of a faster PCI bus (66 MHz and 64 bits wide) and more powerful controller gives the 3ware 2 to 3 times the sequential performance of the 7x less expensive Highpoint card. The situation on random IO is somewhat different. Figure 2 shows that the Highpoint RocketRaid delivers better performance than the 3ware card on up to 4 disks. Random IO speed is not limited by the PCI Bus speed. The appendix has the detailed measurements but Figure 2 summarizes the comparison.  SHAPE \* MERGEFORMAT  In short, the Highpoint card delivers superior random IO performance but saturates at 100MBps on sequential IO (the bandwidth of 2 disks) while the 3ware card delivers approximately twice the sequential IO performance and saturates at 200 MBps (the bandwidth of 4 disks). Our conclusion from this is: if you have a random IO workload, the inexpensive SATA controllers are adequate and can save about 20% to 100% per drive when compared to the 3ware controllers. If you have a heavy sequential workload, the 3ware card has about 100% performance advantage but 4 disks will saturate the card, so you should buy multiple 4-port cards rather than putting 8 or 12 disks on one card. Conclusion The StressTestFromWebLog testing verified that our Storage Bricks with SATA drives can easily accommodate the TerraServer web application work load regardless of which SATA drive we select, which RAID controller we select, or how RAID1 mirroring is implemented. These choices can be made for economic, ease-of-management, and availability characteristics without regard to performance. From these experiments we conclude: Both the Western Digital and Maxtor drives were reliable. The Western Digital drives performed better on most tests including the random I/O tests which are the most similar to the TerraServer application workload and had comparable prices. Therefore, we selected Western Digital 250GB drives for the Storage Bricks. 3ware controllers are superior for high-speed sequential. For random IO the less expensive Highpoint controllers have superior performance, and they are 7x less expensive per port. So if there are enough PCI slots, we recommend using the Highpoint cards for workloads that do not need high-speed sequential IO. We found no performance difference between a 2x4-port RAID controller and a 1x8-port RAID controller? We found no availability or management difference between the 4-port and 8-port 3Ware RAID controller. Using the 4-port controller unnecessarily consumes 64-bit PCI slots on the motherboard and is more expensive per port than the 8-port controller. Therefore, we selected the 8-port 3Ware SATA RAID controller for all Storage Bricks. There are performance and manageability differences between hardware RAID1 and software RAID1, but we are not convinced which is better for TerraServer in the long run. The two management interfaces and availability profiles are very different. We did not completely explore them during our tests. 3Ware offers a web based management and monitoring environment that offers a number of interesting features automatic e-mail and pager alarms, background data integrity checking that are not available out of the box with Windows 2003 software mirroring. However, Windows 2003 is integrated into the Windows Computer Management application, emits serious events to the Event Log, and can be monitored by the Microsoft Operations Management (MOM) environment. Software mirroring allows us to mirror and stripe data across 3Ware controllers whereas a 3Ware RAID-1 mirror is implemented by a single controller. This gives the availability edge to software mirroring. Although in practice, we have never seen a 3Ware controller fail. Changing a mirror configuration with hardware RAID requires an OS reboot (for both Highpoint and 3ware) and 3ware configuration is done in a DOS phase during reboot. Software mirroring allows online change to the configuration. We decided that more study is required. We intend to run half of our Storage Bricks using software mirroring and half with hardware mirroring to see if any long-term benefits or issues with the alternative technologies. Two Storage Bricks can service 400% of the normal peak weekly TerraServer application load. We executed 97% of the actual database requests captured on August 25th, 2003. The two Storage Bricks could process the 24 hours of captured database transactions within less than 7 hours. Even with one simulated web server handicapped by its network connection, each simulated web server executed an average of 89 database calls per second when run in fire hose mode. When run in time synchronization mode, the database calls per second averaged 22.5 per second. Thus we believe that two storage bricks can handle 4 times the normal Monday workload. Based on these tests, we have decided to purchase six Storage Bricks with the following configuration: 3u rack mountable chassis Dual Xeon 2.4ghz hyper threaded processors SuperMicro motherboard 4 GB of RAM 2 3Ware 8-port 8506-8 SATA RAID controllers 16 Western Digital 250gb SATA drives 2 on-board 10/100/1000 Ethernet NICs Three Storage Bricks will be configured with Windows 2003 RAID-1 mirroring. The other three Storage Bricks will be configured using 3Ware hardware mirroring. While we will devise tests to measure our satisfaction with the manageability and availability with software mirroring versus hardware mirroring, we need to get on with the project. Therefore, we believe that part of the TerraServer Storage Brick project will be to evaluate and compare software mirroring to hardware mirroring in a production application. References [3ware] 3ware Serial ATA Controllers.  HYPERLINK "http://www.3ware.com/products/serial_ata.asp" http://www.3ware.com/products/serial_ata.asp [Barclay98] T. Barclay, et. al., The Microsoft TerraServer, June 1998, MS TR 98 17, Microsoft Corp, Redmond, WA.  HYPERLINK "http://research.microsoft.com/research/pubs/view.aspx?tr_id=155" http://research.microsoft.com/research/pubs/view.aspx?tr_id=155 [Barclay99] T. Barclay, T, et. al., Microsoft TerraServer: A Spatial Data Warehouse, June 1999, Microsoft Corp., Redmond WA.  HYPERLINK "http://research.microsoft.com/research/pubs/view.aspx?tr_id=280" http://research.microsoft.com/research/pubs/view.aspx?tr_id=280 [Barclay00] T. Barclay, et. al., About Microsoft TerraServer How TerraServer Works, Microsoft Corp., Redmond, WA.  HYPERLINK "http://terraserver-usa.com/About.aspx?n=AboutStoryOverview" http://terraserver-usa.com/About.aspx?n=AboutStoryOverview [Barclay02] T. Barclay; et. al., TerraService.NET: An Introduction to Web Services, June 2002, Microsoft Corp, Redmond, WA.  HYPERLINK "http://research.microsoft.com/research/pubs/view.aspx?tr_id=588" http://research.microsoft.com/research/pubs/view.aspx?tr_id=588 [MSCS03] Microsoft Corp., Redmond, WA.  HYPERLINK "http://approjects.co.za/?big=windowsserver2003/evaluation/overview/technologies/clustering.mspx" http://approjects.co.za/?big=windowsserver2003/evaluation/overview/technologies/clustering.mspx [NATLMAP03] US Geological Survey, Reston, VA.  HYPERLINK "http://nationalmap.usgs.gov" http://nationalmap.usgs.gov [SM03] Silicon Mechanics, SM542RX ,  HYPERLINK "http://www.siliconmechanics.com/" http://www.siliconmechanics.com/ [SQL] Microsoft SQL Server 7.0  HYPERLINK "http://microsoft.com/SQL/" http://microsoft.com/SQL/ [MSWin2k303] Microsoft Corp., Redmond, WA.  HYPERLINK "http://approjects.co.za/?big=windowsserver2003/default.mspx" http://approjects.co.za/?big=windowsserver2003/default.mspx Appendix: The detailed measurements from the comparison of the 3Ware and Highpoint SATA controllers on 1 to 4 disks (the 3ware controller can manage up to 8 disks and has a 4x faster PCI interface.. Tests done using SQL IO, 4-deep, 256KB sequential 8KB random IOs.     (  PAGE 1 ( Vendor rand/seqr/wdisksRate3wareHighpointRandomRead1MBps0.60.8IOps73.8102.82MBps1.21.6IOps148.2205.43MBps1.72.4IOps220.0305.64MBps2.33.0IOps292.2386.75MBps2.8IOps362.06MBps3.4IOps440.87MBps3.9IOps496.78MBps4.7IOps600.5Write1MBps1.01.4IOps126.1176.52MBps2.02.6IOps256.4336.73MBps3.04.0IOps382.1509.14MBps4.04.9IOps514.8625.45MBps4.9IOps626.16MBps6.0IOps763.77MBps6.8IOps867.38MBps8.0IOps1,018.3SequentialRead1MBps49.149.7IOps196.3795.92MBps99.799.4IOps398.81,590.63MBps146.3109.2IOps585.41,746.84MBps197.5111.8IOps790.01,788.65MBps223.9IOps895.76MBps227.2IOps908.97MBps224.9IOps899.58MBps226.4IOps905.6Write1MBps49.149.6IOps196.2794.12MBps99.378.7IOps397.21,259.93MBps147.766.9IOps591.01,070.64MBps198.558.6IOps793.8937.35MBps198.5IOps794.16MBps201.4IOps #'().9:GS]di  ( ) 5 = B C m n o ~  ? N b k r ْٚٚh+hM6hh=hH56CJ\] hH0JjhHUjhHU hM5\ hL55\hhh/UYhMhz ihH5CJ\hM5CJ\hL55CJ\hH9:;GS\]kl}  )"gd/P$a$ !xzz ~ .rUCEƀzF$a$  , = j v | R S    " + 5 B R j n } .1ao;<hO[ h/Ph2h+G4hx h\]0Jjh\]Ujh\]Uh\]hHh!hRh2hz ihh=hMD<GIKvwhku  .4i ]gnqr{} !;>?BC¼¼ h+aJ hNaJhKhfhNaJh+hjfh*khh=hO[h2h!h\]hHhFh!CJaJJCJKRTU18;Oxz7<=?IJZx}6;{'»´ hWhb28 hWh hWha hWhs hWh/PhjfhahO[h<h*kh?hNhNaJhKhfhNaJ hNaJBgK & FEƀzFgd<K & FEƀzFgd<gK & FEƀzFgd<K & FEƀzFgd<NgK & FEƀzFgd<K & FEƀzFgd<N{gK & FEƀzFgd<K & FEƀzFgd<BvbK & FEƀzF.gdw1gdaK & FEƀzF gd<'+,7Buv~  *0<=>bi  !$%69S U ] ^ !! !!!!8!=!z!~!hahZ hhhh@;hNhd+hNhw1hahtRv?gK & FEƀzF.gdw1K & FEƀzF.gdw1?m'FEƀzF.gdFEƀzFgd@;K & FEƀzF.gdw1 hK & F EƀzFgdw1FEƀzF..gdgd@; !V!gK & F EƀzFgdw1K & F EƀzFgdw1~!!!!!!!!!!!!!!!!""""""""1#X#N%O%%%%%%&C&I&Q&]&^&&'''()G)J)K)P)a)k)u)))))))+*-*.*B***+++++ÿخ hahotCJOJQJ^JaJh/PhahNhO[hNOJQJ^JhhothhO[hb<hJ!hsh7hw1hZDV!!!gK & F EƀzFgdw1K & F EƀzFgdw1!!"1#$dI & F EƀzF.gdJ!gd7K & F EƀzFgdw1$O%&'k!I & F EƀzF.gdJ!I & F EƀzF.gdJ!I & F EƀzF.gdJ!'C(()k%FEƀzF..gdI & F EƀzF.gdJ!I & F EƀzF.gdJ!).*c+~,i-//0<0O0a114idddgdKFEƀzF..gd tFEƀzF.gd tgdgd@; + +$+a+b+c+++~,- -1-6-i-------------.).Z.\.{.~..../////(/)/////////O000a11111111111111111112:2hphd h5 hp5 hK5hKh|;hthah t6hah thO[hhotI:2@2A2222223 333#3$3H3L3R3[3\3a3b3k3m3q3w3333333333333344"4$4)4H4P4V4Z4_4a4f4g4h44444444붲hphp5CJ\aJ hfhL5hfhb<hNhahN5hO[hphp5 hp5h1.hahd5hahp5hphthdhKhahahK5:444444256575<5C5G5H5M5S5\5_5`5h5s5t5{555555555555555666666,6-64686<6A6G6P6T6C788˧˛˛˧˛˧˛˛h/PB*ph hL5hL5hL5hfhp5CJaJhfh/G5CJ\aJhfh/GCJaJhfhpCJaJhfhp5CJ\aJhhp5CJ\aJhphp5CJ\aJh/G5CJ\aJ5444444 555!5vvvvv$$Ifa$gd/Gmkd$$Ifl'0644 la`p $$Ifa$gd/G !5"5)5252$$$$Ifa$gd/GkdY$$Iflr ''8'&';'0644 la`p2256585;5<5$kdq$$Iflr ''8'&';'0644 la`p2$$Ifa$gd/G<5=5C5G5I5L5$$Ifa$gd/G$$Ifa$gd/GL5M5N5S52$$$$Ifa$gd/Gkd$$Iflr ''8'&';'0644 la`p2S5V5Y5[5\5$kd$$Iflr ''8'&';'0644 la`p2$$Ifa$gd/G\5_5h5l5p5s5$$Ifa$gd/G$$Ifa$gd/Gs5t5u5{52$$$$Ifa$gd/Gkd$$Iflr ''8'&';'0644 la`p2{55555$kd$$Iflr ''8'&';'0644 la`p2$$Ifa$gd/G555555$$Ifa$gd/G$$Ifa$gd/G5552$$$Ifa$gd/Gkd$$Iflr ''8'&';'0644 la`p25555555vvv$$Ifa$gd/G$$Ifa$gd/Gmkd $$Ifl'0644 la`p 55552$$$$Ifa$gd/Gkd $$Iflr ''8'&';'0644 la`p255555$kd $$Iflr ''8'&';'0644 la`p2$$Ifa$gd/G555556$$Ifa$gd/G$$Ifa$gd/G66662$$$$Ifa$gd/Gkd $$Iflr ''8'&';'0644 la`p26 6666$kd $$Iflr ''8'&';'0644 la`p2$$Ifa$gd/G666$6(6,6$$Ifa$gd/G$$Ifa$gd/G,6-6.6462$$$$Ifa$gd/Gkd$$Iflr ''8'&';'0644 la`p24686<6@6A6$kd$$Iflr ''8'&';'0644 la`p2$$Ifa$gd/GA6B6G6L6P6S6$$Ifa$gd/G$$Ifa$gd/GS6T6C782--gdL5kd)$$Iflr ''8'&';'0644 la`p28#8;8G888889 999"9`9a9q9u9w9z9{9999999999999999999999ȽȽȽȯ}qf}ffqqfqffhhVCJaJhL5hV5CJaJhfhVCJaJhVhVCJaJhVhV5CJ\aJhh/G5CJ\aJhVh/G5CJ\aJh/G5CJ\aJhph/G5CJ\aJhL55CJ\aJhfB*phhL5hL5B*phhL5h/PB*ph(8!9"9)909<9I9N9[9`9qqqqq$$Ifa$gd/GrkdA$$IflH'T0T644 la`gzp $$Ifa$gd/G `9a9h9q9-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp2q9u9w9z9$$Ifa$gd/Gz9{9|99-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp29999$$Ifa$gd/G9999-$$Ifa$gd/Gkd'$$Iflr H''8'&';'+0T644 la`gzp29999$$Ifa$gd/G9999-$$Ifa$gd/GkdI$$Iflr H''8'&';'+0T644 la`gzp29999$$Ifa$gd/G9999-$$Ifa$gd/Gkdk$$Iflr H''8'&';'+0T644 la`gzp29999$$Ifa$gd/G9999-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp29999$$Ifa$gd/G999-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp29999::":%:&:):-:.:5:@:A:G:R:S:_:d:g:l:m:t:x:|:::::::::::xmbxhx\-5CJ\aJhO[5CJ\aJhh5CJ\aJhphV5CJaJhphVCJaJhL5hV5CJaJhVhV5CJaJhVhVCJaJhVhV5CJ\aJhh/G5CJ\aJhh/GCJaJhVh/GCJaJhVh/G5CJ\aJ#99999::qqq$$Ifa$gd/G$$Ifa$gd/Grkd$$IflH'T0T644 la`gzp :::":-$$Ifa$gd/Gkds$$Iflr H''8'&';'+0T644 la`gzp2":&:):-:$$Ifa$gd/G-:.:/:5:-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp25:9:<:@:$$Ifa$gd/G@:A:B:G:-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp2G:K:O:R:$$Ifa$gd/GR:S:V:_:-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp2_:d:h:l:$$Ifa$gd/Gl:m:n:t:-$$Ifa$gd/Gkd$$Iflr H''8'&';'+0T644 la`gzp2t:x:|::$$Ifa$gd/G::::-$$Ifa$gd/Gkd!$$Iflr H''8'&';'+0T644 la`gzp2::::$$Ifa$gd/G:::-$$Ifa$gdx\-kd?"$$Iflr H''8'&';'+0T644 la`gzp2:::::::;;vvvvv$$Ifa$gdx\-$$Ifa$gdx\-mkda#$$IflH'T0T644 la`p ;;;;2$$$$Ifa$gdx\-kd#$$Iflr H''8'&';'+0T644 la`p2:;";);1;7;=;@;L;Q;T;U;\;_;b;g;m;s;v;;;;;;;;;;;;<<< < <<<<<%<*<.<2<6=X=Y=}==========¶ͪ›hhllhL5hfhhCJaJhh5CJaJhh5CJaJhhCJaJhh5CJ\aJhfh5CJaJhfhCJaJhfh5CJ\aJ7;;;!;";$kd%$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-";#;);,;.;0;$$Ifa$gdx\-$$Ifa$gdx\-0;1;2;7;2$$$$Ifa$gdx\-kd)&$$Iflr H''8'&';'+0T644 la`p27;:;=;?;@;$kdA'$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-@;C;L;O;Q;T;$$Ifa$gdx\-$$Ifa$gdx\-T;U;V;\;2$$$$Ifa$gdx\-kdY($$Iflr H''8'&';'+0T644 la`p2\;_;b;f;g;$kdq)$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-g;h;m;p;s;u;$$Ifa$gdx\-$$Ifa$gdx\-u;v;;2$$$Ifa$gdx\-kd*$$Iflr H''8'&';'+0T644 la`p2;;;;;;;vvv$$Ifa$gdx\-$$Ifa$gdx\-mkd+$$IflH'T0T644 la`p ;;;;2$$$$Ifa$gdx\-kd9,$$Iflr H''8'&';'+0T644 la`p2;;;;;$kdQ-$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-;;;;;;$$Ifa$gdx\-$$Ifa$gdx\-;;;;2$$$$Ifa$gdx\-kdi.$$Iflr H''8'&';'+0T644 la`p2;;;;;$kd/$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-;;;<< <$$Ifa$gdx\-$$Ifa$gdx\- < < <<2$$$$Ifa$gdx\-kd0$$Iflr H''8'&';'+0T644 la`p2<<<<<$kd1$$Iflr H''8'&';'+0T644 la`p2$$Ifa$gdx\-< <%<*<.<1<$$Ifa$gdx\-$$Ifa$gdx\-1<2<6==2-(gdL5gdfkd2$$IflPr H''8'&';'+0T644 la`p2====>> >>2>8><>=>W>X>^>c>>>>>>>>>>???????????2?3?>?R?X?Y?????????????????ҶҶīīīhh. CJaJhfh. CJaJh. h. CJaJhh. 5CJ\aJh. h. 5CJ\aJh. h]k5CJ\aJhJlhllhfh/Ph>RhhO[hJK9=R??????????qqqqq$$Ifa$gdx\-mkd3$$IflH'T0T644 la`p $$Ifa$gdx\-gd t ????2$$$$Ifa$gdx\-kdy4$$Iflr #H'''~''%0T644 la`p2?????$kd5$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-??????$$Ifa$gdx\-$$Ifa$gdx\-????2$$$$Ifa$gdx\-kd6$$Iflr #H'''~''%0T644 la`p2?????$kd7$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-?????? @@@@@!@"@&@'@-@/@0@2@3@5@6@?@@@A@p@q@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ݎh. hx\-5CJ\aJhh. 5CJaJhh. 5CJ\aJh. h]kCJaJh. h]k5CJ\aJh. h. 5CJ\aJhh. CJaJh. h. CJaJh. h. 5CJaJ8?@ @ @@@$$Ifa$gdx\-$$Ifa$gdx\-@@@@2$$$$Ifa$gdx\-kd8$$Iflr #H'''~''%0T644 la`p2@@"@&@'@$kd9$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-'@(@-@0@3@5@$$Ifa$gdx\-$$Ifa$gdx\-5@6@@@2$$$Ifa$gdx\-kd ;$$Iflr #H'''~''%0T644 la`p2@@A@H@O@V@c@p@vvv$$Ifa$gdx\-$$Ifa$gdx\-mkd!<$$IflH'T0T644 la`p p@q@x@@2$$$$Ifa$gdx\-kd<$$Iflr #H'''~''%0T644 la`p2@@@@@$kd=$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-@@@@@@$$Ifa$gdx\-$$Ifa$gdx\-@@@@2$$$$Ifa$gdx\-kd>$$Iflr #H'''~''%0T644 la`p2@@@@@$kd@$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-@@@@@@$$Ifa$gdx\-$$Ifa$gdx\-@@@@2$$$$Ifa$gdx\-kdA$$Iflr #H'''~''%0T644 la`p2@@@@@$kd1B$$Iflr #H'''~''%0T644 la`p2$$Ifa$gdx\-@@@@@@$$Ifa$gdx\-$$Ifa$gdx\-@@ A2$$$Ifa$gdx\-kdIC$$IflPr #H'''~''%0T644 la`p2@@@AAAA AAA A!A_A`ApAsAuAxAyAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB BBBB$B%B,B8B9B?BJBKBhx\-hx\-5CJaJhhx\-CJaJhfhx\-CJaJhx\-hx\-CJaJhx\-hx\-5CJ\aJhhx\-5CJ\aJh. hx\-5CJ\aJhx\-5CJ\aJ> A!A(A/A;AHAMAZA_A$$Ifa$gdx\-$$Ifa$gdx\-FkdaD$$IflH'TT644 la`p _A`AgApAsAuAxAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdD$$Iflr #H'''~''%T644 la`p2xAyAzAAAAAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kd)F$$Iflr #H'''~''%T644 la`p2AAAAAAAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdpG$$Iflr #H'''~''%T644 la`p2AAAAAAAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdH$$Iflr #H'''~''%T644 la`p2AAAAAAAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdI$$Iflr #H'''~''%T644 la`p2AAAAAAAYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdEK$$Iflr #H'''~''%T644 la`p2AAAYK$$Ifa$gdx\-kdL$$Iflr #H'''~''%T644 la`p2AAAAAAB$$Ifa$gdx\-$$Ifa$gdx\-FkdM$$IflH'TT644 la`p B BBBB B$BYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdTN$$Iflr #H'''~''%T644 la`p2$B%B&B,B1B4B8BYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdO$$Iflr #H'''~''%T644 la`p28B9B:B?BCBGBJBYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdP$$Iflr #H'''~''%T644 la`p2JBKBNBWB[B_BcBYKK===$$Ifa$gdx\-$$Ifa$gdx\-kd)R$$Iflr #H'''~''%T644 la`p2KBWBZB[B^B_BbBcBdBkBrBsBwBxB~BBBBBCCDDDDDDDDDDDEEE E EEE"EĶě}r}r}r}r}r}h*5CJ\aJh. h*5CJ\aJh%ZhJlhJKhL5hfh. hx\-5CJaJh. hx\-CJaJh. hx\-5CJ\aJhhx\-CJaJhhx\-5CJaJhx\-hx\-5CJaJhx\-hx\-CJaJhx\-hx\-5CJ\aJ'cBdBeBkBoBsBwBYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdpS$$Iflr #H'''~''%T644 la`p2wBxByB~BBBBYKK===$$Ifa$gdx\-$$Ifa$gdx\-kdT$$Iflr #H'''~''%T644 la`p2BBCD"EYTOA$$Ifa$gd5QCgd tgdfkdU$$IflPr #H'''~''%T644 la`p2"E#E*E1E=EJEOE\EaEvvvvv$$Ifa$gd5QC$$Ifa$gd5QCmkdEW$$IflH'T0T644 la`p "E#EaEbErEwExE{E|EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEFF"F%F&F-F.F5F@FAFGFRFSF_FcFdFfFgFjFkFlFsFzF{FFFFFٶٶٶٶٶٶٶٶٶٶٶhh*5CJaJh1.h*5CJaJhh*CJaJ *h1.h*CJaJh1.h*CJaJh1.h*5CJ\aJhh*5CJ\aJCaEbEiErE2$$$$Ifa$gd5QCkdW$$Iflr #H'''~''%0T644 la`p2rEvExE{E|E$kdX$$Iflr #H'''~''%0T644 la`p2$$Ifa$gd5QC|E}EEEEE$$Ifa$gd5QC$$Ifa$gd5QCEEEE2$$$$Ifa$gd5QCkd Z$$Iflr #H'''~''%0T644 la`p2EEEEE$kd%[$$Iflr #H'''~''%0T644 la`p2$$Ifa$gd5QCEEEEEE$$Ifa$gd5QC$$Ifa$gd5QCEEEE2$$$$Ifa$gd5QCkd=\$$Iflr #H'''~''%0T644 la`p2EEEEE$kdU]$$Iflr #H'''~''%0T644 la`p2$$Ifa$gd5QCEEEEEE$$Ifa$gd5QC$$Ifa$gd5QCEEE2$$$Ifa$gd5QCkdm^$$Iflr #H'''~''%0T644 la`p2EEEEEFFvvv$$Ifa$gd5QC$$Ifa$gd5QCmkd_$$IflH'T0T644 la`p FFF"F2$$$$Ifa$gd5QCkd`$$Iflr #H'''~''%0T644 la`p2"F&F)F-F.F$kd5a$$Iflr #H'''~''%0T644 la`p2$$Ifa$gd5QC.F/F5F9FIRI_IIIII{J|JJJJJJJJJJK LL9LILKLLLgLhLuLͳŠŜh/P5B*CJ\aJphhh/PhFtThfh+$5B*CJ\aJph#h+$h+$5B*CJ\aJphh`hT1_h+$h'4h>Rh%Zhh*CJaJh1.h*CJaJh1.h*5CJaJ3FF2kdf$$IflPr #H'''~''%0T644 la`p2FF\HIJLLuL$$$Ifgd+$gdFtTgd'4FEƀzF..gdfuLvLLLL$$$$Ifa$gd+$mkdg$$Ifl;0644 lalp uLvLLLLLLLLLLLLLLLLLLL M MIMJMNN NANNNNNNNNNNNNOO.O:OXY YYYYYYYZ Z Z ZbZqZZZZZZ([*[;[p[[[,\-\]]Y]Z]q]r]s]t]u]v]]]]]]]O^S^^__ɾɶh%juhllUjhEnUmHnHuhEnjhEnU h-hV hllhV h'h&%h%HhIEFhhNh hNhNOJQJ^JaJhNhg<]Y]v]^!`,`aab[I & F EƀzFgdFgd5@gdFtTFEƀzFgd:gd'4gd&%__"_*_C___ `!`+`,`aaaaa bbbbb=b@bJbYbdbsbbbbbbbbbcccccd%d,dPdRd]ddddʽ孱奠h:h:5 h1} hIEF h1} h1} h1} hEn hEn5hEnh:5hFh`hFh`5 hF5hFhF5hFh:5h5@ h'4hFtThFtThah: hFtTh hEnhIEFhllh%H0bcRcRdk!I & F EƀzFgdFI & F EƀzFgdFI & F EƀzFgdFdddddddddSeTe[e_eeef f ff f$fll~ h'h{Ch'h:5h'hFhF5h'hF5 hRhRhN hR5hRhR5hR h5@5hFh{C5hFh:5hFhF5h5@hFh: h:5h:h:5 hF5h:hF50Rd ffkkk`` h^hgdNI & F EƀzFgdFI & F EƀzFgdFlllllllhninono o,o-o4oooqqqqqq!r"r#rOrPrQrȰ׬רygZOh1.h!CJaJh1.h1.0JCJaJ#jvh1.h5QCCJUaJh1.h1.CJaJjh1.h1.CJUaJ h1.h1. h1.hN h1.h!hHhIEF jhWhUmHnHu hWhIEF hWhRhRh:5hR h'h5@ h'h` h'h@; h'h{Ch'h{CH*kinnndK & F EƀzFgdNgdRI & F EƀzFgdFno,ogK & F EƀzFgdNK & F EƀzFgdN,o9oeogK & F EƀzFgdNK & F EƀzFgdNeoooqgbgdRK & F EƀzFgdNK & F EƀzFgdNqqQrVsgtguxvjwwZxxlygd$a$gdR$a$gd1.FEƀzFgd QrRr]r`rprvrrrrrrrrrrrrrrsssSsTsUsVsbseslszsss#t$t%tdtetftgtstvtttɾɟɔɾwɟɔllh1.ht"CJaJ#j^xh1.hCJUaJh1.hFrCJaJh1.hCJaJh1.h0JCJaJ#jvh1.hCJUaJh1.hCJaJjh1.hCJUaJh1.h1.CJaJh1.hFCJaJh1.hHCJaJ h1.hH*ttttt(u)u*udueufuguhuuuuuu4v5v6vuvvvwvxvyvųЦߛꐛvdWvSHh1.hRCJaJhRh1.h0JCJaJ#j4{h1.hCJUaJh1.hCJaJjh1.hCJUaJh1.hFrCJaJh1.hCJaJh1.h=T0JCJaJ#jyh1.h=TCJUaJh1.h=TCJaJjh1.h=TCJUaJh1.ht"CJaJh1.hFCJaJh1.ht"CJaJyvvv w w wgwhwjwwwwwwwwwwxx6x7x8xXxYxZxyxzxxugu^uWu h1.hHh1.h1.0Jjuh1.h5QCUjh1.h1.U h1.h1. h1.h!h1.hWU%0JCJaJ#j~h1.hWU%CJUaJjh1.hWU%CJUaJh1.hWU%CJaJh1.h)0JCJaJ#j|h1.h)CJUaJjh1.h)CJUaJh1.h)CJaJxxxxxxxxx-y/y0y1yhyiykylyuyvyyyy1zwzxzyz{z|z~zzzzzzzzzzzzzzջͲծٚ{p{hN0JmHnHu h2T0Jjh2T0JU jh2Th2Thx6(jhx6(Uh Nyh1} hgnhh~hb<0Jj̀hb<U h&hchb<jhb<Uhb<hHh1.hH0Jjh1.h1.Uj(h1.h5QCU)lyvywzxzzz{z}z~zzzzzzzzzzzz$a$gdIEF ^`gdgnFEƀzFgdzzzzzzzzzzzzzzzz { { {!{'{({){+{8{9{L{M{S{W{d{e{x{y{{{{{{{{{{{{{{{{˼˲˲|||||| h2ThCJOJQJ^JaJhCJOJQJ^JaJ h1} aJ h2Th1} CJOJQJ^JaJh6Oh1} 5aJh Ny5CJOJQJ^JaJ#h6Oh1} 5CJOJQJ^JaJ#h6Oh6O5CJOJQJ^JaJhx6(h2T jh2T0J0zzzzzzzzzzzzz$$Ifa$gd2T$$Ifa$gd2T zzzzzzzzYKKKKKK$$Ifa$gd2Tkd"$$Iflbri &&'&&>&T644 lagp2zzzzzA33%$$Ifa$gd2T$$Ifa$gd2Tkdc$$Iflbֈi &&'&&>&&644 lagp<zzzzz%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2Tzzzz{{ {$$Ifa$gd2T$$Ifa$gd2T { {{{{A33%$$Ifa$gd2T$$Ifa$gd2Tkdх$$Iflbֈi &&'&&>&&644 lagp<{{{ {!{%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T!{#{%{'{,{2{8{$$Ifa$gd2T$$Ifa$gd2T8{9{;{={?{A33%$$Ifa$gd2T$$Ifa$gd2Tkd[$$Iflbֈi &&'&&>&&644 lagp<?{D{H{L{M{%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2TM{O{Q{S{X{^{d{$$Ifa$gd2T$$Ifa$gd2Td{e{g{i{k{A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<k{p{t{x{y{%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2Ty{{{}{{{{{$$Ifa$gd2T$$Ifa$gd2T{{{{{A33%$$Ifa$gd2T$$Ifa$gd2Tkdo$$Iflbֈi &&'&&>&&644 lagp<{{{{{%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T{{{{{{{$$Ifa$gd2T$$Ifa$gd2T{{{{{A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<{{{{{%kd0$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T{{{{{{{$$Ifa$gd2T$$Ifa$gd2T{{{{{A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<{{{{{{||||||(|)|@|A|G|K|X|Y|l|m|s|w|||||||||||||||||||}}}}}}(})}:};}A}E}N}O}`}a}g}k}v}w}}}}}}}}}}}}}}}}}}~~ h2Th NyCJOJQJ^JaJh NyCJOJQJ^JaJ h2Th1} CJOJQJ^JaJ h1} aJP{{{{{%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T{{{{{||$$Ifa$gd2T$$Ifa$gd2T|||| |A33%$$Ifa$gd2T$$Ifa$gd2Tkd $$Iflbֈi &&'&&>&&644 lagp< |||||%kdD$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T|||| |&|(|$$Ifa$gd2T$$Ifa$gd2T(|)|+|1|3|A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<3|8|<|@|A|%kdΘ$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2TA|C|E|G|L|R|X|$$Ifa$gd2T$$Ifa$gd2TX|Y|[|]|_|A33%$$Ifa$gd2T$$Ifa$gd2Tkd!$$Iflbֈi &&'&&>&&644 lagp<_|d|h|l|m|%kdX$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2Tm|o|q|s|x|~||$$Ifa$gd2T$$Ifa$gd2T|||||A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<|||||%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T|||||||$$Ifa$gd2T$$Ifa$gd2T|||||A33%$$Ifa$gd2T$$Ifa$gd2Tkd5$$Iflbֈi &&'&&>&&644 lagp<|||||%kdl$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T|||||||$$Ifa$gd2T$$Ifa$gd2T|||||A33%$$Ifa$gd2T$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<|||||%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T|||||}}$$Ifa$gd2T$$Ifa$gd2T}}}} }A33%$$Ifa$gd2T$$Ifa$gd2TkdI$$Iflbֈi &&'&&>&&644 lagp< }}}}}%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T}}}} }&}(}$$Ifa$gd2T$$Ifa$gd2T(})}+}-}/}A33%$$Ifa$gd2T$$Ifa$gd2TkdӦ$$Iflbֈi &&'&&>&&644 lagp</}4}8}:};}%kd $$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T;}=}?}A}F}L}N}$$Ifa$gd2T$$Ifa$gd2TN}O}Q}S}U}A33%$$Ifa$gd2T$$Ifa$gd2Tkd]$$Iflbֈi &&'&&>&&644 lagp<U}Z}^}`}a}%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2Ta}c}e}g}l}t}v}$$Ifa$gd2T$$Ifa$gd2Tv}w}y}{}}}}A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<}}}}}}3kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T}}}}}$$Ifa$gd2T$$Ifa$gd2T}}}}}}A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<}}}}}3%$$Ifa$gd2Tkdү$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T}}}}}}$$Ifa$gd2T$$Ifa$gd2T}}}}}}A3333$$Ifa$gd2Tkd $$Iflbֈi &&'&&>&&644 lagp<}}}}}3%$$Ifa$gd2Tkd\$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T}}}}}~$$Ifa$gd2T$$Ifa$gd2T~~~ ~ ~~A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<~ ~~~ ~7~8~>~B~Q~R~e~f~l~p~y~z~~~~~~~~~~~~~~~~~~~  #$9:@DSTjkqu8:FN`bU h1} aJh NyCJOJQJ^JaJ h2Th1} CJOJQJ^JaJT~~~ ~"~3%$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T"~$~&~+~1~7~$$Ifa$gd2T$$Ifa$gd2T7~8~:~<~>~C~A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<C~I~Q~R~T~3%$$Ifa$gd2Tkdp$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2TT~V~X~]~c~e~$$Ifa$gd2T$$Ifa$gd2Te~f~h~j~l~q~A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<q~w~y~z~|~%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T|~~~~~~~$$Ifa$gd2T$$Ifa$gd2T~~~~~~A3333$$Ifa$gd2Tkd1$$Iflbֈi &&'&&>&&644 lagp<~~~~~%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T~~~~~~$$Ifa$gd2T$$Ifa$gd2T~~~~~~A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<~~~~~%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T~~~~~~$$Ifa$gd2T$$Ifa$gd2T~~~~~~A3333$$Ifa$gd2TkdE$$Iflbֈi &&'&&>&&644 lagp<~~~~~%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T~~~ $$Ifa$gd2T$$Ifa$gd2T  A3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<#$&3%$$Ifa$gd2Tkd"$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T&(*/49$$Ifa$gd2T$$Ifa$gd2T9:<>@EA3333$$Ifa$gd2TkdY$$Iflbֈi &&'&&>&&644 lagp<EKSTV3%$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2TVXZ_ej$$Ifa$gd2T$$Ifa$gd2TjkmoqvA3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<v|3%$$Ifa$gd2Tkd6$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T$$Ifa$gd2TA3333$$Ifa$gd2Tkdm$$Iflbֈi &&'&&>&&644 lagp<3%$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T$$Ifa$gd2TA3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<%kdJ$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T$$Ifa$gd2T$$Ifa$gd2TA3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<805.57MBps198.7IOps794.78MBps201.2IOps804.8  Figure 2: The random IO throughput (IOps) of a Highpoint RocketRaid"! 33/32 PCI card and a 3ware 8-port 66/64 PCI card. 8KB requests are 4-deep.  Figure 1: S %kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2T(48$$Ifa$gd2T$$Ifa$gd2T8:>BFPA3333$$Ifa$gd2Tkd $$Iflbֈi &&'&&>&&644 lagp<P\`bf%kd^$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2Tfjnx$$Ifa$gd2T$$Ifa$gd2TA3333$$Ifa$gd2Tkd$$Iflbֈi &&'&&>&&644 lagp<%kd$$Iflbֈi &&'&&>&&644 lagp<$$Ifa$gd2T$$Ifa$gd2TgdIEFgd&%gdEngd1} *jlhH h-h'hV Uh'hV h'5 hV h2TjhhV hV U hEnhEnhNhEnjhEnhEnUh2Th1} equential throughput of a 4-disk Highpoint RocketRaid"! 1540 controller and an 8-disk 3ware Escalade"! 8506 controller. 21h:phF/ =!8"8#$% / 01h/ =!8"8#$% 3 01h/ =!8"8#$% P DyK Dslutz}@microsoft.comyK :mailto:Dslutz}@microsoft.comDyK http://terraserver-usa.comyK 8http://terraserver-usa.com/$$If`!vh5#v:V l06,59a`p $$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5#v:V l06,59a`p $$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5585&5;5#v#v8#v&#v;#v:V l06,5585&5;59a`p2$$If`!vh5T#vT:V l0T6,5T9a`gzp $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2$$If`!vh5T#vT:V l0T6,5T9a`gzp $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2 $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`gzp2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V l0T6,5585&5;5+9a`p2$$If`!vh5585&5;5+#v#v8#v&#v;#v+:V lP0T6,5585&5;5+9a`p2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V lP0T6,555~55%9a`p2$$If`!vh5T#vT:V lT6,5T9/  / a`p E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2$$If`!vh5T#vT:V lT6,5T9/ /  a`p E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ /  / / / /  / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / /  / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lT6,555~55%9/ / / / / / / a`p2E$$If`!vh555~55%#v#v#v~#v#v%:V lPT6,555~55%9/ /  / / / /  / a`p2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh5T#vT:V l0T6,5T9a`p $$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V l0T6,555~55%9a`p2$$If`!vh555~55%#v#v#v~#v#v%:V lP0T6,555~55%9a`p2$$Ifl!vh5#v:V l;06,5alp $$Ifl!vh555#v#v#v:V l06,555alp$$Ifl!vh555#v#v#v:V l06,5559alp$$Ifl!vh555#v#v#v:V l06,5559alp$$Ifl!vh555#v#v#v:V l06,5559alp$$Ifl!vh555#v#v#v:V l06,5559alp$$Ifl!vh555#v#v#v:V l,06,5559alp$$IfN!vh5#v:V l054aN$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2$$IfN!vh5V58555#vV#v8#v#v#v:V l0,5V585554aNp2Dd D  3 @@"?DyK yK Zhttp://www.3ware.com/products/serial_ata.aspuDyK @http://research.microsoft.com/research/pubs/view.aspx?tr_id=155yK http://research.microsoft.com/research/pubs/view.aspx?tr_id=155uDyK @http://research.microsoft.com/research/pubs/view.aspx?tr_id=280yK http://research.microsoft.com/research/pubs/view.aspx?tr_id=280aDyK ;http://terraserver-usa.com/About.aspx?n=AboutStoryOverviewyK vhttp://terraserver-usa.com/About.aspx?n=AboutStoryOverviewuDyK @http://research.microsoft.com/research/pubs/view.aspx?tr_id=588yK http://research.microsoft.com/research/pubs/view.aspx?tr_id=588DyK \http://www.microsoft.com/windowsserver2003/evaluation/overview/technologies/clustering.mspxyK http://www.microsoft.com/windowsserver2003/evaluation/overview/technologies/clustering.mspxDyK http://nationalmap.usgs.govyK :http://nationalmap.usgs.gov/DyK yK Bhttp://www.siliconmechanics.com/DyK yK 4http://microsoft.com/SQL/UDyK 8http://www.microsoft.com/windowsserver2003/default.mspxyK phttp://www.microsoft.com/windowsserver2003/default.mspx?$$Ifg!vh5n5'5>5>5#vn#v'#v>#v:V lb6,55'55>5T9/ / / / /  / / agp25$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/ /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/ /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<{$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/ / / / / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/ /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<Q$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / /  /  / agp<5$$Ifg!vh5n5'5>5>55f#vn#v'#v>#v#vf:V lb6,55'55>559/  / / / agp<I Dd )||0  # A2 O'dgMZog~O cz`! O'dgMZog~On8H/$g x[}p\U?o?&Y:R;#; )u҄:N+ 8C 6Ml~Th׺Q| SZ1!MK ZZll-*?t(:ˌ_}}M켾s~sϽs}րkyxFRPg0*DKR\>`rF#q滦>1Ț?j4ؤ}Zi&CfӗlX)aUhm- ,^#\]޾uBƬ7^-gՖ˲|дmf(HQ"Z_`0d%/B/7KB Hދ9c4[+>mLN7F|gZaN=BT6BJk/#=zp(O;Vy)[ ߾Ж?_h}+~eJk{PUO`39RK3ĐGSCjU LUwO.hu0V%c;aڈyMܬON"OP35$3auB/X X2?Mp;ڃ=@!tR K$,NX:$P!oƮY[WFD6) R)kQᛴPf}pS`l,05a2@a $,IX8auVIX%af[NRkܻHX#a}݄'8aa1aבׯ#1rKy\?fL_~{ eLW!flH9r5x@-_syAWH.8wc$ˈKQh&&%@%)"z70 %FAp̵ dH*xV{ !oO1J^B.xOI'.ON< AÏ`>xTJ4)`EDoĠ3Ȏvƈ{;ظa6@6F oP3t;F-оۙҎ!z[-`W/|jq&J&#):pMӸ~8io,:Mʢ~=yr^Ƣ!MR*g~x1֢5u*Wƚw86yvO8o{GJ&g9Umpd1s2al20>8U5㩲 \&qƪ}UYugxG}ƪXx$;yoJjF^Y0$L[5Sߛav鲳8n >fdEm ]=6ڸ/z={b(BW ~~J^q\b4>˵~:aR-K Ko6ŚcѕݽkzP.Oϴ/9-i?e̩ʱz;Nu8U]U%!K/r_LUA9qUSԋU{[_!+8RmAqƂ q :GZqd)5rϑjߙ=]ɎK8;N󃈺 rDŽ'#XuVuٰEY[A>+|gzcv(rngnSgAtxj%J ͂ Dd ekk0  # A2 *4*g s1 z`! *4*g s1;L0$ x[kl\G>w~vׄ65 q0ة(v -ڛqL jU k;6%McqKR VQUETVÖE" @ ;scww/jќ9͙3s=ܱ __+ >aT/ j*%K*y|6NZF_CRGHc^,hWdb p >HMiŶ?]f@ej>&{tZÁ+FS/1+b|о'k۟2~H 7äGvwj-/蚅- X4ynsw9nmfscf(,ͬUyxSnYkP,8TrԣҶ…+lն9vr y;yJm?&I) :_:󌈸WA Rfɀe$ZP`A12F1Fw_ ʜߘO.d?B@2˼m j bqqdֺ1pio9꓾wӰ_ Yѓ)%^/ZJpcs5՘yw5<<,D-v5T@߲2S*EsO. Iǀ.: 7IIxk%^ jg2eO.C:wxwxLj2^&]!^xi% %ց<|z1/[a]_j=Ɉ;@z2%s"4;x$$)AmYAmA{jfa?Ii|khkT 5ӻǺ&w/ uj!Q 3ȝl3Il;8X?#'[;D}N lt&uWS=0ϐwtCS0@$aax:*B*phH>@"H Title $xa$5B*CJ\aJph@&1@ Footnote ReferenceH*F/@BF Listhx^h` B*aJph4@R4 Header  !4 @b4 Footer  !.)@q. Page NumberDD  Footnote Text B*aJph>> TOC 2 ^:B*aJphFV@F FrFollowedHyperlink >*B* phH@H z i Balloon TextCJOJQJ^JaJjj %H Table Grid7:V0 VWw 0/ &  VW   w4e,,!-"-;-<-L-M-[-\-s-t-----------....,.-.@.A.S.T.!1`1a1z1{111111111111122-2.2@2A2R2S2l2m222222233!3"30313?3@3T3U3f3g3u3v33333333333 4 4441424777777778&858@8p8888888 9!9_9`9x9y9999999:$:8:J:c:w::"=#=a=b={=|=============>>->.>@>A>R>S>k>l>>>>>uDvDDDDDDDDDDD E EHHHHII0I1IQIRImInIIIIIwK00nK00P K00PnK000K00nK00 K00nK00 K00 nK0 0 00K0 0 0nK0 0 h K0 0 hnK00 K00nK00 K00nK00H0K00HnK00 K00nK00 K00nK00 K00nK00( K00(nK00 K00bK00P K00bK00P K00nK00PnK00 K00nK00 K00nK00 K00 nK0 0 0 K0 0 0nK0 0 h K0 0 hnK00 K00nK00 K00nK00H0K00HnK00 K00nK00 K00nK00 K00nK00( K00(nK00K0;0<B0K0=0>B0K00nK00P K00PnK00 K00nK00 K00nK00 K00 nK0 0 0 K0 0 0nK0 0 h K0 0 hnK00 K00nK00 K00nK00H K00HnK00 K00nK00 K00nK00 K00nK00( K00(nK0;0<BK0;0<BK0=0>BK0=0>BK00K00nK00P K00PnK00 K00nK00 K00nK00 nK0 0 0nK0 0 hnK00nK00nK00HnK00nK00nK00nK00(nK0;0<BK0=0>BK00nK00P K00PnK00 K00nK00 K00nK00 nK0 0 0nK0 0 hnK00nK00nK00HnK00nK00nK00nK00(nK0;0<BK0=0>BK00nK00P K00PnK00 K00nK00 K00nK00 K00 nK0 0 0 K0 0 0nK0 0 h K0 0 hnK00 K00nK00 K00nK00H K00HnK00 K00nK00 K00nK00 K00nK00( K00(nK0;0<BK0;0<BK0=0>BK0=0>BK00X1K00X10 K0010 K0010 K002@0 K0082@0 K00p2@0 K002@0 K0044@0 K00l4K004K004K004K004K005K005K00L5K00L5K005K005K005K005@0 0:;GS\]kl}~ . r U N{Bv?V1OC !."c#~$i%'/(<(O(a)),,,,,, ---!-"-)-2-6-8-;-<-=-C-G-I-L-M-N-S-V-Y-[-\-_-h-l-p-s-t-u-{-----------------------------.... ......$.(.,.-...4.8.<.@.A.B.G.L.P.S.T.C/0!1"1)101<1I1N1[1`1a1h1q1u1w1z1{1|111111111111111111111111111111111112222"2&2)2-2.2/25292<2@2A2B2G2K2O2R2S2V2_2d2h2l2m2n2t2x2|22222222222222223333333!3"3#3)3,3.303132373:3=3?3@3C3L3O3Q3T3U3V3\3_3b3f3g3h3m3p3s3u3v3333333333333333333333333333344 4 4 444444 4%4*4.41424655R777777777777777777777777777778 8 8888888"8&8'8(8-808385868@8A8H8O8V8c8p8q8x888888888888888888888888888888888888 9!9(9/9;9H9M9Z9_9`9g9p9s9u9x9y9z999999999999999999999999999999999999: :::: :$:%:&:,:1:4:8:9:::?:C:G:J:K:N:W:[:_:c:d:e:k:o:s:w:x:y:~:::::;<"=#=*=1===J=O=\=a=b=i=r=v=x={=|=}===================================>>>>">&>)>->.>/>5>9><>@>A>B>G>K>O>R>S>V>_>d>g>k>l>m>s>w>{>>>>>>>>>>\@ABLDuDvDDDDDDDDDDDDDDDDDDDDDDE E EFVHHHHHHHHHHII IIII!I$I*I0I1I:I@IDIKIQIRI\I`IcIiImInIuIyI{IIIIIIIIIIJJFLLMMNOPRSSSUVV0XTXXYYZ\5]abbdPejeeeee f/f5h@hhijklmbnn>oooppppqqqqqqqqqqqqqq q"q$q&q(q/q0q1q:q>qDqIqOqYqZqaqfqhqmqqquqvqxqzq|qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqr rrrrrrr r"r#r%r'r)r.r4r6r7r9r;r=rBrFrHrIrKrMrOrTrZr\r]r_rarcrhrlrnrorqrsrurzrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrsss s ssssssss$s*s0s1s3s5s7st@tBtDtItNtStTtVtXtZt_tetmtntptrtttytttttttttttttttttttttttttttttttttttttttttuu u uuuuuuu!u"u$u&u(u-u3u5u6u8u:u0>0>0>0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0>0>0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 00J0J 00L0L0L0L0L0L0L0L0L0L 00V0V 0V 0V 0V 0V 0V 0V0V0V 0V0V 0V 0V 0V 0V 0V 0V 0V0V 005h05h05h05h05h05h05h05h05h05h 00o0o@0@0@0@0@0@0 0000000000000 0 0 0 00 0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 0 0 0 0 0 0 0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 @0 @0 0 0 @0 @0 @0 @0 @0 0 @0 @0 @0 @0 0 0 0 0 0 0 0 0000000000:;GS\]kl}~ . r U N{Bv?V1OC !."c#~$i%'/(<(O(a)),,,,,, ---!-"-)-2-6-8-;-<-=-C-G-I-L-M-N-S-V-Y-[-\-_-h-l-p-s-t-u-{-----------------------------.... ......$.(.,.-...4.8.<.@.A.B.G.L.P.S.T.C/0!1"1)101<1I1N1[1`1a1h1q1u1w1z1{1|111111111111111111111111111111111112222"2&2)2-2.2/25292<2@2A2B2G2K2O2R2S2V2_2d2h2l2m2n2t2x2|22222222222222223333333!3"3#3)3,3.303132373:3=3?3@3C3L3O3Q3T3U3V3\3_3b3f3g3h3m3p3s3u3v3333333333333333333333333333344 4 4 444444 4%4*4.41424655R777777777777777777777777777778 8 8888888"8&8'8(8-808385868@8A8H8O8V8c8p8q8x888888888888888888888888888888888888 9!9(9/9;9H9M9Z9_9`9g9p9s9u9x9y9z999999999999999999999999999999999999: :::: :$:%:&:,:1:4:8:9:::?:C:G:J:K:N:W:[:_:c:d:e:k:o:s:w:x:y:~:::::;<"=#=*=1===J=O=\=a=b=i=r=v=x={=|=}===================================>>>>">&>)>->.>/>5>9><>@>A>B>G>K>O>R>S>V>_>d>g>k>l>m>s>w>{>>>>>>>>>>\@ABLDuDvDDDDDDDDDDDDDDDDDDDDDDE E EFVHHHHHHHHHHII IIII!I$I*I0I1I:I@IDIKIQIRI\I`IcIiImInIuIyI{IIIIIIIIIIJJFLLMMNOPRSSSUVV0XTXXYYZ\5]abbdPejeeeee f/f5h@hhijklmbnn>oooppppqqq0q1qYquqqqqqqqr"r6rHr\rnrrrrrrrss0sDs\snsssssssst%t=tStmttttttt u!u5uIu]uquuuuuuvv3vGv[vovvvvvvvvvvvkwmwoww000000000000000000000000000000000 00~0~0~0~0~0~0~0~0~ 0~ 0~ 0~ 0~ 0~ 0~ 0~ 0~ 0~0~0~ 0~ 0~ 0~ 0~ 0 00( 000 0 0 0 0 0 000 0 0 0 0 0 0( 00 0 0 0 0 0  0( 0S(S(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`( 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`( 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(, 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(0`(4 ( 0S(S(0>0>0>0>0>0>0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0>0>0>0>D 0>0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0> 0>0> 00K0K 00(M0(M0(M0(M0(M0(M0(M0(M0(M0(M 00V0V 0V 0V 0V 0V 0V 0V0V0V 0V0V 0V 0V 0V 0V 0V 0V 0V0V 00Ig0Ig0Ig0Ig0Ig0Ig0Ig0Ig0Ig0Ig 00oa<I00I00I00I00I00K00D*N @00 K00CGPK00BLGPK00AGPK00@ GPK0 0? GPK0 0> ,HPK00=dHPK00<HPK00;HPK00: IPK009DIPK008|IPK007IPK006IPK005$JPK0 04!\JPK0"03#JPK0$02%PK0&01'ȇPK0(00)PK0*0/+8PK0,0.-pPK0.0-/PK000,1PK020+3PK040*5PPK060)7PK080(9PK0:0';PK0<0&=0PK0>0%?hPK0@0$APK0B0#C؊PK0D0"EPK0F0!GHPK0H0 IPK0J0KPK0L0MPK0N0OPK0P0QLPK0R0SPK0T0UPK0V0WPK0X0Y,PK0Z0[dPK0\0]PK0^0_ԬPK0`0a PK0b0cDPK0d0e|PK0f0gPK0h0iPK0j0k$PK0l0m\PK0n0 oPK0p0 q̮PK0r0 sPK0t0 u?@ABCDEFGHIJKLMNPQRSTUVWXYZ[\]^_`abcdefghijklmnopstuvwxyzBBn~SSSfhhhDiiiUjjj_kkkflll mmmnDn`nnnnn"o%>  .#  s"*?` - c $X99? >%> N /    >%>   N & 3 & P 0   "`  B S  ?FSew&.o'ZT.Ht0&-T/</d//d/#/"//</ʆ/L}/.|/"///l!/!/>w  > *urn:schemas-microsoft-com:office:smarttags PostalCode9 *urn:schemas-microsoft-com:office:smarttagsState8 *urn:schemas-microsoft-com:office:smarttagsCity: *urn:schemas-microsoft-com:office:smarttagsStreet9 *urn:schemas-microsoft-com:office:smarttagsplace;*urn:schemas-microsoft-com:office:smarttagsaddressV*urn:schemas-microsoft-com:office:smarttags PersonNamehttp://www.microsoft.com 81  ppppppppqqqqqqw$-_,a,ppppppppqqqqqqw33G]}~{p !&'/(O(, - ---"-)-2-6-8-G-I-N-S-_-h-----..P.T."122247p8q88!9:#=>DDD EHHHHHHHIIII I I I!I#I$I)I*I?I@ICIDIJIKIPIRI_I`IbIcIhIiIxIyIzI{IIIIIIIIIIIIJJ]]Pejee f f.f5h@hooppppppppqqqqqqqqqq qqqqqqqqqqqqqqrrr"r#r)r-r6r7rHrIrOrSr\r]rnrorrrrrrrrrrrrrssss0s1sDsEs\s]snsosssssssssssssss%t&t=t>tStTtmtnttttttttttttt uu!u"u5u6uIuJu]u^uquruuuuuuuuuuuvvvv3v4vGvHv[v\vovpvvvvvvvvvvkwlww{{u!!+,0066 D DDDIDIDiDiDjDjDIINONOZOeOzOOOOOO{P{PUUZZ____`aee f f.f.fFhMhppppppppqqqqqqwww yae2  1i#K2&%Xs.v`ai0S7 1Ei9ވdI!Pzn::cPp}eW|QP]5!tc^<,5TtK+bJ P^`P@@^@`.0^`0..``^``... ^` .... ^` ..... ^` ...... `^``....... 00^0`........hh^h`OJQJo(hH88^8`OJQJ^Jo(hHo^`OJQJo(hH  ^ `OJQJo(hH  ^ `OJQJ^Jo(hHoxx^x`OJQJo(hHHH^H`OJQJo(hH^`OJQJ^Jo(hHo^`OJQJo(hHhhh^h`OJQJo(hHh88^8`OJQJ^Jo(hHoh^`OJQJo(hHh  ^ `OJQJo(hHh  ^ `OJQJ^Jo(hHohxx^x`OJQJo(hHhHH^H`OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh ^`hH.h ^`hH.h pLp^p`LhH.h @ @ ^@ `hH.h ^`hH.h L^`LhH.h ^`hH.h ^`hH.h PLP^P`LhH.h ^`hH.h ^`hH.h pLp^p`LhH.h @ @ ^@ `hH.h ^`hH.h L^`LhH.h ^`hH.h ^`hH.h PLP^P`LhH.h hh^h`hH.h 88^8`hH.h L^`LhH.h   ^ `hH.h   ^ `hH.h xLx^x`LhH.h HH^H`hH.h ^`hH.h L^`LhH.h^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh ^`hH.h ^`hH.h pLp^p`LhH.h @ @ ^@ `hH.h ^`hH.h L^`LhH.h ^`hH.h ^`hH.h PLP^P`LhH.h hh^h`hH.h 88^8`hH.h L^`LhH.h   ^ `hH.h   ^ `hH.h xLx^x`LhH.h HH^H`hH.h ^`hH.h L^`LhH.h ^`o(hH.h^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh hh^h`hH.h 88^8`hH.h L^`LhH.h   ^ `hH.h   ^ `hH.h xLx^x`LhH.h HH^H`hH.h ^`hH.h L^`LhH. yaei9Tt07 1s.eW:cPI!P+b&%5!tc1i#ai0                                                                                                             aX = ^&= cB|II1N7dS= -DY=  ]x< c9cv?32%p&%WpE 1} V otR@SW[SLB. < J!@!t"#1#y^#+$WU%9'x6(+d+ ,x\-1.iK.1w12'4+G47b28H:@;|;h=5@6`@5QC|D?F-FIEF/G%HJKvLNBN/P(2Q=TFtT/UYZ%Z=6\\]l]p_T1_6`aaSbz iEi#5kJlllNWngnDqtuuDv Ny[} MEV?zIO_!'g={CMO[`um*N[}H*k?VL5b<K5&a t3+ gt'/nFr'YEn6hFI4wW]hF)f+\ [dZ>RR,4jf:s2WJ{WD!xa<9. E2TVX`L`sj$aR]k6O,,,,,,-!-"-)-2-6-8-;-<-=-C-G-I-L-M-N-S-V-Y-[-\-_-h-l-p-s-t-u-{-----------------------------.... ......$.(.,.-...4.8.<.@.A.B.G.L.P.S.T.0!1"1)101<1N1`1a1h1q1u1w1z1{1|111111111111111111111111111111111112222"2&2)2-2.2/25292<2@2A2B2G2K2O2R2S2V2_2d2h2l2m2n2t2x2|222222222222222333333!3"3#3)3,3.303132373:3=3?3@3C3L3O3Q3T3U3V3\3_3b3f3g3h3m3p3s3u3v3333333333333333333333333333344 4 4 444444 4%4*4.41424R7777777777777777777777777778 8 8888888"8&8'8(8-808385868@8A8H8O8V8c8p8q8x888888888888888888888888888888888888 9!9(9/9;9M9_9`9g9p9s9u9x9y9z999999999999999999999999999999999999: :::: :$:%:&:,:1:4:8:9:::?:C:G:J:K:N:W:[:_:c:d:e:k:o:s:w:x:y:~:::::<"=#=*=1===O=a=b=i=r=v=x={=|=}===================================>>>>">&>)>->.>/>5>9><>@>A>B>G>K>O>R>S>V>_>d>g>k>l>m>s>w>{>>>>>>>>>LDuDvDDDDDDDDDDDDDDDDDDDDDDE E EVHHHHHHHHHHII IIII!I$I*I0I1I:I@IDIKIQIRI\I`IcIiImInIuIyI{IIIIIIIIIIq q"q$q&q(q0q1q:q>qDqIqOqYqZqaqfqhqmqqquqvqxqzq|qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqr rrrrrrr r"r#r%r'r)r.r4r6r7r9r;r=rBrFrHrIrKrMrOrTrZr\r]r_rarcrhrlrnrorqrsrurzrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrsss s ssssssss$s*s0s1s3s5s7st@tBtDtItNtStTtVtXtZt_tetmtntptrtttytttttttttttttttttttttttttttttttttttttttttuu u uuuuuuu!u"u$u&u(u-u3u5u6u8u:u?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~Root Entry FpData ~1TablegWordDocumentSummaryInformation(DocumentSummaryInformation8CompObjq  FMicrosoft Office Word Document MSWordDocWord.Document.89q