Evaluating the Opportunity for DC Power in the Data Center - by Mark Murrill and B.J. Sonnenberg, Emerson Network Power
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
A White Paper from the Experts in Business-Critical Continuity™ Evaluating the Opportunity for DC Power in the Data Center by Mark Murrill and B.J. Sonnenberg, Emerson Network Power
Summary With data center managers struggling to increase efficiency while maintaining or improving availability, every system in the data center is being evaluated in terms of its impact on these two critical requirements. The power system has proven to be one of the more difficult systems to optimize because efficiency and availability are often in conflict; the most efficient approach to critical power is rarely the most reliable. One solution to power system optimization that deserves serious consideration is DC power. Since utility AC power must ultimately be converted to DC for use by IT system components and because stored energy systems (batteries, flywheel, etc.) provide DC power for backup, a DC power architecture requires fewer total conversions from grid to chip, creating the opportunity to reduce costs and increase efficiency. A data center-optimized, row-based DC power protection system is now available to help data center operators take advantage of that opportunity. This system, combined with the availability of 48V DC-powered IT equipment from major manufacturers, makes DC power an ideal solution for small and midsize data centers seeking to optimize efficiency, reliability and scalability. Other applications include high-density equipment rows with a consistent footprint and pod-based data centers. As the leading provider of AC and DC power systems, Emerson Network Power is uniquely positioned to help organizations evaluate the suitability of DC power and determine whether a row-based DC infrastructure is appropriate for a given application. 2
Introduction The first decade of the twenty-first concerns among data center managers as The industry responded with a new century was one of incredible growth they struggled to adapt to a 400 to 1,000 focus on energy efficiency and began and change for data centers. The demand percent increase in rack density. implementing server virtualization, higher- for computing and storage capacity efficiency server power supplies, and new The dramatic increase in data center exploded, and many IT organizations approaches to cooling. Yet, while signifi- energy consumption created both financial struggled to deploy servers fast enough cant progress has been made in some and environmental challenges. Energy to meet the needs of their businesses. At areas, the critical power system has yet to costs, which once had been relatively the same time, the trend to consolidate be fully optimized. While individual com- inconsequential to overall IT management, data centers and centralize computing ponents have been improved, the overall became more significant as the rise in resources resulted in fewer opportunities system complexity is high, which can consumption was exacerbated by a for planned downtime while also increas- create inefficiency and add operational steady—and in some years significant— ing the cost of unplanned outages. risk. Faced with the choice of increasing increase in the cost of electricity. In system efficiency or adding risk, many Data center operators were able to meet addition, increased awareness of the role continue to choose proven approaches the demand for increased compute that power generation plays in atmospheric that deliver high availability but do not capacity by deploying more powerful carbon dioxide levels prompted the U.S. deliver the highest efficiency. servers—often in the same physical space EPA to investigate large energy consumers as the servers being displaced—creating a such as data centers. In 2007 the EPA However, a close examination of the dramatic rise in data center power con- presented a report to the U.S. Congress that available options reveals that, in many sumption and density. Between 2004 and included recommendations for reducing cases, efficiency can be improved without 2009, power and heat density became top data center energy consumption. sacrificing overall availability. Established Data Center Power Distribution Options Traditional AC power distribution systems in North America bring 480V AC power into a UPS, where it is converted to DC to charge batteries, and then inverted back to AC. The power is then stepped down to 208V within the distribution system (PDU) for delivery to the IT equipment. The power supplies in the IT equipment convert the power back to DC and step it down to lower voltages that are consumed by processors, memory and storage [Figure 1]. Double Conversion UPS Server Bypass PDU PSU 480V AC 480V AC 208V AC AC DC 12V DC Rectifier Inverter Transformer Load DC DC Battery Figure 1.1. Figure Typical 480V480V Typical AC to 208V AC toAC208V data center power AC data systempower center configuration. system configuration. 3 Double Conversion UPS Server Bypass PDU PSU
In recent years, manufacturers have In the event of a power interruption, a First, by connecting loads phase-to-neutral, increased component efficiencies and switching function is required to bring the additional harmonic currents may be intro- developed new operating modes, such load back on to the inverter, and mains duced, offsetting some of the anticipated as the “eco-mode” available on some UPS need to be quickly isolated to prevent efficiency gains. Second, because of the systems, to improve efficiency. power feedback into the distribution elevated voltages at the rack, arc flash system. No matter how fast or reliably concerns for IT personnel working in the Figure 2 shows AC UPS operation when this switching occurs, there is still a chance rack will increase. Finally, there are no in eco-mode. Efficiency is improved by it will fail when it is needed most. commercially available power supplies at bypassing the conversion in the inverter; this input level. All in all, this architecture however, this introduces the potential for There has been discussion in the industry needs further development before it can reduced system reliability. Critical loads about simplifying the AC distribution be considered a true alternative. are no longer isolated from the mains or scheme by eliminating the step-down in the subject to the tight voltage regulation power distribution unit and delivering higher Consequently, 48V DC power may prove to Double Double Conversion Conversion UPS Server normally provided by the inverter, and UPS voltages to the IT equipment. One exam- Server be the best alternative to 480V/208V AC complex synchronization circuits are Bypass Bypass ple, illustrated in Figure 3, takes 277V AC power distribution for organizations seeking PDU PDU PSU PSU needed to ensure reliable power transfer. output from the UPS and delivers it directly to optimize reliability and efficiency today. 480V 480V AC AC Rectifier Inverter to the server power 480V AC 480V AC supply. This approach 208V AC 208V AC AC AC DC DC 12V 12V DC DC Load Rectifier Inverter Transformer Transformer Load DC DC DC DC may have potential in the future but is not viable today for a number of reasons. Battery Battery Figure Figure 1. 1. Typical Typical 480V 480V AC AC to to 208V 208V AC AC data data center center power power system system configuration. configuration. Double Double Conversion Conversion UPS UPS Server Server Bypass Bypass PDU PDU PSU PSU 480V AC 480V AC 480V AC 480V AC 208V AC 208V AC AC AC DC DC 12V DC 12V DC Rectifier Rectifier Inverter Inverter Transformer Transformer Load Load DC DC DC DC Battery Battery Figure Figure 2. 2.2.In eco-mode, incoming power power bypasses Figure InIneco-mode, incoming eco-mode, incoming powerthe bypasses the inverter inverter to bypasses to increase increase the UPS UPS system inverter system efficiency. efficiency. to increase UPS system efficiency. Double Conversion Double Conversion UPS UPS Server Server Bypass Bypass PDU PDU PSU PSU 480V AC 480V AC 480V/277V AC 480V/277V AC 277V AC 277V AC AC AC DC DC 12V DC 12V DC Rectifier Rectifier Inverter Inverter Load Load DC DC DC DC Battery Battery Figure Figure 3.3.480V Figure 3. 480V 480V AC AC277V AC to to to 277V 277V dataAC AC data AC data center center center power power power system system system configuration. configuration. configuration. Switch Switch 4 PSU PSU PDU PDU
480V AC Rectifier Inverter 480V AC Transformer 208V AC AC DC 12V DC Load Rectifier Inverter Transformer DC DC Load DC DC Battery Battery Figure 1. Typical 480V AC to 208V AC data center power system configuration. Figure 1. Typical 480V AC to 208V AC data center power system configuration. Double Double Conversion UPS Server A Practical Approach toConversion DC in the UPSData Center Server Bypass Bypass PDU PSU Historical Background Today, telephone central offices PDU -48V power supplies PSU at the point of use. The 48V DC 480V ACpower has a long history in tele- (exchanges) are 480Vstill AC powered by -48V DC. 208V AC power supplies AC step DC the voltage 12V DC down to 480V AC Rectifier Inverter 480V AC Transformer 208V AC AC DC 12V DC Load Rectifier Inverter These facilities exhibit levels of availability Transformer 12V and lower DC DC voltages DC for internal Load uses. communication networks. It is inherently DC DC simple and reliable with few conversion as high as 9-nines, as confirmed in a study It is easy to see the benefits of this Battery of 23,000 DC power systems over 10 years stages to the point of use. In AlexanderBattery approach when applied to the data center. Graham Bell’s day, 48V DC was chosen as by NTT Facilities1. Downstream of the AC/DC rectifiers, the the standard for two reasons: In most telecommunications Figure 2. In eco-mode, incoming power bypasses the inverter to increase UPS system efficiency. installations, the power is completely isolated from the Figure 2. In eco-mode, incoming power bypasses the inverter to increase UPS system efficiency. 1. DC power was felt to be more reliable 48V DC power system is deployed as three mains and is considered “safety extra low than AC because it could be directly distinct elements as shown in Figure 4: voltage” (SELV) per IEC/UL 60950 and can connected to backup Double batteries during 1. 480V AC to -48V DC modular be maintained live by trained personnel. Double Conversion Conversion UPSUPS Server Server grid outages; power system In addition, there is no need to derate Bypass capacity to account for phase balancing or 2. 48V was considered the optimal Bypass 2. Battery banks for extended PDUrun time PDU PSU PSU tradeoff between transmission harmonics, which are not a factor with DC 480V AC (at least 8 hours) 480V/277V AC 277V AC AC DCsafety 12V DC 480V AC distance and human Rectifier Rectifier safety because Inverter Inverter 480V/277V AC 277V power. AC Personnel AC DC is also 12V DC improved Load Load DC DC 3. Load distribution cabinets (BDFB/PDU) when DC servicing DC equipment in the rack it is considered safe to touch during maintenance or accidentalBattery exposure. These elements are connected with large because arc flash is not a concern at -48V. Battery copper bus bars and wires routed around However, unlike telecommunications cen- the facility to distribute power directly to tral offices, data centers are not designed Figure 3. 480V AC to 277V AC data center power system configuration. Figure 3. 480V AC to 277V AC data center power system configuration. with large copper bus bars to distribute DC power to racks. Fortunately, with new row-based DC topologies, these large Switch Switch copper runs are no longer necessary. In this row-based configuration, power is PSU PDU PSU converted from AC to DC very close to PDU 480V AC Rectifiers 48V DC 48V DC DC 12V DC the point of use, which decreases the 480V AC Rectifiers 48V DC 48V DC DC 12V DC Load (N+m) DC Load (N+m) DC conductor size and cost [see Figure 5]. Battery Data centers are entering a new stage of Battery maturity where the emphasis is on reliably delivering more computing capacity to Figure 4.4.-48VDC Figure -48V power as typically DC power implemented as typically in telecommunications implemented central offices. in telecommunications central offices. the organization at lower cost and with Figure 4. -48VDC power as typically implemented in telecommunications central offices. reduced power consumption. Row-based DC power represents a practical and Row-Based Row-Based DC DC UPS UPS Server Server affordable solution for reducing data center complexity, increasing efficiency PSU PSU and enabling growth. 480V AC Rectifiers 48V DC DC 12V DC 480V AC Rectifiers PDU 48V DC DC 12V DC Load (N+m) PDU DC Load (N+m) DC Battery Battery Figure 5. A row-based DC UPS minimizes the amount of copper and floor space required for installation in the data center. Figure 5. A row-based DC UPS minimizes the amount of copper and floor space required for installation in the data center. Figure 5. A row-based DC UPS eliminates the long copper runs and is optimized for the data center. 5
Factors Affecting Power Distribution Selection When selecting an optimal power architec- Intel concluded that DC power (both 400V and 48V) was ture for a particular facility, the number of factors data center designers and consistently more efficient than AC at all voltages. managers have to consider is substantial. Understanding the impact of each factor While each study created scenarios that Power offers an efficiency-optimizing is essential to making an informed choice allowed for direct comparison of different feature that maintains near-peak effi- for your data center. configurations, many aspects of the power ciencies at system loads as low as 5%, distribution system are complex and vary allowing the system to achieve both Efficiency by application, making general compari- high availability and efficiency in real End-to-end power system efficiency sons difficult. Aspects which were not world conditions. can be difficult and time consuming to considered in either study include the Harmonics accurately calculate due to the complexity degree of desired redundancy, the impact Harmonics are a distortion of the of variables involved, but it is still useful to of harmonics and load variations, and the normal AC waveform generally trans- consider and approximate efficiency based added load on cooling created by inefficient mitted by non-linear loads. In the on specific site conditions and priorities. systems. In cases where these factors are data center, the switch-mode power In light of recent improvements in the present, the gaps between DC and AC supplies used in AC servers represent efficiency of both AC and DC architectures, would likely be wider, as the following a non-linear load that can create two vendor-neutral studies have been analysis reveals. harmonics. Harmonic currents accu- published by The Green Grid and Intel Redundancy mulate in the neutral wire, causing to provide an objective, yet simplified, The degree of redundancy in the distribution losses and increased heat comparison of the efficiency of various power distribution system can generation. If the cumulative level of power system configurations. impact both availability and harmonics—known as total harmonic The results from The Green Grid’s study, efficiency. In the case of AC power distortion—becomes too high, damage “Quantitative Efficiency Analysis of Power distribution, these two factors may to sensitive electronics and reduced Distribution Configurations for Data be mutually exclusive—greater efficiency can result and may require Centers”2, for the power architectures redundancy increases availability equipment derating to overcome. discussed here, evaluated at 50 percent but lowers overall efficiency. These losses are difficult to predict in operating load, are summarized below: complex AC distribution systems, but The highest availability is achieved could be quite significant. Harmonics 480V AC to 208V AC (legacy): through 2N redundancy in which are not present in DC systems because 63 percent efficient [Figure 1] each UPS module is designed to run there are no waveforms to contend with. at no more than 50 percent load. 480V AC to 208V AC (current): However, since the data center is Data Center Load Variations 88 percent efficient [Figure 1] rarely operating at peak capacities, Many data centers are operating at 480V AC to 277V AC: significantly less than 100 percent of the redundant systems are almost 89 percent efficient [Figure 3] design load much of the time, with always operating at less than 50 480V AC to 48V DC: percent capacity. With derating, real-time loads changing frequently. 90 percent efficient [Figures 4 & 5] maximum capacity may be closer to This variation further complicates 40 percent, and, in normal operation, efficiency modeling. As servers are Intel conducted a similar analysis, each unit may operate at 20 percent turned on/off and utilized at different “Evaluation of 400V DC Distribution in load or less. Most equipment rates, the loading on each phase of Telco and Data Centers to Improve Energy efficiency curves drop dramatically a 3-phase power system changes, Efficiency”3 and found that the highest below 30 percent load, resulting in making alignment between the three overall system efficiencies are achieved actual operating efficiencies signifi- phases difficult. Unbalanced loads with a 400V DC system; however, this tech- cantly less than ideal conditions. lower total system efficiency and nology is not yet commercially available. produce additional heat. Overall, this study concluded that DC In the case of DC power, redundancy power (both 400V and 48V) was consis- can be seamlessly integrated into the tently more efficient than AC at all voltages. DC UPS. In addition, Emerson Network 6
In the white paper “Phase Balancing: Availability With a complete data center power and The Last Few Inches of a High- Both AC and DC power systems can be cooling infrastructure integrated into a fully Efficiency Power System”4, Server designed to achieve high availability. The enclosed container, these systems can be Technology analyzes the impact of highest availability facilities will likely con- delivered to a site and quickly plugged into unbalanced loads on a 30A, 240/415V tinue to use 2N redundancy because of its power, communications and chilled water 3-phase circuit loaded in a Wye ability to support a dual-bus architecture systems to enable additional computing configuration. If the load is balanced, that can eliminate downtime from system capacity. With its built-in redundancy and the current through each input phase failures across the entire power distribution compact footprint, a DC UPS is an excellent is 8A, and losses can be calculated at chain. However, that availability comes solution for these applications. 19 watts per 100 feet of cable. In with a cost, both in terms of upfront This same philosophy is also manifesting the most unbalanced case, current equipment and ongoing operating itself in the move toward row-based equip- through one phase is 24A and losses efficiencies. Many organizations can ment, which enables modular expansion escalate to 115 watts per 100 feet of achieve desired levels of availability from of existing facilities or support for high cable. This shows that power loss and N +1 redundancy in which redundancy is density rows. Both AC and DC UPS systems heat generated in feeder cables can maintained at the module level. provide scalability in a row-based format, increase by as much as 600 percent At the system level, there are two ways that but the DC UPS has the benefit of built-in because of unbalanced loads. DC power systems offer high availability. redundancy and will occupy at least 50 Additionally, keeping track of which First, the DC power conversion system has percent less floor space than a comparable base loads are on which phase can be fewer components than a comparable AC row-based AC system. time consuming and tedious depend- ing on the sophistication of the site system, which contributes to a higher Cost and the tools available. Load balancing mean time between failure (MTBF) rate and more uptime. In addition, the DC UPS uses The main purpose of the critical power issues are also specific to three-phase an array of discrete rectifiers to deliver con- system is to eliminate power-related AC power and are not a concern ditioned, isolated power to a distribution downtime; any cost comparison should with DC. bus. These rectifiers provide built-in redun- consider the level of availability required Power-Related Heat Loads and the cost of downtime. In general, row- dancy; the system can accommodate the After the IT equipment itself, the cool- based DC power applications will be less failure of any individual rectifier without ing system is the next largest user of expensive to install, operate and maintain immediately affecting performance or power in the data center. Not only is than a comparable AC system and will capacity. Individual rectifiers can be safely cooling required for the heat gener- support redundancy levels of at least N + 1. hot-swapped out in the field without ated from the IT equipment, but it Some savings may be offset by increased IT impacting system operation, thus minimiz- also must offset the heat generated equipment costs in a DC system—list prices ing system mean-time-to-repair (MTTR), by power conversion losses. With for servers with DC power supplies can be a major contributor to unavailability. fewer required conversions and up to 10 percent higher than the more greater overall efficiencies, a DC Scalability common AC servers. These costs may be power system generates less heat With equipment and rack density rising negotiable and should come down with than an AC system, reducing data steadily, the power system can become a the higher volumes that accompany center cooling energy consumption. constraint to growth. As a result, modular increased adoption. Overall, a DC-based As a rule of thumb, each watt of heat approaches to data center design and end-to-end power architecture can be as generation removed from the data expansion are gaining in popularity as much as 30 percent less expensive than center leads to an additional 1.4 to 2 demonstrated by the trend toward an AC system, depending on many of the watts saved in cooling. “containerized” data centers. factors above. 7
NetSure™ ITM Row-Based DC UPS The NetSure™ ITM is a data center-optimized row-based 48V DC UPS from Emerson Network Power. It is designed to support the move to higher efficiency, density and flexibility enabled by row-based power and cooling. Instead of deploying one large DC power plant, which would require DC power to be distributed across the facility, 480V AC power is delivered directly to the NetSure™ ITM, which then provides protected power to racks of 48V equipment in close proximity [Figure 6]. When additional racks of IT and commu- nications equipment are required for data center growth, new DC UPS units can be added without disruption. The NetSure™ ITM integrates power conversion, battery backup and branch distribution into a compact, highly reliable Figure 6. The NetSure™ ITM is positioned within the row to provide power to multiple racks. power protection system. (12) individual AC/DC power conversion units (PCUs) deliver 70kW per module of conditioned, isolated 48V power to an internal distribu- This DC UPS comes pre-assembled and tion bus that charges the batteries and factory tested to simplify installation. The distributes power directly to the IT or com- only field connections required are the AC munications equipment. Backup time is input to the module and the output to DC 5-10 minutes at full load with the included loads. In addition, individual components 10 year-VRLA batteries; traditional DC are user replaceable and can be swapped systems may be more appropriate if more out while the system is operating, allowing than 60 minutes of backup is required. live maintenance without bypass or generator switchover. The system provides up to 22 branch circuits protected by 100A to 200A circuit The NetSure™ ITM features an integrated breakers. Each branch is monitored in the control and monitoring system that pro- controller for load and trip condition and is vides data on input and output parameters designed to support varying rack densities. (including branch circuit loads), as well Power distribution in the rack is provided as advanced Albér battery management. by NetSure™ RDB Series zero-U distribution An Energy Optimization Mode featuring Figure 7. A zero-U power distribution unit provides units that feature plug-and-play connectors Intelligent Power Matching ensures near- power to 48V input loads. for easy installation and tool-less peak efficiency for loads down to 5 percent maintenance [Figure 7]. 8
Conclusion References by keeping a minimum number of PCUs Direct current is already a fundamental [1] “Reliability Field Data of near full load and putting the rest on part of your IT infrastructure: critical loads Power Systems.” NTT Facilities. standby with the ability to respond quickly consume DC power and all backup sources INTELEC 2007. to changing load conditions. generate it. Furthermore, the electric grid [2] “Quantitative Efficiency Analysis of distributes AC power. So, the question Power Distribution Configurations for Because it eliminates the need for large becomes, where is the optimal point at Data Centers.” The Green Grid. 2008. copper runs and offers the reliability and which to convert AC to DC power while scalability inherent in DC power, the [3] Pratt, Annabelle et al. “Evaluation of providing suitable protection from outages? NetSure™ ITM is ideal for a variety of 400 VDC Distribution in Telco and If the conversion occurs too early, data center applications, including: Data Centers to Improve Energy DC power must be transported long Sites with an existing constrained distances, which requires large conductors Efficiency.” Intel. 2007. power infrastructure or problems to reduce losses. If it occurs too late, addi- [4] “Phase Balancing: The Last Few Inches with harmonics tional conversions are introduced into the of a High-Efficiency Power System.” Data centers seeking to optimize process which may compromise efficiency Server Technology. 2010. efficiency and availability while and reliability and increase costs. establishing a path for growth by In many applications, the ideal point for using DC power in concert with power conversion and energy storage is as existing AC systems close as possible to the load. A row-based, Modular or pre-configured data 48V DC UPS allows for optimal efficiency, center layouts reliability and flexibility in these data High-density scalable blade deployments center environments. The NetSure™ ITM from Emerson Network Power, combined with the availability of DC-powered IT equipment from major manufacturers, makes 48V DC power a practical, efficient and cost-effective solution for managing the conversion from AC grid power to DC power required by IT equipment. The simplicity, efficiency and reliability inherent in the NetSure™ ITM system can reduce complexity and costs in a variety of applications and should be seriously considered for mid-size data centers and facilities with increasing capacity needs. 9
Emerson (NYSE:EMR), based in St. Louis, Missouri (USA), is a global leader in bringing technology and engineering together to provide innovative solutions for customers in industrial, commercial, and consumer markets through its network power, process management, industrial automation, climate technologies, and appliance and tools businesses. For more information, visit: Emerson.com. Emerson Network Power, a business of Emerson (NYSE:EMR), is the global leader in enabling Business-Critical Continuity™ from grid to chip for telecommunication networks, data centers, health care and industrial facilities. Emerson Network Power provides innovative solutions and expertise in areas including AC and DC power, precision cooling, embedded computing and power, integrated racks and enclosures, power switching and controls, infrastructure management, and connectivity. All solutions are supported globally by local Emerson Network Power service technicians. For more information on Emerson Network Power’s full suite of solutions specifically supporting the communications network infrastructure, visit: EmersonNetworkPower.com/EnergySystems. Learn more about the NetSure™ ITM DC UPS from Emerson at: EmersonNetworkPower.com/NetSureITM. This publication is issued to provide outline information only which (unless agreed by Emerson Network Power Energy Systems, North America, Inc. in writing) may not be used, applied or reproduced for any purpose or form part of any order or contract or be regarded as a representation relating to the products or services concerned. Emerson Network Power Energy Systems, North America, Inc. reserves the right to alter without notice the specification, design or conditions of supply of any product or service. Emerson®, Emerson Network Power™, Business-Critical Continuity™ and NetSure™ are trademarks of Emerson Electric Co. and/or one of its subsidiaries. Additionally, this brochure references the following company trademarks: NTT Facilities, The Green Grid, Intel and Server Technologies. Names of companies and products are trademarks or registered trademarks of the respective companies. Any questions regarding usage of trademark names should be directed to the original manufacturer. Emerson Network Power Energy Systems, North America 4350 Weaver Parkway, Warrenville, IL 60555 Toll Free: 800-800-1280 (USA and Canada) Telephone: 440-246-6999 Fax: 440-246-4876 Web: EmersonNetworkPower.com/EnergySystems Emerson Network Power. The global leader in enabling Business-Critical Continuity™. EmersonNetworkPower.com AC Power Embedded Computing Outside Plant Racks & Integrated Cabinets Connectivity Embedded Power Power Switching & Controls Services DC Power Infrastructure Management & Monitoring Precision Cooling Surge Protection © 2010 Emerson Network Power Energy Systems, North America, Inc. All rights reserved. Code: 124W-DCDATA / 1010
You can also read