[email protected] +44 20 8123 2220 (UK) +1 732 587 5005 (US) Contact Us | FAQ |

Sea Change Series: Orchestration Software in the Mega Data Center

April 2017 | 115 pages | ID: S68B597D1CAEN
WinterGreen Research

US$ 4,200.00

E-mail Delivery (PDF)

Download PDF Leaflet

Accepted cards
Wire Transfer
Checkout Later
Need Help? Ask a Question
The mega data center described in the study is effective because it leverages the economies of scale. This orchestration software infrastructure study module is part of a longer study that addresses the business issues connected with data center modernization.

There are 26 module parts to the larger study comprised of detailed analysis of how new infrastructure layers will work to support management of vast quantities of data. Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth.

We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process. The Internet has grown by a factor of 100 over the past 10 years. To accommodate that growth, mega data centers have evolved to provide processing at scale.

Facebook for one, has increased the corporate data center compute capacity by a factor of 1,000, virtually eliminating much manual process. Orchestration software is a key aspect of that process. To meet future demands on the Internet over the next 10 years, companies with that capacity need to increase capacity by the same amount again while the other companies struggle to catch up.

Nobody really knows how to get to increasing compute capacity by another factor of 1,000. Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input.

Mega data centers change that pattern of IT manual process. Realigning data center cost structures is a core job of orchestration software. The enterprise data centers and many cloud infrastructure operations all have similar problems of being mired in administrative expense. Containers address that issue by creating vastly more efficient operations for data center infrastructure.

According to Susan Eustis, lead author of the team that prepared the study, “The only way to realign cost structure is to automate infrastructure management and orchestration. Mega data centers automate server and connectivity management using orchestration software to manage multiple application containers. Other systems automate switching and storage, along with hypervisor, operating system, and virtual machine provisioning.

“As IT relies more on virtualization and cloud mega data center computing, the physical infrastructure is flexible and agile enough to support the virtual infrastructure. Comprehensive infrastructure management and orchestration is essential.

The Enterprise Data Center has become a bottleneck, it needs to be completely replaced. Category 5 and Category 6 Ethernet cable is spread throughout the existing enterprise data centers and is too slow to handle all the digital data coming through the data center. Cat 5 and Cat 6 Ethernet utilized by the servers to achieve data transport using that cable does not keep up with the data coming through the data center the way optical cable and optical transceivers do.

The existing servers and cable are a problem because they are too slow for modern systems. The cable is too slow to handle all the data coming at us in the new digital age, and the associated technology that operates at Ethernet category 5 and category 6 cable speeds is too slow as well, this is why the entire set of existing enterprise data centers is a bottleneck. Mobile data traffic is set to increase by a factor of eight between 2015 and 2020. Growth is anticipated at 53 percent per year, faster than systems revenue or industry revenue.

The theme of this study is that the pace of data expansion creates the need for more modern means of managing data. There are some companies that are doing a better job, better than others of adapting to IT infrastructure to the wild influx of data.

The four superstar companies that are able to leverage IT to achieve growth, Microsoft, Google, Facebook, and the leader AWS all use Clos architecture. What is significant is that systems have to hit a certain scale before Clos networks work Clos networks are what work now for flexibility and supporting innovation in an affordable manner. There is no dipping your toe in to try the system to see if it will work, it will not and then the IT says, “We tried that, we failed,” but what the executive needs to understand is that scale matters. A little mega data center does not exist. Only scale works.

Many companies are using digital technology to create market disruption. Amazon, Uber, Google, IBM, and Microsoft represent companies using effective strategic positioning that protects the security of the data. As entire industries shift to the digital world, once buoyant companies are threatened with disappearing. A digital transformation represents an approach that enables organizations to drive changes in their business models and ecosystems leveraging cloud computing, and not just hyperscale systems but leveraging mega data centers. Just as robots make work more automated, so also cloud based communications systems implement the IoT digital connectivity transformation.

WinterGreen Research is an independent research organization funded by the sale of market research studies all over the world and by the implementation of ROI models that are used to calculate the total cost of ownership of equipment, services, and software. The company has 35 distributors worldwide, including Global Information Info Shop, Market Research.com, Research and Markets, electronics.ca, and Thompson Financial. It conducts its business with integrity.

The increasingly global nature of science, technology and engineering is a reflection of the implementation of the globally integrated enterprise. Customers trust wintergreen research to work alongside them to ensure the success of the participation in a particular market segment. WinterGreen Research supports various market segment programs; provides trusted technical services to the marketing departments. It carries out accurate market share and forecast analysis services for a range of commercial and government customers globally. These are all vital market research support solutions requiring trust and integrity.
SEA CHANGE SERIES: ORCHESTRATION SOFTWARE IN THE MEGA DATA CENTER

Sea Change Series: Orchestration Software in the Mega Data Center, Amazon, Google, Microsoft, Facebook
Aim to Realign IT Cost Structure
Internet Has Grown by a Factor of 100 Over The Past 10 Years
Table of Contents
Orchestration Software Automates Data Center Infrastructure
Software Containers
Orchestration Schedulers Manage Containers
Orchestration Software Supports Container Automation
Realigning Data Center Cost Structures
IT Relies On Replacing Virtual Machine: VM Virtualization
Microservices
Microservices Features
Microservices Modules
Difficulties with Virtual Machines
Hypervisor a Difficulty
Virtual Machines Use Bare Metal, Containers Use Orchestration Software
Bare Metal an Inefficient Use of Compute Resource
Bare Metal Less Efficient
Industry Uses Robots Because Manual Labor Is Slow And Error Prone
IT processes Replace Manual Labor
Mega Data Center Orchestration Software
Large Fabric Network Not The Kind Of Environment That Can Be Realistically Configured And Operated In A Manual Way32
To Automate the Data Center Fabric
Value of Data Center Fabric
Google Shift from Bare Metal To Container Controllers
Google Container Controller Shift From Bare Metal In A Mixed Workload And In Nested Compute Units
Google Kubernetes Groups Software Containers
Fabric Services Inside A Container
Architecting Microsoft Cloud
Microsoft Managed Clustering and Container Management: Docker and Mesos
Microsoft Azure Service Fabric
Microsoft Is Seen As The Overall Winner In The Move To Application Containers
Microsoft Dublin Cloud 2.0 Mega Data Center
Microsoft Data Center, Dublin, 550,000 Square Feet
Microsoft Dublin Center Operates at a Power Usage Effectiveness (PUE) of 1.25.
Microsoft Data Center Container Area in Chicago.
Microsoft
Advantages of Using Containers
Orchestration Software Used to Create Containers
Disadvantages of Using Containers
Advantages of Virtual Machines
Container Orchestration
Use of Containers Eliminate Manual Process
IT Pros Increasingly Turn to Chef and Puppet
Hardware Containers Do Not Scale
Facebook Data Center Positioning
IBM Data Center Orchestration Software Automates Application Integration
Docker Orchestration & Docker Swarm
Docker Container Platform
Common Feature Sets For Orchestration Tools
Not all Orchestrators Are Created Equal
AWS Cloud Container Adoption Criteria
AWS Cloud Adoption Methodology
AWS Cloud Adoption Framework
AWS Market Leader In Cloud Computing
Facebook Fabric and Node are Core Structures Leveraging Software Orchestration
Apache Mesos Orchestration Software
Google Kubernetes Container
Google Container Builder Step Toward Building Pluggable Components in a Pipeline
Google Programmable Access To Network Stack
Google Andromeda Software Defined Networking (SDN)-
Google Compute Engine Load Balancing
Google Compute Engine Load Balanced Requests Architecture
Google Scaling Of The Compute Engine Load Balancing
Google Compute Engine (GCE) TCP Stream Performance Improvements
Google Cloud Platform TCP Andromeda Throughput Advantages
Google Open Sourced Its Container Management System Called Kubernetes
Facebook
Ability To Move Fast And Support Rapid Growth At The Core Of Facebook Infrastructure Design Philosophy
The Right Type of Cloud: Mega Data Centers
AWS Has Been Able To Adapt To Change
Manual Labor Is Slow And Error Prone
Mega Data Center Orchestration Software
Amazon, Google, Microsoft, Facebook
Cloud 2.0 Mega Data Center Fabric Implementation
Fabric and Node are Core Structures Leveraging Software Orchestration
Multi-Threading, Dynamic Systems
Oracle Multi-Threading Mega Data Center
Orchestration Tools Manage A Cluster As A Single Deployment
Microservice Monitoring with Google Kubernetes
Docker Container
Cluster Functions and Pod Benefits
Mesosphere DC/OS an Open-Source Project Built on Apache Mesos
Mesosphere Enterprise DC/OS Orchestration Software
Mesosphere DC/OS Production Containers Uses
Mesosphere DC/OS Orchestration Software
Mesosphere DC/OS Extending Capabilities Within Container Orchestration
Mesosphere DC/OS Certification Compliance
Mesosphere Market Leadership Position
Mesosphere DC/OS Runs Data Services on One Single Platform
Cloud Computing Not Enough: Entire Warehouse Building As A Single Mega Data Center System
Red Hat Ansible
Red Hat Ansible Architecture, Agents, And Security
Red Hat Ansible Advanced Features
Red Hat / Ansible
Red Hat Ansible Tower 3 Job Run Metrics
Cisco Integrated Infrastructure Management
Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
Mesosphere DC/OS: Mesos Features
Heart of DC/OS: Apache Mesos
DC/OS Implements Containers

LIST OF FIGURES

Figure 1. Slow Growth Mode of Companies with Enterprise Data Centers
Figure 2. Mega Data Center Fabric Implementation
Figure 3. Business Innovation and Technology
Figure 4. Docker Orchestration Software Creates Containers
Figure 5. Docker Compose
Figure 6. Mesosphere Marathon
Figure 7. Google Kubernetes
Figure 8. Orchestration Software Supports Container Automation
Figure 9. Orchestration Software Decreases Data Center Cost Structure
Figure 10. Files Bundled into a Container
Figure 11. Microservices: Suite Of Independently Deployable Service Modules with a Unique Process And Well-Defined, Lightweight Communication Portal: Mechanism To Serve A Business Goal
Figure 12. Microservices Distinct Features: Taxi Hailing Example
Figure 13. Microservices Market Segments
Figure 14. Microservices Modules
Figure 15. Hypervisor Virtualization Operating System Interface
Figure 16. Hypervisor Virtualization Operating System Interface
Figure 17. Virtual Machines Less Efficient Than Containers
Figure 18. Difference Between Virtual Machines and Containers
Figure 19. Bare Metal Management Replaced by Container Controllers
Figure 20. Containers vs. VMs
Figure 21. Industrial Robots Eliminate Manual Labor
Figure 22. Industry Uses Robots To Replace Manual Labor
Figure 23. Data Centers Need The Precision and Automation Similar to that Provided by Multi-Step Sequential Task Industrial Robots
Figure 24. Mega Data Center Orchestration Software
Figure 25. Single-Fabric Data Center Network Architecture
Figure 26. Bare Metal Presents a Lot of Extra Parameters and Metrics, Significantly More than With Containers
Figure 27. Nested Compute Units
Figure 28. Kubernetes Orchestration Software Groups Containers That Make Up An Application Into Logical Units
Figure 29. Kubernetes Orchestration Software Functions
Figure 30. Container Features as it integrates with the Service Fabric Runtime
Figure 31. Microsoft Setting Up A Secure Service Fabric Cluster in Azure using the Azure Portal.
Figure 32. Microsoft Data Center, Dublin, 550,000 Sf
Figure 33. Container Area In The Microsoft Data Center In Chicago
Figure 34. Microsoft Cloud Network Features
Figure 35. Like Physical Containers on a Ship, Software Containers Bring Many Servers Densely Packed
Figure 36. Advantages of Using Containers
Figure 37. Software Orchestration Container Challenges
Figure 38. Manual Process
Figure 39. Containers Need Orchestration Software
Figure 40. Virtual Machine Data Center Management Tasks:
Figure 41. FaceBook Open Compute Project
Figure 42. Facebook Data Center Modernization Functions
Figure 43. Facebook Altoona Iowa Cloud 2.0 Mega Data Center
Figure 44. Manual Process for Application Integration Deployment
Figure 45. Feature Sets For Orchestration Tools
Figure 46. Issues for Orchestration Software
Figure 47. AWS Cloud Container Adoption Criteria
Figure 48. AWS Cloud Container
Figure 49. AWS Cloud Adoption Framework
Figure 50. AWS Market Leader In Cloud Computing
Figure 51. Description of the Orchestration Software
Figure 52. Advantages of Using the Container Builder Cloud Architecture as a Service:
Figure 53. Google Andromeda Cloud High-Level Architecture
Figure 54. Google Andromeda Software Defined Networking (SDN)- Based Substrate Functions
Figure 55. Google Andromeda Performance Factors Of The Underlying Network
Figure 56. Google Compute Engine Load Balanced Requests Architecture
Figure 57. Google Compute Engine Load Balancing
Figure 58. Google Cloud Platform TCP Andromeda Throughput Advantages
Figure 59. IoT: Open Source IoT High Level Platform, OpenStack and Kubernetes
Figure 60. Facebook DuPont Fabros Technology Ashburn, VA Data Center
Figure 61. Cloud 2.0 Mega Data Centers Support 1.5 Billion Facebook Users Worldwide.
Figure 62. AWS Market Leader In Cloud Computing
Figure 63. Data Centers Need The Precision and Automation Provided by Multi-Step Sequential Task Industrial Robots
Figure 64. Mega Data Center Orchestration Software Functions
Figure 65. Multiple Pathways Open To Processing Nodes In The Cloud 2.0 Mega Data Center Functions
Figure 66. Dynamic Load Balancing
Figure 67. Mesosphere Customer References
Figure 68. Mesosphere DC/OS Certification Compliance
Figure 69. Cloud Is Not Enough
Figure 70. Red hat Ansible Playbook Language Advanced Features
Figure 71. Red Hat Ansible Tower 3 Job Run Metrics Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
Figure 72. Cisco UCS Helps Manage Administrative Costs And Reign In Complexity
Figure 73. Cisco UCS Director Delivers Comprehensive Infrastructure Management and Orchestration107
Figure 74. Mesosphere DC/OS: Mesos Features:
Figure 75. Native Mesos Containerizer Functions


More Publications