A cloud based infrastructure is actually an ecosystem for your IT needs. It contains numerous components starting from the base hardware to the cloud software and then the analytics and monitoring tools. All these are integrated well and work in a pluggable architecture to provide you with capabilities for fault-proof computations, data storage and access, delivering services, development and deployment of your products etc. A good cloud infrastructure needs to have the crucial features of high availability, horizontal scalability in which your enhanced and growing IT and business requirements are met by adding up more logical resources rather than adding separate physical systems including hardware and computational platforms. Here is where the popular open source cloud softwareOpenStack helps you in achieving your goals. OpenStack helps in building your IT environment as Infrastructure as a Service (IaaS) that runs on commodity hardware and a number of component services on top of it that provides services for compute, storage, networking, dashboard etc. This service segregation into components is the key to achieving horizontal scalability with OpenStack.
Automation and Deployment Management for Cloud
Much of cloud’s as well as OpenStack’s effectiveness lies in the degree of automation of various phases of the cloud like installation, setup, deployment, configuration, management, scaling, troubleshooting etc. While OpenStack provides you with the foundation for this automation, its co-projects as well as numerous projects – open source as well as enterprise – provides tools for achieving this automation. It is upto you to select and use the right tool for this automation, and this decides how much you have tapped into the power of cloud automation and scaling and thus saving costly effort and man power. An automated deployment management system handles every aspect of cloud administration like creating virtual instances, starting and stopping them, resource provisioning, configuration management, integration management, DNS configuration and overall system administration with little human intervention. We will examine some of the best and effective tools that help you in achieving this automation, deployment and configuration management with clouds like OpenStack.
Fuel – the OpenStack Deployment and Management Tool
It is a GUI based tool for deploying and managing the OpenStack cloud along with its components. It accelerates the complex deployment and testing process by deploying and running various configuration flavors of OpenStack. It handles the life cycle processes of the OpenStack cloud scale up/down, configuration management and plugin management. Fuel performs the following operations as part of the cloud deployment, testing and configuration process.
1. Discovers hardware, both bare metal and virtual nodes that can be configured to boot from network.
2. Allows for configuring the hardware through the GUI.
3. Provides capability to spawn and manage multiple OpenStack clusters.
4. Performs pre-deployment check and network validations.
5. Performs post-deployment checks and tests for validating the deployed OpenStack cloud.
6. Provides access to logs in real time through the GUI.
The power of Fuel lies in both the intuitive GUI it provides for the setup, testing, validation, deployment and configuration of the OpenStack cloud as well as its capability to perform the complex deployment related processes like resource and network detection, pre and post deployment testing and validation, configuration testing etc. The step-by-step wizard provides an array of configuration options that user can select, some of which are:
1. The host OS.
2. The hypervisor.
3. Storage back end.
4. Network topology.
5. Controller configuration that best suits the HA (high availability) requirements of the cloud.
On a global cloud level, Fuel assists in creating and deleting nodes, assign roles and accesses to nodes, replicate nodes, create and manage volumes and partition hard disks that span across the multiple physical hosts in the cloud and provides configuration and network templates to select the best suited one.
More about Fuel: https://www.fuel-infra.org
Mistral – the OpenStack Workflow Service
Each process associated with a cloud platform workflow is identified as a task and a set of tasks define a process. In a distributed cloud environment these multiple distinct processes should be interconnected and executed in a particular order. Mistral provides the mechanism to define these tasks and their relationships called the workflows and then execute them to create the cloud workflow. The typical workflow processes Mistral handles are: state management, synchronization, parallelization, high availability and task scheduling. Mistral also helps to enhance fault-tolerance by which the user can split the business tasks as parallel processes to be executed on multiple nodes synchronously. Mistral coordinates such that if any one node crashes, then the partially completed job is continued at another node at the same check point before the crash actually occurred.
Mistral can be used by a user or a client software or framework to specify cloud deployment workflows that includes creation of multiple VMs and the associated resources and components. Mistral can be used to schedule running of tasks that are essential for the cloud workflow. These are tasks like local processes (shell scripts or binaries) on the virtual instances, virtual instance management tasks like creation, starting, stopping and terminating, batching of tasks to define specific workflows and executing them as per a schedule using parallel execution etc.
Mistral can be used for data crawling that aids in Big Data analysis. It also works in live VM migration when a particular VM exceeds its maximum limit of CPU usage which is triggered by the Ceilometer OpenStack component that measures resource usage.
More about Mistral: https://docs.openstack.org/mistral/latest/
Compass – Automating Deployment and Management of OpenStack
It helps in bootstrapping the server pool associated with any cloud platform including OpenStack from the bare metal nodes. Compass and its integrated plugins assist in discovering hardware, deploying OS and hypervisor and also provide configuration management. Compass helps in the following areas of OpenStack cloud deployment and management:
1. Assists in infrastructure bootstrapping process and also offers programmability for operators to this.
2. Allows for implementing different configuration flavors through meta-data.
3. Implements extensibility through the integration of a number of tools like Chef and Ansible for OpenStack cluster configuration. By default Ansible is used for OpenStack installation. The Compass core blends with other tools for resource discovery, OS provisioning and Package deployment.
More about Compass: http://www.syscompass.org/install.html
Ansible – the OpenStack Configuration and Orchestration Tool
Ansible provides automation capabilities for the provisioning, configuration, intra-service orchestration and deployment of applications on cloud. It provides playbooks and roles for performing various deployment and configuration tasks in an OpenStack cloud. Ansible playbooks are a way to describe the automation jobs in YAML, also called Orchestration of the cloud. Ansible provides a control language that uses modules or routines to perform the intended tasks on nodes and the user can control this through the Shell CLI. It uses SSH communication for high security. Ansible provides the following capabilities:
1. Parallel task execution
2. Orchestration through Playbooks.
3. Dynamic building of configuration files.
4. Provides a simple language for overall automation
More about Ansible: https://www.ansible.com/how-ansible-works
Chef – Integration Framework for Cloud Infrastructure Automation
It is an open source cloud integration framework for automating cloud deployment. Chef enables programmers to create the machines and other cloud infrastructure components programmatically. Thus it transforms infrastructure into code. Regardless of the number of servers, Chef helps to manage this entire infrastructure by turning it into code. It provides configuration management and other infrastructure management tasks using Recipes, which are reusable definitions for automating infrastructure tasks. Chef has the following components:
1. Chef cookbooks are units of configuration and policy distribution. That is; the code that describes the desired state of the infrastructure.
2. A Chef Server is a central repository for cookbooks and for every node it manages.
3. A Chef Client runs on each node in the cloud and communicates with the Chef server to get the latest configuration description. This is used as instructions for bringing that node to the desired state.
4. Chef DK (Development Kit) provides the tools needed to develop the infrastructure automation code, directly from our workstation.
More about Chef: https://docs.chef.io/chef_overview.html
Puppet – a State based Configuration Management System
Puppet is a configuration management system that allows users to define the state of the IT infrastructure and then facilitates for automatic and continuous enforcement of the correct state. It can handle thousands of physical machines as well as logical VMs. Puppet also allows defining the infrastructure as a code. With this approach, multiple teams can collaborate on the cloud infrastructure development process through agile methodologies; utilize automated testing and continuous delivery. Thus, through configuration management, Puppet provides standardization in cloud management.
More about Puppet: https://puppet.com