Deploy OpenStack with Chef, Part 0: Basic Concepts & Planning Environment

In this series of post I will demonstrate deploying OpenStack with Chef, using Rackspace Private Cloud’s cookbooks. This guide largely follows Rackspace’s documentation (Getting Started Guide, list of documentation), with notes by myself.

One thing to clarify is that Chef doens’t do the bootstrapping of nodes for you. I misunderstood this at first. You need to install and configure the OS, networking, network bridges required by Neutron manually.

My setup will be

  1. controller node: x.y.54.50, 172.17.1.1
  2. compute node: x.y.54.51, 172.17.1.2
  3. network node: x.y.54.54, 172.17.1.5, runs Neutron, with OVS plugin and GRE tenant network
  4. storage node: x.y.54.53, 172.17.1.4, runs Cinder

Each node has 2 interfaces, first one (not necessarily eth0, since there will be br-ex on network node) connects to public network x.y.54.0/24, each node has a public IP address; the second interface connects to a private subnet 172.17.1.0/24 .

First install Ubuntu 12.04 on each node, set up public IP addresses on eth0 so that you can access by SSH, we’ll reconfigure some part of them later.

Chef Architecture

chef-server stores the environment file, cookbooks, roles for each node.

Environment file describes your environment by overriding default attributes.

Cookbooks are chain of actions to bring the node to certain “state", cookbook are collections of attributes and recipes.

Role is a role for a machine, a role can be composed of many recipes and certain attributes or even other roles.

chef-server needs not to be in the same subnet as the nodes, it only needs to be accesible by all nodes. Putting chef-server on the Internet is also entirely fine.

chef-client needs to be installed on all nodes, whenever you run chef-client, it contacts chef-server and see if there’s any change of the environment or the role of the node. It updates the node to desired state by executing recipes.

Also take a look at http://www.rackspace.com/knowledge_center/article/rackspace-private-cloud-installation-prerequisites-and-concepts

Install chef-server and bootstrap the nodes

I highly recommend setting up FQDN for all nodes and chef-server, and set up SSH passwordless login for root on each node.

Put id_rsa in the chef-server, copy id_rsa.pub to all nodes.

chef-server should be acessible by all nodes at port 443 and 80.

Make sure all nodes and chef-server’s hostname are correctly configured. /etc/hostname should contain <hostname> and /etc/hosts should have entry like this:

x.y.54.50   controller.openstack.example.com        controller

Then just follow Rackspace’s guide: Install Chef Server, Cookbooks, and chef-client

Environment file

{
  "name": "swarm1",
  "description": "OpenStack Demo : Swarm 1",
  "cookbook_versions": {
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "default_attributes": {
  },
  "override_attributes": {
    "nova": {
      "network": {
        "provider": "neutron"
      }
    },
    "neutron": {
      "ovs": {
        "network_type": "gre"
      }
    },
    "horizon": {
      "endpoint_type": "publicURL",
      "endpoint_scheme": "http"
    },
    "mysql": {
      "allow_remote_root": true,
      "root_network_acl": "%"
    },
    "cinder": {
      "storage": {
        "lvm": {
          "volume_group": "storage-vg"
        }
      }
    },
    "osops_networks": {
      "nova": "x.y.54.0/24",
      "public": "x.y.54.0/24",
      "management": "172.17.1.0/24"
    }
  }
}

The horizon part is the fix for this bug. Additionaly, patch chef-cookbooks/cookbooks/horizon/recipes/server.rb like this patch. More information on this bug

Provider network settings like below is not needed in GRE setup. (They had it on Getting Started Guide p.33 so I misunderstood that I need it)

 "provider_networks": [
    {
        "label": "ph-eth1",
        "bridge": "br-eth1",
        "vlans": "1:1000"
    }
]

The osops_networks section defines networks that services should listen on (endpoints). Following list is copied from Rackspace Private Cloud Installation Prerequisites and Concepts.

Network Services
nova
  • keystone-admin-api
  • nova-xvpvnc-proxy
  • nova-novnc-proxy
  • nova-novnc-server
public
  • graphite-api
  • keystone-service-api
  • glance-api
  • glance-registry
  • nova-api
  • nova-ec2-admin
  • nova-ec2-public
  • nova-volume
  • neutron-api
  • cinder-api
  • ceilometer-api
  • horizon-dash
  • horizon-dash_ssl
management
  • graphite-statsd
  • graphite-carbon-line-receiver
  • graphite-carbon-pickle-receiver
  • graphite-carbon-cache-query
  • memcached
  • collectd
  • mysql
  • keystone-internal-api
  • glance-admin-api
  • glance-internal-api
  • nova-internal-api
  • nova-admin-api
  • cinder-internal-api
  • cinder-admin-api
  • cinder-volume
  • ceilometer-internal-api
  • ceilometer-admin-api
  • ceilometer-central

nova network should be accessible by future Horizon user, since the VNC proxy (nova-novnc-proxy) listens on that network, it must be accessible by clients so that their VNC console in Horizon will work. In nearly all cases you should put nova and public on the same network.

Chef will not set up these networks for you. During chef-client run of each node, it searches for the node’s IP on that network, and bind (some) services to that IP. For example, my management network is on 172.17.1.0/24, when deploying mysql on the controller node, chef-client searches for the node’s IP on that network, the node actually have 2 IPs, x.y.54.50 and 172.17.1.1, so it binds mysql to 172.17.1.1 .

發表迴響

在下方填入你的資料或按右方圖示以社群網站登入:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 變更 )

Twitter picture

You are commenting using your Twitter account. Log Out / 變更 )

Facebook照片

You are commenting using your Facebook account. Log Out / 變更 )

Google+ photo

You are commenting using your Google+ account. Log Out / 變更 )

連結到 %s