Deploy OpenStack with Chef, Part 2: Network

In this part we’ll set up the network node, Neutron plugins and create base Neutron networks. Rackspace’s documentation to set up Neutron was confusing to me, so I checked the official OpenStack Documentation and figured out some things need to be done differently from Rackspace’s documentation.

Set up br-ex

Execute commands in this section on the network node.

After the chef-client run on network node finishes, br-ex is created, we need to manually add our external interface to br-ex, during the procedure the external network connectivity will be lost so it’s a good idea to SSH into network node via its private IP (In my case 172.17.1.5), also just in case that external network never get back up.

Remove IP address on eth0 because we want to add it to a bridge. Network connectivity on eth0 will be lost. I learned this command here.

# ip addr flush dev eth0

Add eth0 to the bridge br-ex

# ovs-vsctl add-port br-ex eth0

Assign public IP to br-ex and add default gateway

# ifconfig br-ex inet x.y.54.54 netmask 255.255.255.0
# route add default gw x.y.54.254

Now you can verify if external connectivity works. If not, check your routing table, mine looks like this:

0.0.0.0         x.y.54.254      0.0.0.0         UG        0 0          0 br-ex
140.113.54.0    0.0.0.0         255.255.255.0   U         0 0          0 br-ex
172.17.1.0      0.0.0.0         255.255.255.0   U         0 0          0 eth1

Edit /etc/network/interfaces as my example to make these settings persistent.

Create base Neutron networks

Execute commands in this section on the controller node.

Mostly the same as official OpenStack documentation.

Read credentials:

# source ~/openrc

Create net network:

# neutron net-create ext-net --router:external=True \
  --provider:network_type gre --provider:segmentation_id 2 --shared

Create subnet for ext-net:

# neutron subnet-create ext-net --allocation-pool start=x.y.54.1,end=x.y.54.10 \
  --gateway=x.y.54.254 --enable_dhcp=False x.y.54.0/24

Create router:

# neutron router-create ext-to-int

Connect the router to ext-net by setting the gateway for the router as ext-net:

# neutron router-gateway-set EXT_TO_INT_ROUTER_ID EXT_NET_ID

Create internal network and subnet:

# neutron net-create admin-net
# neutron subnet-create admin-net 192.168.2.0/24 --gateway 192.168.2.1

Connect the internal subnet to router:

# neutron router-interface-add EXT_TO_INT_ROUTER_ID INTERNAL_SUBNET_ID

Update internal subnet’s DNS information:

# neutron subnet-update INTERNAL_SUBNET_ID \
  --dns_nameservers list=true 8.8.8.8 8.8.8.7

Now you can boot up an instance, select the interface to admin-net, and see if it gets IP and DNS settings correctly.

Deploy OpenStack with Chef, Part 3: Storage

Cinder needs a LVM volume group to create volumes. Make sure you have free space on the VG (on storage node):

# vgdisplay
  --- Volume group ---
  VG Name               storage-vg
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               3
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               546.21 GiB
  PE Size               4.00 MiB
  Total PE              139831
  Alloc PE / Size       13824 / 54.00 GiB
  Free  PE / Size       126007 / 492.21 GiB
  VG UUID               MfihR0-9350-o1T9-OQhU-aL2E-RtwA-JO9I7d

If your have no free space left, you can follow this guide to shrink existing partitions (LV).

In the environment file, you can specify which VG cinder should use:

    "cinder": {
      "storage": {
        "lvm": {
          "volume_group": "<VG_name>"
        }
      }
    },

Then on chef server, add the cinder-all role:

# knife node run_list add storage.openstack.xxxx.tw 'role[cinder-all]'

Run chef-client on storage node:

# chef-client -E <environment_name>

Run chef-client on controller node again, since cinder-api and cinder-scheduler run on it:

# chef-client -E <environment_name>

Problems

I encountered a weird problem that cinder services on storage node cannot connect to MySQL on controller. Investigation found that /etc/cinder.conf has wrong mysql password. I copied over mysql password for user cinder from controller’s /etc/cinder.conf and it worked. This is just a temporary fix, I guess it would be overwritten on the next chef-client run, not sure why chef-client gets the wrong password.

Deploy OpenStack with Chef, Part 0: Basic Concepts & Planning Environment

In this series of post I will demonstrate deploying OpenStack with Chef, using Rackspace Private Cloud’s cookbooks. This guide largely follows Rackspace’s documentation (Getting Started Guide, list of documentation), with notes by myself.

One thing to clarify is that Chef doens’t do the bootstrapping of nodes for you. I misunderstood this at first. You need to install and configure the OS, networking, network bridges required by Neutron manually.

My setup will be

  1. controller node: x.y.54.50, 172.17.1.1
  2. compute node: x.y.54.51, 172.17.1.2
  3. network node: x.y.54.54, 172.17.1.5, runs Neutron, with OVS plugin and GRE tenant network
  4. storage node: x.y.54.53, 172.17.1.4, runs Cinder

Each node has 2 interfaces, first one (not necessarily eth0, since there will be br-ex on network node) connects to public network x.y.54.0/24, each node has a public IP address; the second interface connects to a private subnet 172.17.1.0/24 .

First install Ubuntu 12.04 on each node, set up public IP addresses on eth0 so that you can access by SSH, we’ll reconfigure some part of them later.

Chef Architecture

chef-server stores the environment file, cookbooks, roles for each node.

Environment file describes your environment by overriding default attributes.

Cookbooks are chain of actions to bring the node to certain “state", cookbook are collections of attributes and recipes.

Role is a role for a machine, a role can be composed of many recipes and certain attributes or even other roles.

chef-server needs not to be in the same subnet as the nodes, it only needs to be accesible by all nodes. Putting chef-server on the Internet is also entirely fine.

chef-client needs to be installed on all nodes, whenever you run chef-client, it contacts chef-server and see if there’s any change of the environment or the role of the node. It updates the node to desired state by executing recipes.

Also take a look at http://www.rackspace.com/knowledge_center/article/rackspace-private-cloud-installation-prerequisites-and-concepts

Install chef-server and bootstrap the nodes

I highly recommend setting up FQDN for all nodes and chef-server, and set up SSH passwordless login for root on each node.

Put id_rsa in the chef-server, copy id_rsa.pub to all nodes.

chef-server should be acessible by all nodes at port 443 and 80.

Make sure all nodes and chef-server’s hostname are correctly configured. /etc/hostname should contain <hostname> and /etc/hosts should have entry like this:

x.y.54.50   controller.openstack.example.com        controller

Then just follow Rackspace’s guide: Install Chef Server, Cookbooks, and chef-client

Environment file

{
  "name": "swarm1",
  "description": "OpenStack Demo : Swarm 1",
  "cookbook_versions": {
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "default_attributes": {
  },
  "override_attributes": {
    "nova": {
      "network": {
        "provider": "neutron"
      }
    },
    "neutron": {
      "ovs": {
        "network_type": "gre"
      }
    },
    "horizon": {
      "endpoint_type": "publicURL",
      "endpoint_scheme": "http"
    },
    "mysql": {
      "allow_remote_root": true,
      "root_network_acl": "%"
    },
    "cinder": {
      "storage": {
        "lvm": {
          "volume_group": "storage-vg"
        }
      }
    },
    "osops_networks": {
      "nova": "x.y.54.0/24",
      "public": "x.y.54.0/24",
      "management": "172.17.1.0/24"
    }
  }
}

The horizon part is the fix for this bug. Additionaly, patch chef-cookbooks/cookbooks/horizon/recipes/server.rb like this patch. More information on this bug

Provider network settings like below is not needed in GRE setup. (They had it on Getting Started Guide p.33 so I misunderstood that I need it)

 "provider_networks": [
    {
        "label": "ph-eth1",
        "bridge": "br-eth1",
        "vlans": "1:1000"
    }
]

The osops_networks section defines networks that services should listen on (endpoints). Following list is copied from Rackspace Private Cloud Installation Prerequisites and Concepts.

Network Services
nova
  • keystone-admin-api
  • nova-xvpvnc-proxy
  • nova-novnc-proxy
  • nova-novnc-server
public
  • graphite-api
  • keystone-service-api
  • glance-api
  • glance-registry
  • nova-api
  • nova-ec2-admin
  • nova-ec2-public
  • nova-volume
  • neutron-api
  • cinder-api
  • ceilometer-api
  • horizon-dash
  • horizon-dash_ssl
management
  • graphite-statsd
  • graphite-carbon-line-receiver
  • graphite-carbon-pickle-receiver
  • graphite-carbon-cache-query
  • memcached
  • collectd
  • mysql
  • keystone-internal-api
  • glance-admin-api
  • glance-internal-api
  • nova-internal-api
  • nova-admin-api
  • cinder-internal-api
  • cinder-admin-api
  • cinder-volume
  • ceilometer-internal-api
  • ceilometer-admin-api
  • ceilometer-central

nova network should be accessible by future Horizon user, since the VNC proxy (nova-novnc-proxy) listens on that network, it must be accessible by clients so that their VNC console in Horizon will work. In nearly all cases you should put nova and public on the same network.

Chef will not set up these networks for you. During chef-client run of each node, it searches for the node’s IP on that network, and bind (some) services to that IP. For example, my management network is on 172.17.1.0/24, when deploying mysql on the controller node, chef-client searches for the node’s IP on that network, the node actually have 2 IPs, x.y.54.50 and 172.17.1.1, so it binds mysql to 172.17.1.1 .

Run chef-server and OpenStack controller on same node

Update: This did not end up well so I gave up, actually you can put Chef server anywhere as long as it can be reached by clients, even in another VM would work after some patches to RackSpace scripts.

Chef’s own rabbitmq-server will conflict with OpenStack’s, so I had to configure Chef to use the system rabbitmq.

Use this guide: https://community.rackspace.com/products/f/45/t/1669

Also add
bookshelf['url'] = "https://#{node['ipaddress']}:4000"
to /etc/chef-server/chef-server.rb.

ref: https://groups.google.com/forum/#!topic/opscode-chef-openstack/0DnbYhxQT90

瀏覽器分頁群組、歷史管理與同步

Firefox 覺得 Panorama (Tab Group) 不是個很重要的功能啊⋯⋯他們說很少人用

翻到 Tab Candy 這個計劃,我有一陣子也想要做幾乎一模一樣的東西,只是後來因為 Firefox addon 太難寫所以放棄,他的構想和我根本一模一樣。

要是真的有的話我覺得會是個革命性的功能,分頁管理應該是很多人共同的頭痛⋯⋯

一直覺得現在的瀏覽器並不是針對目前人們的瀏覽行為所設計。

上網很多時候是為了尋找解答,像是解 bug 、找衣服、筆電、螢幕、相機,而且常會同時進行不同主題的搜尋。我記得 IE 的分頁會自動上色,還有 Firefox 的分頁群組功能都部分解決了這個問題。

另一個我覺得很頭痛的點是很難找到以前曾經瀏覽的網頁,因為我通常記不住網頁的標題,只有關鍵字。要是瀏覽歷史會記住我是怎麼打開這個頁面就更好了(我是從哪裏點開這個連結的?)

可惜 Tab Candy 的提案者跑去開公司,不做這個專案了⋯⋯

我是真的很希望有這些功能,但是這些貌似是要直接改 Firefox 核心,就算寫成 addon 形式也要用比較底層的 API 。我不想寫 C++ 啊⋯⋯而且就算是用 Javascript 寫 addon 還是要了解 Firefox 那複雜的架構。

筆電換高解析度的螢幕

前幾天不小心絆到電源線,筆電就從桌上摔下來了……螢幕裂掉報銷了,超痛

靈機一動想說不知道可不可以趁機換個解析度高一些的螢幕,我的 Acer TimelineX 4830tg 14吋只有 1366×768 ,有時候覺得像素有點粗啊,於是展開了這次的研究。

近幾年筆電面板的接頭(從面板到主機板)一般是 LVDS 或 eDP ,LVDS 通常用於解析度小於等於 1366×768 的面板(雖然 LVDS 可以支援更高解析度),更高解析度的面板以前也會用 LVDS 但是比較新的筆電應該都換用 eDP 了。

4830tg 面板固定在上蓋的方式是用面板突出的螺絲孔,其他的我沒看過就不清楚了。

同一種 form factor (尺寸、螺絲位置)的面板,常常會有很多型號的筆電共用,也會同時有很多製造商出。Acer 和 Asus 兩家有很多型號的筆電都是共用一種 form factor 的面板的。

爬文的結果顯示一般的筆電,尤其是 Acer 和 Asus 兩家,主機板輸出訊號的時候,都會自動偵測面板解析度,調成正確的,不需要對 BIOS 做特別調整。其他家就比較有可能會在 BIOS 裏面把輸出解析度鎖死,要改 BIOS 才能解決這個障礙。有爬到大陸那邊很多人成功把筆電換高解析度面板的,都是直接拆開換上新面板,開機就會動了。

LVDS 又分爲單通道和雙通道,就是 1ch 和 2ch ,若要用 1366×768 以上解析度就一定要用 2ch 的,雖然也有廠商出 1ch 1366×768 以上解析度的面板,但是這類面板很難自行取得,尤其是在臺灣。(在大陸可能容易一些但還是不太可能)

要達到雙通道有幾個要件:

  1. 主機板要有雙通道的輸出,去看筆電主機板的 spec 。通常是先去搜尋筆電用的主機板型號,然後用 型號+schematic 搜尋。如果主機板沒有把第二個 channel 的訊號輸出那就完全沒希望囉
  2. 依照規格 LVDS 線有 40 個 pin ,其中一部分是通道一和通道二(還有其他東西像是電源),把 LVDS 接頭的膠布撕開來看,如果發現有腳位是空接的那八成就是通道二的線沒有接,這個情況下可以自己去買完整的 LVDS 的線回來換
  3. 當然螢幕要是雙通道囉

另外 LVDS 好像並沒有定義腳位的用途,所以不同廠商的主機板輸出信號的腳位位置可能有差別,螢幕也是,輸入的信號位置耶有可能不同,這就要去查 schematic ,然後自己把 LVDS 裏面的線換到對的位置。

不過查了 4830tg 的主機板 schematic 發現根本沒有輸出通道二的信號,最後還是認命換了 LVDS 1ch 1366×768 的螢幕⋯⋯