Part 2: Openstack TripleO Architecture and Step By Step Guide for installation of undercloud and overcloud nodes (compute, controller, ceph-storage)

by
Below list of topics are covered in this article

Bringup Undercloud setup

Verifying undercloud Installation

The successful installation of openstack director would generate the file "stackrc" that contains the environment variables to access the "undercloud" and the "undercloud-passwords.conf" file, which contains all the auto generated passwords when the parameters for passwords are not supplied during the "undercloud" installation.

[stack@undercloud-director ~]$ ls -l
total 28
-rw-------. 1 stack stack   609 Aug  7 22:52 stackrc
-rw-r--r--. 1 stack stack 10729 Aug  7 22:44 undercloud.conf
-rw-rw-r--. 1 stack stack  1650 Aug  7 22:46 undercloud-passwords.conf

Below is my "stackrc" file which was created after successful installation of undercloud

[stack@undercloud-director ~]$ cat ~/stackrc
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
NOVA_VERSION=1.1
export NOVA_VERSION
OS_PASSWORD=$(sudo hiera admin_password)
export OS_PASSWORD
OS_AUTH_URL=http://192.168.122.30:5000/v2.0
export OS_AUTH_URL
OS_USERNAME=admin
OS_TENANT_NAME=admin
COMPUTE_API_VERSION=1.1
OS_BAREMETAL_API_VERSION=1.15
OS_NO_CACHE=True
OS_CLOUDNAME=undercloud
OS_IMAGE_API_VERSION=1
export OS_USERNAME
export OS_TENANT_NAME
export COMPUTE_API_VERSION
export OS_BAREMETAL_API_VERSION
export OS_NO_CACHE
export OS_CLOUDNAME
export OS_IMAGE_API_VERSION

Check the openstack service status

[stack@undercloud-director ~]$ source stackrc
[stack@undercloud-director ~]$ openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-compute:                 active
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-cert:                    active
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     inactive  (disabled on boot)
== Horizon service ==
openstack-dashboard:                    404
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       inactive  (disabled on boot)
neutron-metadata-agent:                 inactive  (disabled on boot)
neutron-openvswitch-agent:              active
== Ceilometer services ==
openstack-ceilometer-api:               inactive  (disabled on boot)
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         active
openstack-ceilometer-notification:      active
== Heat services ==
openstack-heat-api:                     active
openstack-heat-api-cfn:                 active
openstack-heat-api-cloudwatch:          inactive  (disabled on boot)
openstack-heat-engine:                  active
== Ironic services ==
openstack-ironic-api:                   active
openstack-ironic-conductor:             active
== Support services ==
openvswitch:                            active
dbus:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
/bin/openstack-status: line 267: keystone: command not found
== Glance images ==
+----+------+-------------+------------------+------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+----+------+-------------+------------------+------+--------+
+----+------+-------------+------------------+------+--------+
== Nova managed services ==
+----+----------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary         | Host                        | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+----------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+
| 1  | nova-cert      | undercloud-director.example | internal | enabled | up    | 2018-08-12T11:24:15.000000 | -               |
| 2  | nova-scheduler | undercloud-director.example | internal | enabled | up    | 2018-08-12T11:24:17.000000 | -               |
| 3  | nova-conductor | undercloud-director.example | internal | enabled | up    | 2018-08-12T11:24:19.000000 | -               |
| 4  | nova-compute   | undercloud-director.example | nova     | enabled | up    | 2018-08-12T11:24:19.000000 | -               |
+----+----------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
WARNING: Command network-list is deprecated and will be removed after Nova 15.0.0 is released. Use python-neutronclient or python-openstackclient instead.
+--------------------------------------+----------+------+
| ID                                   | Label    | Cidr |
+--------------------------------------+----------+------+
| c8006f76-c2ec-4dde-a215-6aa5dcb34557 | ctlplane | -    |
+--------------------------------------+----------+------+
== Nova instance flavors ==
+--------------------------------------+---------------+-----------+------+-----------+------+-------+-------------+-----------+
| ID                                   | Name          | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+--------------------------------------+---------------+-----------+------+-----------+------+-------+-------------+-----------+
| 0d75914c-f87d-42bb-9814-25ebb41b4304 | control       | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
| 2e566c13-84ba-454f-8bb0-3a6f7e4c9e2d | compute       | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
| 53f70631-3ebd-4e72-9f5c-04c1c9638a0c | ceph-storage  | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
| 5883f1c8-8d56-42b3-b1a1-45e3e2957314 | block-storage | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
| 746208f6-609f-4a8a-a7de-6a3c02d0b542 | baremetal     | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
| 9989fb29-5796-4597-98e1-efe981358659 | swift-storage | 4096      | 40   | 0         |      | 1     | 1.0         | True      |
+--------------------------------------+---------------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+----+------+-----------+--------+------------+-------------+----------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+----+------+-----------+--------+------------+-------------+----------+
+----+------+-----------+--------+------------+-------------+----------+

You can also check the active and loaded Openstack Platform services using the below command

[stack@undercloud-director ~]$ systemctl list-units openstack-*
UNIT                                       LOAD   ACTIVE SUB     DESCRIPTION
openstack-aodh-evaluator.service           loaded active running OpenStack Alarm evaluator service
openstack-aodh-listener.service            loaded active running OpenStack Alarm listener service
openstack-aodh-notifier.service            loaded active running OpenStack Alarm notifier service
openstack-ceilometer-central.service       loaded active running OpenStack ceilometer central agent
openstack-ceilometer-collector.service     loaded active running OpenStack ceilometer collection service
openstack-ceilometer-notification.service  loaded active running OpenStack ceilometer notification agent
openstack-glance-api.service               loaded active running OpenStack Image Service (code-named Glance) API server
openstack-glance-registry.service          loaded active running OpenStack Image Service (code-named Glance) Registry server
openstack-heat-api-cfn.service             loaded active running Openstack Heat CFN-compatible API Service
openstack-heat-api.service                 loaded active running OpenStack Heat API Service
openstack-heat-engine.service              loaded active running Openstack Heat Engine Service
openstack-ironic-api.service               loaded active running OpenStack Ironic API service
openstack-ironic-conductor.service         loaded active running OpenStack Ironic Conductor service
openstack-ironic-inspector-dnsmasq.service loaded active running PXE boot dnsmasq service for Ironic Inspector
openstack-ironic-inspector.service         loaded active running Hardware introspection service for OpenStack Ironic
openstack-mistral-api.service              loaded active running Mistral API Server
openstack-mistral-engine.service           loaded active running Mistral Engine Server
openstack-mistral-executor.service         loaded active running Mistral Executor Server
openstack-nova-api.service                 loaded active running OpenStack Nova API Server
openstack-nova-cert.service                loaded active running OpenStack Nova Cert Server
openstack-nova-compute.service             loaded active running OpenStack Nova Compute Server
openstack-nova-conductor.service           loaded active running OpenStack Nova Conductor Server
openstack-nova-scheduler.service           loaded active running OpenStack Nova Scheduler Server
openstack-swift-account-reaper.service     loaded active running OpenStack Object Storage (swift) - Account Reaper
openstack-swift-account.service            loaded active running OpenStack Object Storage (swift) - Account Server
openstack-swift-container-updater.service  loaded active running OpenStack Object Storage (swift) - Container Updater
openstack-swift-container.service          loaded active running OpenStack Object Storage (swift) - Container Server
openstack-swift-object-updater.service     loaded active running OpenStack Object Storage (swift) - Object Updater
openstack-swift-object.service             loaded active running OpenStack Object Storage (swift) - Object Server
openstack-swift-proxy.service              loaded active running OpenStack Object Storage (swift) - Proxy Server
openstack-zaqar.service                    loaded active running OpenStack Message Queuing Service (code-named Zaqar) Server
openstack-zaqar@1.service                  loaded active running OpenStack Message Queuing Service (code-named Zaqar) Server Instance 1

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

32 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

To get the endpoint list

[stack@undercloud-director ~]$ openstack endpoint list
+----------------------------------+-----------+------------------+-------------------------+
| ID                               | Region    | Service Name     | Service Type            |
+----------------------------------+-----------+------------------+-------------------------+
| 83bc86eeb23f4a1eb8e3ee4375b8ab86 | regionOne | keystone         | identity                |
| e7cfd0f8a21c444b881af6f5d8391acb | regionOne | aodh             | alarming                |
| f7435f5a2ccf4b418a93bbe157355a42 | regionOne | mistral          | workflowv2              |
| d281dc7c6a044745a4fe242df2ea031a | regionOne | nova             | compute                 |
| 2fd6e5cf3ef64ec792664a28d51f8872 | regionOne | swift            | object-store            |
| cca985abdb994a1483f347d0b87e6549 | regionOne | heat             | orchestration           |
| e4f0b0137dc34fb0bd6e5d86d450175b | regionOne | glance           | image                   |
| 3bdae87eb724444fb9d172a3f2637253 | regionOne | neutron          | network                 |
| d3090c35e27346a89aec45310016151b | regionOne | ironic           | baremetal               |
| 794ee6f898764eda86d67e0e038dfaa3 | regionOne | zaqar-websocket  | messaging-websocket     |
| 483f411376b0421096e9a9507ebef45f | regionOne | ceilometer       | metering                |
| 927cd356089c48409596e7502121f150 | regionOne | zaqar            | messaging               |
| b7489e7d55b54b7b966a6f774f2aa124 | regionOne | ironic-inspector | baremetal-introspection |
+----------------------------------+-----------+------------------+-------------------------+

Os-net-config

Os-net-config is a simple tool to configure networking of Linux hosts. Combined with Cloud-init, it’s suitable for network configuration of virtual machines when booting in the cloud. By default, os-net-config reads the configuration file at /etc/os-net-config/config.yaml.

[stack@undercloud-director ~]$ python -m json.tool /etc/os-net-config/config.json
{
    "network_config": [
        {
            "addresses": [
                {
                    "ip_netmask": "192.168.122.30/24"
                }
            ],
            "members": [
                {
                    "mtu": 1500,
                    "name": "eth1",
                    "primary": "true",
                    "type": "interface"
                }
            ],
            "mtu": 1500,
            "name": "br-ctlplane",
            "ovs_extra": [
                "br-set-external-id br-ctlplane bridge-id br-ctlplane"
            ],
            "type": "ovs_bridge"
        }
    ]
}

Creating Directories for Templates and Images

The director uses system images and Heat templates to create the overcloud environment. To keep these files organized, we recommend creating directories for images and templates:

[stack@undercloud-director ~]$  mkdir ~/images
[stack@undercloud-director ~]$  mkdir ~/templates

Install director images

There are prebuilt images that are required for provisioning the overcloud nodes:

  • Discovery kernel and ramdisk: Used during the bare-metal discovery and introspection
  • Deployment kernel and ramdisk: Used for the first stage of provisioning and deployment
  • Overcloud kernel,ramdisk and full image: Base for each overcloud node created.

NOTE: For CentOS you can collect the images from "https://images.rdoproject.org/master/". The image files required are:

ironic-python-agent.initramfs
ironic-python-agent.kernel
overcloud-full.initrd
overcloud-full.qcow2
overcloud-full.vmlinuz

Install the below rpms to get these images which will be used for overcloud deployment.

[stack@undercloud-director ~]$ sudo yum -y install rhosp-director-images rhosp-director-images-ipa

Extract the archives to the images directory on the stack user’s home (/home/stack/images)

Copy the installed image files to (/home/stack/images)

$ cp -rvf /usr/share/rhosp-director-images/overcloud-full-latest-10.0.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-10.0.tar ~/images/
‘/usr/share/rhosp-director-images/overcloud-full-latest-10.0.tar’ -> ‘/home/stack/images/overcloud-full-latest-10.0.tar’
‘/usr/share/rhosp-director-images/ironic-python-agent-latest-10.0.tar’ -> ‘/home/stack/images/ironic-python-agent-latest-10.0.tar’

Extract the images from the tarfile

[stack@undercloud-director images]$ for tarfile in *.tar; do tar -xf $tarfile;done

[stack@undercloud-director images]$ ls -l
total 1923056
-rw-r--r--. 1 stack stack  425681543 Jun 29 02:20 ironic-python-agent.initramfs
-rwxr-xr-x. 1 stack stack    6390064 Jun 29 02:20 ironic-python-agent.kernel
lrwxrwxrwx. 1 stack stack         79 Aug 10 19:19 ironic-python-agent-latest-10.0.tar -> /usr/share/rhosp-director-images/ironic-python-agent-10.0-20180628.2.el7ost.tar
-rw-r--r--. 1 stack stack   63037676 Jun 29 02:34 overcloud-full.initrd
lrwxrwxrwx. 1 stack stack         74 Aug 10 19:19 overcloud-full-latest-10.0.tar -> /usr/share/rhosp-director-images/overcloud-full-10.0-20180628.2.el7ost.tar
-rw-r--r--. 1 stack stack 1467678720 Jun 29 02:39 overcloud-full.qcow2
-rwxr-xr-x. 1 stack stack    6390064 Jun 29 02:34 overcloud-full.vmlinuz

Remove the tar files

[stack@undercloud-director images]$ rm -f *.tar

Below is the list of images

[stack@undercloud-director images]$ ls -l
total 1923048
-rw-r--r--. 1 stack stack  425681543 Jun 29 02:20 ironic-python-agent.initramfs
-rwxr-xr-x. 1 stack stack    6390064 Jun 29 02:20 ironic-python-agent.kernel
-rw-r--r--. 1 stack stack   63037676 Jun 29 02:34 overcloud-full.initrd
-rw-r--r--. 1 stack stack 1467678720 Jun 29 02:39 overcloud-full.qcow2
-rwxr-xr-x. 1 stack stack    6390064 Jun 29 02:34 overcloud-full.vmlinuz

Upload overcloud images

Load the images into the containerized undercloud Glance:

[stack@undercloud-director ~]$ openstack overcloud image upload --image-path ~/images
Image "overcloud-full-vmlinuz" was uploaded.
+--------------------------------------+------------------------+-------------+---------+--------+
|                  ID                  |          Name          | Disk Format |   Size  | Status |
+--------------------------------------+------------------------+-------------+---------+--------+
| 544770a7-1864-4f2e-bda1-7c69196820c3 | overcloud-full-vmlinuz |     aki     | 6390064 | active |
+--------------------------------------+------------------------+-------------+---------+--------+
Image "overcloud-full-initrd" was uploaded.
+--------------------------------------+-----------------------+-------------+----------+--------+
|                  ID                  |          Name         | Disk Format |   Size   | Status |
+--------------------------------------+-----------------------+-------------+----------+--------+
| f4b75144-2078-4989-bc40-a38382552b6e | overcloud-full-initrd |     ari     | 63037676 | active |
+--------------------------------------+-----------------------+-------------+----------+--------+
Image "overcloud-full" was uploaded.
+--------------------------------------+----------------+-------------+------------+--------+
|                  ID                  |      Name      | Disk Format |    Size    | Status |
+--------------------------------------+----------------+-------------+------------+--------+
| 76d46575-f273-4d2f-89db-7a80cc0db3ee | overcloud-full |    qcow2    | 1467678720 | active |
+--------------------------------------+----------------+-------------+------------+--------+
Image "bm-deploy-kernel" was uploaded.
+--------------------------------------+------------------+-------------+---------+--------+
|                  ID                  |       Name       | Disk Format |   Size  | Status |
+--------------------------------------+------------------+-------------+---------+--------+
| 24b5292c-22dc-464d-ac39-e7d397bccf9f | bm-deploy-kernel |     aki     | 6390064 | active |
+--------------------------------------+------------------+-------------+---------+--------+
Image "bm-deploy-ramdisk" was uploaded.
+--------------------------------------+-------------------+-------------+-----------+--------+
|                  ID                  |        Name       | Disk Format |    Size   | Status |
+--------------------------------------+-------------------+-------------+-----------+--------+
| cb7bb24e-375e-490f-8420-97ac8961c776 | bm-deploy-ramdisk |     ari     | 425681543 | active |
+--------------------------------------+-------------------+-------------+-----------+--------+

This uploads the following images into the director:

  • bm-deploy-kernel
  • bm-deploy-ramdisk
  • overcloud-full
  • overcloud-full-initrd
  • overcloud-full-vmlinuz

Both "bm-deploy-kernel" and "bm-deploy-ramdisk" are used for baremetal deployment of the machines to be used for the overcloud, while the overcloud-full-vmlinuz, overcloud-full-initrd support the deployment of the Red Hat Openstack Platform environment itself.

View a list of the images in the CLI:

[stack@undercloud-director ~]$ openstack image list
+--------------------------------------+------------------------+--------+
| ID                                   | Name                   | Status |
+--------------------------------------+------------------------+--------+
| cb7bb24e-375e-490f-8420-97ac8961c776 | bm-deploy-ramdisk      | active |
| 24b5292c-22dc-464d-ac39-e7d397bccf9f | bm-deploy-kernel       | active |
| 76d46575-f273-4d2f-89db-7a80cc0db3ee | overcloud-full         | active |
| f4b75144-2078-4989-bc40-a38382552b6e | overcloud-full-initrd  | active |
| 544770a7-1864-4f2e-bda1-7c69196820c3 | overcloud-full-vmlinuz | active |
+--------------------------------------+------------------------+--------+

This list will not show the introspection PXE images. The director copies these files to /httpboot

[stack@undercloud-director ~]$ ls -l /httpboot
total 421956
-rwxr-xr-x. 1 root             root               6390064 Aug 12 17:37 agent.kernel
-rw-r--r--. 1 root             root             425681543 Aug 12 17:37 agent.ramdisk
-rw-r--r--. 1 ironic-inspector ironic-inspector       476 Aug 12 16:32 inspector.ipxe

Setting a Nameserver on the Undercloud’s Neutron Subnet

If you intend for the overcloud to resolve external hostnames, such as cdn.redhat.com, it is recommended to set a nameserver on the overcloud nodes. For a standard overcloud without network isolation, the nameserver is defined using the undercloud’s neutron subnet.

Use the following commands to define nameservers for the environment:

[stack@undercloud-director ~]$ source stackrc

[stack@undercloud-director ~]$ openstack subnet list
+--------------------------------------+------+--------------------------------------+------------------+
| ID                                   | Name | Network                              | Subnet           |
+--------------------------------------+------+--------------------------------------+------------------+
| deeaeb7d-e9e5-473c-88c3-10ca96b56f84 |      | c8006f76-c2ec-4dde-a215-6aa5dcb34557 | 192.168.122.0/24 |
+--------------------------------------+------+--------------------------------------+------------------+

[stack@undercloud-director ~]$ neutron subnet-update deeaeb7d-e9e5-473c-88c3-10ca96b56f84 --dns-nameserver 10.43.138.12
Updated subnet: deeaeb7d-e9e5-473c-88c3-10ca96b56f84

[stack@undercloud-director ~]$ openstack subnet show deeaeb7d-e9e5-473c-88c3-10ca96b56f84
+-------------------+------------------------------------------------------------+
| Field             | Value                                                      |
+-------------------+------------------------------------------------------------+
| allocation_pools  | 192.168.122.150-192.168.122.160                            |
| cidr              | 192.168.122.0/24                                           |
| created_at        | 2018-08-12T11:08:52Z                                       |
| description       |                                                            |
| dns_nameservers   | 10.43.138.12                                               |
| enable_dhcp       | True                                                       |
| gateway_ip        | 192.168.122.30                                             |
| host_routes       | destination='169.254.169.254/32', gateway='192.168.122.30' |
| id                | deeaeb7d-e9e5-473c-88c3-10ca96b56f84                       |
| ip_version        | 4                                                          |
| ipv6_address_mode | None                                                       |
| ipv6_ra_mode      | None                                                       |
| name              |                                                            |
| network_id        | c8006f76-c2ec-4dde-a215-6aa5dcb34557                       |
| project_id        | d7f4df43c5db41f08572f582344b4e6f                           |
| project_id        | d7f4df43c5db41f08572f582344b4e6f                           |
| revision_number   | 3                                                          |
| service_types     | []                                                         |
| subnetpool_id     | None                                                       |
| updated_at        | 2018-08-12T13:18:48Z                                       |
+-------------------+------------------------------------------------------------+

Create VMs for overcloud deployment

You need to create the virtual machines for your overcloud deployment. If you are good with command line then you can use "virsh" to create the vms and assign the network or you can login to your Desktop of the KVM Host and manually create the VMs using "Virt-Manager".

On my setup I have created all the requred nodes and below are the details

[root@openstack ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 29    director-undercloud            running
 -     overcloud-ceph.example         shut off
 -     overcloud-compute.example      shut off
 -     overcloud-controller.example   shut off