Trove, the OpenStack database-as-a-service provides database provisioning and life cycle management capabilities for a number of databases. It exposes a single REST’ful API that can be used to access all the capabilities provided by the service. A command line interface and the Horizon web interface interact with Trove via this REST API.Trove allows interactions on this REST API to be secured using secured sockets layer (SSL.)
The previous two parts describe SSL, what it is, how it works, how one interacts with the Trove REST API, how the OpenStack Keystone service catalog is used in discovering service end-points. The final (third) part builds on the earlier two and describes how to enable SSL in Trove, and some useful trouble-shooting tips.

Enabling SSL with Trove

To enable SSL with Trove, we need to do several things. These are described in detail below.

Get a keypair and a certificate for the Trove Controller

The Trove Controller machine needs a keypair and a certificate and you can either get these from a trusted vendor (Certificate Authority) or you can create your own private CA and issue these yourself. While a private CA is sufficient for development and testing, if you wish to operate the service in production with clients on the internet, you will likely want to get a certificate and keypair issued by a trusted public CA.
We describe below the steps involved in creating your own keypair and certificate using a private CA. We follow the instructions found at https://help.ubuntu.com/lts/serverguide/certificates-and-security.html
Using the steps described there, we generate the server key (server.key), the server certificate (server.crt) and the CA Certificate (cacert.crt) and copy them into /etc/trove along with the Trove configuration files.
We can then add the following line to trove.conf

[ssl]
cert_file=/etc/trove/server.crt
key_file=/etc/trove/server.key

The client needs access to the CA Certificate as it uses this to validate the public key and certificate that the server will send to it.
Update the service catalog to indicate that Trove offers SSL
First recreate the database service

amrith@trove-controller:~$ openstack service list
+----------------------------------+-------------+----------------+
| ID                               | Name        | Type           |
+----------------------------------+-------------+----------------+
| 06a410d51a26492baa0b40a73b4feb1f | glance      | image          |
| 2ae1579a4aae41a6b294cd7fffaa16cc | cinder      | volume         |
| 3f96de164d07400ba16af8eaea76d470 | cinderv2    | volumev2       |
| 4c4b209eab1140e7a968fdc25aab6dee | swift       | object-store   |
| 5543cb00401d45c69fca7265a862c83e | nova_legacy | compute_legacy |
| 63fb77352fea42329ba47063e8a91f34 | ec2         | ec2            |
| 6417b68b2dc64c179088a7ae806cef88 | trove       | database       |
| 6db4050bd9d0487f9ae453d7bafb7011 | keystone    | identity       |
| 8ecd1cd5bd924afa85e6964852cd6ed9 | heat-cfn    | cloudformation |
| 9338774d87534473a3ae8b883e180696 | nova        | compute        |
| a05c11b57b614e8bb89f66632fd52e32 | heat        | orchestration  |
+----------------------------------+-------------+----------------+
amrith@trove-controller:~$ openstack service delete 6417b68b2dc64c179088a7ae806cef88

amrith@trove-controller:~$ openstack service create --name trove --description 'Database as a Service with SSL' database
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Database as a Service with SSL   |
| enabled     | True                             |
| id          | 70ed0eb9558e4a73b6c884e5660be26a |
| name        | trove                            |
| type        | database                         |
+-------------+----------------------------------+

amrith@trove-controller:~$ openstack endpoint create 
> --publicurl 'https://192.168.115.20:8779/v1.0/$(tenant_id)s' 
> --adminurl 'https://192.168.115.20:8779/v1.0/$(tenant_id)s' 
> --internalurl 'https://192.168.115.20:8779/v1.0/$(tenant_id)s' 
> --region RegionOne database
+--------------+------------------------------------------------+
| Field        | Value                                          |
+--------------+------------------------------------------------+
| adminurl     | https://192.168.115.20:8779/v1.0/$(tenant_id)s |
| id           | d87532f2d036451eae2be193a945fef5               |
| internalurl  | https://192.168.115.20:8779/v1.0/$(tenant_id)s |
| publicurl    | https://192.168.115.20:8779/v1.0/$(tenant_id)s |
| region       | RegionOne                                      |
| service_id   | 70ed0eb9558e4a73b6c884e5660be26a               |
| service_name | trove                                          |
| service_type | database                                       |
+--------------+------------------------------------------------+

Finally, restart the Trove API service

How exactly you do this depends on how you installed OpenStack on your trove-controller. If it is a development machine which was installed using devstack, then reconnect to the devstack screen session and restart the Trove API process. . If you use the Tesora distribution, then you would restart the trove-taskmanager service (on Ubuntu) or the openstack-trove-taskmanager service (on Centos or RHEL).

Retry your trove CLI command

You can now retry your trove CLI command by providing the CA Certificate to the CLI as shown below.

amrith@trove-client:~$ trove --os-auth-url http://192.168.115.20:5000/v2.0 
> --os-tenant-name admin 
> --os-username admin 
> --os-password 3de4922d8b6ac5a1aad9 
> --os-cacert ./cacert.crt list
+----+------+-----------+-------------------+--------+-----------+------+
| ID | Name | Datastore | Datastore Version | Status | Flavor ID | Size |
+----+------+-----------+-------------------+--------+-----------+------+
+----+------+-----------+-------------------+--------+-----------+------+

Why do you need to provide the CA Certificate to the CLI?

Recall that SSL uses Certificates to establish identity. In the example above, the CLI will attempt to establish a secure connection with the server at https://192.168.115.20:8779. Note that this is the endpoint provided in the earlier ‘endpoint create’ command.
We are illustrating how to configure Trove to use SSL using a self-signed certificate. Therefore, when the server provides the client with its certificate to prove that it is, in fact, the true server “192.168.115.20”, this certificate would have been issued by the CA that we setup. Providing the CA certificate to the client enables the client to validate the server’s certificate.
If we had instead purchased a certificate from a well-known third party CA, and the client machine already had a server certificate for that CA installed, it would be able to validate the server certificate using that well known CA’s server certificate.

Some errors that you may encounter

We now describe some errors that you may encounter in configuring SSL.
Providing the wrong command line option on the CLI
We have shown above how you configure server side SSL. For this, client certificates are not validated. Therefore, if you were to accidentally pass the CA Certificate to the trove CLI with –os-cert (instead of the correct –os-cacert option) you would get an error like this.

amrith@trove-client:~$ trove --os-auth-url http://192.168.115.20:5000/v2.0 --os-tenant-name admin --os-username admin --os-password 3de4922d8b6ac5a1aad9 --os-cert ./cacert.crt list
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:100: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
ERROR: SSL exception connecting to https://192.168.115.20:8779/v1.0/b0aea5682aea4074b6f215931eaebf56/instances: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Incorrectly specifying client certificate validation

The Trove configuration file has three SSL settings and we have illustrated how to configure server side SSL. That requires only two of the settings.

[ssl]
cert_file=/etc/trove/server.crt
key_file=/etc/trove/server.key

If you were to also provide the ca_file setting in trove.conf, you are instructing Trove API to validate client certificates and you will get a failure like this:

SSLError: SSL exception connecting to https://192.168.115.20:8779/v1.0/b0aea5682aea4074b6f215931eaebf56/instances: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
ERROR: SSL exception connecting to https://192.168.115.20:8779/v1.0/b0aea5682aea4074b6f215931eaebf56/instances: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

Enabling client side certificates

To enable client certificates (which allow the server to validate the client) you must install a certificate on the client machine and provide that to the trove command with the –os-cert command line option. If the certificate is from a trusted CA, then you need not provide anything further. However, if you have a private CA, you must provide the CA Certificate to the Trove API Service via the ca_file option in the [ssl] section.
Examining the CLI to Server interactions with SSL enabled
Finally, we conclude this write-up by looking at the interactions between the CLI and the Controller when SSL is enabled for Trove. We illustrate this with the trove flavor-list command.

amrith@trove-client:~$ trove --debug --os-auth-url http://192.168.115.20:5000/v2.0 --os-tenant-name admin --os-username admin --os-password 3de4922d8b6ac5a1aad9 --os-cacert ./cacert.crt flavor-list
[…]
INFO (connectionpool:207) Starting new HTTP connection (1): 192.168.115.20
[…]

DEBUG (v2:86) Making authentication request to http://192.168.115.20:5000/v2.0/tokens
[…]

INFO (connectionpool:756) Starting new HTTPS connection (1): 192.168.115.20
[…]

+-----+----------------+-------+
|  ID | Name           |   RAM |
+-----+----------------+-------+
|   1 | m1.tiny        |   512 |
|  10 | eph.rd-smaller |   768 |
|   2 | m1.small       |  2048 |
|   3 | m1.medium      |  4096 |
|   4 | m1.large       |  8192 |
|  42 | m1.nano        |    64 |
| 451 | m1.heat        |   512 |
|   5 | m1.xlarge      | 16384 |
|   6 | tinier         |   506 |
|   7 | m1.rd-tiny     |   512 |
|   8 | m1.rd-smaller  |   768 |
|  84 | m1.micro       |   128 |
|   9 | eph.rd-tiny    |   512 |
+-----+----------------+-------+

References

Ubuntu Guide for Self Signed Certificates, Certificate Signing Requests, and private CA’s at https://help.ubuntu.com/lts/serverguide/certificates-and-security.html
What is SSL? at https://www.digicert.com/ssl.htm
Wikipedia entry about SSL at https://en.wikipedia.org/wiki/SSL

Amrith Kumar, an active technical contributor and core reviewer of the OpenStack Trove project, as well as founder and CTO of Tesora Inc. He’s also the co-author of the “OpenStack Trove” book, available online.

Superuser is always interested in how-tos and other contributions, get in touch at [email protected]

Cover Photo // CC BY NC