Kilo Update: Ceilometer

Welcome to the PTL overview series, where we will highlight each of the projects and the upcoming features that will be in the next OpenStack release: Kilo. These updates are posted on the OpenStack Foundation YouTube channel, and each PTL is available for questions on IRC.

Eoghan Glynn is the PTL for Ceilometer, the telemetry service of OpenStack. In his PTL update webinar last week, he explained the main priorities for Ceilometer in the Kilo release cycle, with the primary effort focused on Gnocchi, OpenStack’s Time Series Data as a Service. Gnocchi has been a multi-cycle effort which Eoghan says will be finished in this release cycle.

Ceilometer’s Mission

Eoghan started the webinar by re-explaining the mission of the project:

“To reliably collect measurements of the utilization of physical & virtual resources comprising deployed clouds, persist this data for subsequent retrieval & analysis, and trigger actions when defined criteria are met."

In simpler terms, Eoghan explained, Ceilometer’s job is to surface insights into what’s going on in your cloud. In order to do that, it must collect measurements of the physical and virtual resources that comprise a cloud, how they are being used, how they are performing, and who is actually using them.

Then, Ceilometer must prolong this data so that it can be retrieved and analyzed. Finally, it has to be able to trigger actions when a defined criteria is met.

Priorties and Themes for Kilo

At the recent Paris Summit, the Ceilometer team spent time discussing plans for the upcoming release. The biggest focus is a multi-cycle effort that will be completed by the end of the Kilo cycle: Time Series Data as a Service, called gnocchi, and the full migration of Ceilometer.

In addition to that effort, the team will also be working parallel updates and activities, all detailed below.

Time Series Data as a Service

To recap, the goal of Gnocchi is to provide efficient metrics storage. There are drawbacks to the current approach that Ceilometer takes. One is that it snapshots resource metadata alongside each individual data point. While that provides a lot of flexibility and a good record of the evolution of the resource metadata, it’s not efficient since most of the metadata is static or near-static.

The other downside is that Ceilometer is currently predicated on the idea of all aggregation being done on-demand. The team plans to shift to a model in which Ceilometer pre-aggregates and rolls up this data as they are being adjusted.

Those are the two key fixes the team is working on during this release cycle. At the conclusion of Kilo, they plan to migrate the Ceilometer core to using Gnocchi as its primary metrics storage layer.

legbvd5h1upsg8xgprlc

The additional work planned for Kilo includes:

Recast v2 query API (that allows you to slice and dice Ceilometer in many different ways) over gnocchi semantics
Rebase Ceilometer alarm evaluation on the v3 metrics API
Migration of pre-existing metering datastores built up with classic ceilometer
Detailed profiling of performance benefits, publish results and indicate potential benefits

Test Coverage

The Ceilometer team plans to capitalize on a trend that’s occurring across many different OpenStack projects — moving test coverage out from the global Tempest test suite and into functional tests that are more directly associated with the project. The new tests are called in-tree functional tests because they live alongside the code that’s being tested.

This will allow Ceilometer to be less restricted in terms of how these integration tests are actually constructed.

As a part of this, the team also plans to produce a set of declarative API tests that allow you to read the tests as well as provide test coverage and an organic form of documentation that lays out how the API is meant to be used.

They will also introduce rally-based scenario tests which allow users to measure performance improvements as well as catch performance degradations early.

Richer Configuration of Tenant/Role Segregation

Currently, the admin vs. non-admin role is built on an "all or nothing" model. In other words, admin users are omniscient while normal users can only see the data.

This configuration is too absolute and the Ceilometer team is working on a more nuanced model that uses role-based access control mechanisms that Openstack provides. They will start leveraging the more forward-looking features of Keystone for this, including the idea of domains.

Ultimately, the admin role won’t have to necessarily be global — it will be something that can be partitioned between different, but related, groups of users.

Deployment Flexibility

The team is also focused on making deployments simpler and more flexible.

For small deployments, Ceilometer will provide a merged central and compute agent.

For larger deployments, the team will centralize the pipeline configuration, which is currently stored in a flat file that has to be rolled out to each individual node. Storing it centrally will allow it to be changed in a more global fashion across large deployments.

Finally, Ceilometer will allow the metrics gathering over SNMP to be truly declarative and driven by config. With this, a user can easily decide which SNMP metrics he or is she is interested in and change them on the fly.

Contractualized Service Notification

The Ceilometer team wants to schematize notifications that are emitted by services like Nova, Neutron, and Glance, for instance. Currently, these notifications are just free form dictionaries without any contract or stability involved. The goal is to allow these notification-based interactions to become as contract-based and as stable as a true API. The Ceilometer team is working jointly with Stacktack to make this happen in the Kilo release.

Event Processing Improvements

Ceilometer will improve the events pipeline in Kilo so that the event database is completely split off from the metrics database. Users can then decide to store their events in Mongo, for example, and store their data points in Gnocchi or any of the other storage drivers supported by Ceilometer.

Collaboration with Monasca

Monasca is an interesting and related project focused on monitoring at a large scale. The project grew out of an effort from HP and now involves contributions from several different large OpenStack companies including IBM and Rackspace.

In the upcoming release, the Ceilometer team plans to determine how best to collaborate and identify where Monasca and Ceilometer’s interests align. There are several potential opportunities that Eoghan said he is currently investigating, including the repurposing of Monasca’s anomaly detection engine and using Apache Kafka for high-throughput internal messaging.