)
Sometimes you need just a drop, but other times the right flow of information is the whole rushing stream. That’s the idea behind an OpenStack service called Firehose, a single source for consuming Infra messages.
For example, if you have a script or tool listening for events from an Infra service, or one that has a poll loop (i.e. anything using Gerritlib), Firehose offers a one-stop service for consuming those messages.
There are two interfaces to subscribe to topics on Firehose, either via the MQTT protocol on the default port or via a WebSockets over port 80, for people on networks with stricter firewalls. The service launched mid-September with a Gerrit event stream (via germqtt) and launchpad bug events (via lpmqtt) sending events to Firehose — more are in the works.
Superuser talked to Matthew Treinish, member of the OpenStack Technical Committee, about how it’s evolving and how you can get involved.
Who could benefit from Firehose?
The main user would be anyone who has an application or use case where they depend on something running as part of the upstream OpenStack Infra service. Right now the only things reporting are events from Gerrit and launchpad bugs, which are the services most developers would interact with. So if anyone has any event-triggered automation based on those services, Firehose is a good place to turn to. Especially for launchpad bugs where there is no native event stream, Firehose enables you to subscribe to bug events.
One use case we had in mind when starting this work was third-party continuous integration (CI) operators. A lot of these CI systems run in environments behind restrictive firewalls. (Especially those in China behind the national firewall there.) Firehose provides a WebSocket interface on port 80, which means that it should be much easier to consume events needed to trigger CI jobs. There are two obstacles to doing this right now though: one, we’ve hit a bug in Mosquitto, the MQTT broker that we use for Firehose, causing instability:
http://docs.openstack.org/infra/system-config/firehose.html#connection-info
and Zuul doesn’t yet have support for consuming Gerrit events over MQTT. We have a example for doing this in gerritbot, (which I linked in the announcement email) but Zuul development has other priorities right now, specifically the Zuul v3 work. The MQTT support will come once the higher-priority tasks are complete.
What other improvements are you working on?
We’ve been working on getting all log lines being processed from logstash reporting into the Firehose for the past couple of weeks. We’ve hit reliability issues with the logstash plugin that would crash logstash. So we’ve had to disable it for now, but the plan is to get that running again in the near future, once we sort out that issue.
I’ve also been working on getting all the Ansible infra runs to start reporting events into the Firehose. I have a patch up to Ansible:
https://github.com/ansible/ansible-modules-extras/pull/2700
When that lands, it should enable us to start reporting events from any Ansible runs in infra.
We have some longer term goals for adding MQTT event output to other infra services, like Zuul, Nodepool, Storyboard, etc. But those are further down the road and not something anyone has started yet.
The other big thing that will be really helpful for people is when we can get Zuul to start consuming events from mqtt. Especially for the third-party CI use case mentioned above.
What are you doing at the Summit around Firehose and how can people get involved?
There will be a lightning talk on Firehose at Summit during the upstream track:
https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/15195/lightning-talks
Jeremy Stanley and I were going to give this together, but unfortunately I won’t be able to make it to Summit this time. (I’m not thrilled about this, it’s the first summit I’ve missed since Portland) So it’ll just be Jeremy presenting.
There will also likely be a Design Summit session on it during the Infra track. The Infra design summit sessions haven’t been finalized yet, but it’s likely going to be selected.
Other than that, there will be the infra contributors meetup on Friday afternoon of the summit where you can can interact with the infra team and others who are involved with the Firehose work and ask any questions:
Get started
The docs, which contain details on how the services are setup and includes some basic
steps and examples on how to start consuming events from the Firehose, are available here:
http://docs.openstack.org/infra/system-config/firehose.html
- OpenStack Homebrew Club: Meet the sausage cloud - July 31, 2019
- Building a virtuous circle with open infrastructure: Inclusive, global, adaptable - July 30, 2019
- Using Istio’s Mixer for network request caching: What’s next - July 22, 2019
