Report from the Mid-Cycle Meetup for Operators

I recently attended an OpenStack “Mid-cycle meetup” for Operators, generously hosted by Rackspace in San Antonio, Texas with the help of Matt Van Winkle. I’d like to tell you a little bit about my experience, and why the “mid-cycle meetup” is another trend to watch in the evolution of how software gets written in the cloud era.

What is a mid-cycle meetup? It’s another way stackers come together to keep the momentum going as a community, in between Summits, in smaller, focused groups. Each of the projects (think “Nova”, “Neutron”, etc.) have started having mid-cycle meetups focused on their specific domains, and Operators are now joining in on the fun.

Sure, we all collaborate online every day on mailing lists, IRC, and a dozen other forums, but there is no substitute for sitting down for breakfast tacos across the table from your peers and working on the thorny issues all operators face.

rxwijy0laba4ig7ofvyk

When it comes to events, we do things a little differently in OpenStack. The traditional model is for a single dominant vendor to have their own event, typically costing thousands of dollars, where attendees hear from the mountain top just what “their future looks like” from the vendors point of view. Products are unveiled that were years in the making. Call it the annual Moscone pilgrimage.

vd15ajkdpl2a8pjqd73u

While those events do seem to be bigger and better than ever at generating buzz, we’ve always found that two Summits a year, led and planned by the community in the open, are far more effective in building what users really want. To provide even more face time, we’re introducing the mid-cycle meetup series between Summits, where smaller groups of peers can share knowledge and help direct the future of the platform. I think we’ll find the next Summit in Paris more productive as a result.

Focus, Not Flash

At the ops meetup I was really struck by the atmosphere: all focus, no flash. Quite a contrast to the other event going on at that exact moment in San Francisco, VMworld. Don’t get me wrong, the news coming out of VMworld was awesome, with the introduction of “VMware Integrated OpenStack” (VIO), and members of the Foundation were in attendance to show their support. But at that moment I was glad I drew the Operators assignment. No PowerPoint? No problem.

fti2lx2jp1xylvxwizcg

Real life Superusers from companies like Comcast, Time Warner Cable, GoDaddy, Yahoo, Sony Playstation, Symantec, Cisco, Workday, IBM, Bluebox, Intel, and Paypal made the time to attend and collaborate. We even had two members of the team from NTT fly in from Japan to contribute, not to mention one member of the VMware team that skipped their event (name withheld to protect the witness). All of these operators came ready to work, to talk about the pain points they’d encountered running thousands of nodes, battles with upgrades, and making tough configuration decisions.

Also in attendance were several prominent leaders from the technical community responsible for setting the roadmap, including Michael Still (PTL of Nova), John Dickson (PTL of Swift) and Mark McClain (of Neutron fame). They were all very excited to hear straight from users about the areas they’d like to see improved going forward. More than once I heard an operator voice a problem they were having, and Michael Still would jump to the front of the room and say “We can solve that! Tell me more.” It was amazing to see the future of software development first hand.

Decisions, Decisions

One issue on everyone’s mind was the path to Neutron from Nova Networking. After much back and forth, some interesting upgrade scenarios were identified that most users thought they could live with. I won’t try to capture those here but an overall meta etherpad was maintained here, and you can find links to deep dives like the networking session here.

Networking migration is a very tough issue to solve, and that’s exactly the reason we come together. This is why the Nova PTL was willing to fly from Australia to San Antonio to sit down with users and tackle problems head-on.

As a community, OpenStack has never shied away from tough problems, or been afraid to make a change when things aren’t working (anyone remember the original Keystone?). In the case of Nova Network, the original networking stack in OpenStack that many users still rely on, the date for phasing it out has been extended more than once as users ask for more time, and the new promised land of Neutron continues to evolve until it hits the mark for our users.

Another big topic was upgrades. Many users who are still running older versions were happy to learn that upgrades from Havana to Icehouse were possible without impacting end users (with some caveats) and that upgrade testing was now a regular part of testing in the Juno cycle and beyond. That said, it was clear that users want to see upgrades become more and more seamless, while also looking for long term support for older versions when the upgrade just didn’t make sense in their context. It’s a balancing act, but if we keep listening to users we’ll make the right calls as a community.

Documentation was another area of active discussion, and Anne Gentle (Docs PTL) was in attendance to gather feedback on areas for improvement.

Last, but not least, there were several breakout sessions with smaller teams, and the one I attended was on the operator feedback loop. The initial changes to the blueprint process (to start with a design document prior to blueprint approval) received positive feedback, but more work is definitely needed to refine this loop so that developers continue to receive valuable feedback on what’s working and what isn’t from an operator point of view. Meeting notes for this breakout can be found here.

Doing the Right Thing

At the end of the day, whether you’re talking to developers, users, or companies within the ecosystem, the focus is on doing the right thing for the long term, because that’s the culture that has defined OpenStack, and will see us through many more twists and turns in this very dynamic cloud market.

For more details on the operators meetup, look for a post from Sriram coming to Superuser soon. I would also like to encourage you to get more involved in these discussions.

To stay current on future Operators events, join the operators mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators, and attend the upcoming Summit in Paris Nov 3-8th. The event includes a special Operators Summit breakout event.

And finally, please give your feedback via the user survey — we want your voice to be heard: https://www.openstack.org/user-survey/Login