openstack API update

Earlier this year, I reported on our progress in updating the many bits of API information for OpenStack services in “What’s next for OpenStack application developer guides?” Since then, we have gathered even more information and lessons learned as OpenStack has also grown to add more and more REST API services.

At last count in the TC governance file, there are 28 REST API services under the OpenStack framework. This release, we have migrated many of those services from WADL and have collected all the links to API information for each of the projects in the projects.yaml file so links to documentation are published for each project. The collection is now available at developer.openstack.org/api-guide/quick-start/.

Here are some common questions and answers.

What drove the switch over from Web Application Description Language?

Back in 2010, when we first started documenting OpenStack APIs, WADL was a great way to do that. It was a standard submitted to the W3C, and we had hired an XML tools specialist at Rackspace to ensure we could do two things with WADL: filter requests to make sure they were valid for our cloud, and keep the docs true to the implementation. It was a win-win and we were documenting two APIs this way: Object Storage and Compute.

Fast-forward to 2014. The WADL specification from 2009 had not gotten the traction as a standard, the doc-specific tools team was disbanded, and trying to do two things (filter and document) with one tool (WADL) proved unwieldy. Plus, it’s not easy to community-source updates on a super complex tool like XML and WADL. Not to mention the allergy many developers have to XML. We needed to work on a new solution to document API reference information.

For the Liberty and Mitaka releases, I worked on two specifications to transform the API documentation. The main problems to solve are summarized as:

API reference information has been historically difficult to maintain using community or company resources due to the specialized nature of both the tools and the REST API knowledge.
API reference information needs to be sourced where API writers, API developers, contributor developers, and the API working group members can review together.

By switching from WADL, we could move to a format that was more modern and fitting for developers to maintain going forward. We sure made a good run at getting OpenAPI (Swagger) working. I believe parts of OpenAPI can be useful to the OpenStack community, but the lesson learned from WADL for me was that dual-purpose solutions eventually only have one purpose, and many solutions do not primarily exist for docs. Once we looked at using OpenAPI for a large API like Compute and evaluating whether we could document microversions, we opted for a new solution in using RST (ReStructuredText) plus YAML (Yet Another Markup Language) and a lot of web development to make sure we could get decent display and interaction with the reference parts. And, by using standard formats that OpenStack developers already use heavily, we made it easy for developers to contribute.

What advantages do you see in re-using the new format for downstream vendor documentation?

Our vision is to make developer.openstack.org a useful place for developers who are using OpenStack cloud resources to get the information they need to make requests against OpenStack endpoints. API reference information enables Software Development Kit (SDK) developers to create code, examples, and documentation that you use to create OpenStack cloud applications in the language of your choice. By using a format that our community can distribute maintenance on, and produce better web experiences, we can offer the most accurate docs across more REST API services.

The results are looking really quite good. Take a look at a list of all the APIs documented this way, and the Compute API and DNS API sites offer great examples of the new API reference, maintained by the project team rather than a central docs team.

We’re still working on a bit more interface “glue” to let users browse across all the OpenStack APIs, and with feature freeze at the end of August, we got all the moving parts of our tools (a Sphinx theme and a Sphinx extension) to fit together.

This was an amazing cross-purpose, cross-function, and cross-project effort accomplished by Karen Bradshaw, Sean Dague, Graham Hayes, Doug Hellman, Auggy Ragwitz, and Russell Sim with consistent assistance from the indefatigable Andreas Jaeger. This work had to span three releases (18 months), so we’re super happy to have this new framework in place and in use.

What can I do to try out this new API doc framework?

If you’re interested in contributing to these docs, go to a local copy, or clone your favorite (or least favorite, ha!) OpenStack service repository and look for an api-ref directory. If there’s not one already, create it using the contributor documentation.

If you’re already active in a project with an api-ref directory, try building the docs locally with:

$ tox -e api-ref

Read what your project’s API docs say, test it against actual requests and responses, and see if you can add to the accuracy of the documentation by making sure the docs match reality. As always, I’m here to help, and you can ask questions on the openstack-dev list with the [api] or [doc] topic, as you take a closer look.

Cover Photo // CC BY NC

Tags: API