)
This article tackles the topic of unifying the OpenStack Operators community around a common goal of authoring, supporting and maintaining a rich set of documents that help operators in the field deploy and manage their OpenStack environments. Previous documents were static and maintained in a repository that was difficult to contribute to. Migrating them to the OpenStack Wiki allows them to become “living” documents.
The community around OpenStack is very broad, much like the various components that comprise OpenStack, both in terms of skills and regional locality. With such a diverse and dispersed project team, it becomes very difficult to pull them together to work for a common goal.
One very important and often-overlooked component of the OpenStack project, is its documentation, which serves as a tool for many, to help bridge this gap.
Today, the documentation is maintained in its own Git repository for anyone who wishes to checkout the source and build them, paired up with each OpenStack release. The process of building the OpenStack manuals from the repository requires some initial setup and dependencies to be installed on your build machine, but once you’ve done that, they should build cleanly without any issues.
Until very recently, those manuals were directly tied to the specific upstream releases of OpenStack. This meant that when an upstream release was marked EOL (“End of Life”) by the OpenStack project team, its repository tag was removed and the manuals would disappear with it. When the build process fell apart, you’d have to spend time patching or debugging the failures. This was the case with the previous EOL releases of the openstack-manuals, which relied on deprecated tags and legacy dependencies that caused build failures.
Removing the manuals for a release that has gone EOL leaves a gap between the versions of OpenStack that are currently running out there in production, and the availability of the documentation needed to install and operate it. Typically production deployments will lag behind development by a point release or two. This means that if you were running a release of OpenStack two versions behind the latest supported version, finding the latest version of documentation for that older release, was nearly impossible, unless you knew where to look, or were equipped to build the documentation yourself.
To date, many companies providing support for OpenStack, including Canonical, have provided long-term support (“LTS”) of OpenStack running on their supported Linux distributions, if you were running supported release of both components. The burden of providing the “missing” documentation fell upon each distribution and packager to maintain and provide for their customers and clients.
With some effort and discussion with the OpenStack Documentation project team, that gap was closed when the project team agreed to own and maintain the EOL releases of their documentation (the “openstack-manuals”) permanently, starting with the Newton release of OpenStack. The result is that these manuals will continue to be available online, searchable well after Newton and future releases have gone EOL. For more on how this was resolved, checkout a previous article posted to Superuser.
This effort directly helps the operators out in the field who are running these OpenStack versions in production today and in the future. But those manuals will remain static, unchanged, once their last commit has been made. Only security fixes to the underlying Javascript and CSS that makes them functional in a browser will be made.
There is a second repository of material specific to the operators of OpenStack called the “Ops Guide,” and it too was held in its own repository alongside the OpenStack manuals (“ops-guide“). Because there were very few to no contributions made to that guide, it was frozen along with the Newton release of OpenStack.
I’ve taken that body of work, built it using the existing tools within the openstack-manuals repository (Sphinx, extensions and other rendering and transformation tools), converted the rendered HTML those tools produce into Markdown and then converted that Markdown into Wikitext intended for import into the upstream OpenStack wiki, currently powered by Mediawiki.
This wiki is intended to be a live, editable resource for the OpenStack operators out in the field who are running current or older versions of OpenStack, to maintain, update and add resources and content to, to make their collective jobs easier running OpenStack beyond what the “openstack-manuals” tied to each release can provide.
The process of building from the Python RST format documentation in the openstack-manual repository to the final output in Wikitext required “building” the manuals to produce the “ops-guide” rendered HTML output. From there, I took those HTML pages and converted them into Markdown using a tool called “pandoc“. There were a few passes of pandoc to massage the content from HTML -> Markdown -> Wikitext. That process looks something like this:
pandoc -f html+lhs -t markdown-raw_html-native_divs-native_spans -o “${fname%.html}”.md “$fname”
pandoc -f markdown+lhs -w mediawiki “$line” -t mediawiki -o “${fname%.md}”.wiki
You might ask, why do we have to go to Markdown first and then to Wikitext?
There are several subtle, legacy HTML constructs that don’t directly convert well from HTML into native Wikitext, if at all. Through testing I’ve found that using a secondary transformation format as a neutral control gives us the best rendering output for the final version; Wikitext. This includes spans, divs, tables and some other elements. The work to roll those back into the upstream rendered HTML isn’t considered high priority, as the wiki will diverge quickly with changes and edits once it has been made live and released to the operators.
Once the conversion was done, each page was then imported into the OpenStack Operator’s wiki site, and visually adjusted or corrected to provide the best view for visitors and operators/editors. Some of these changes included altering subtle coloring or table formatting, but mostly it was ensuring that the content, sections and headings rendered correctly when the Mediawiki filters and post-processing ran over each page.
The OpenStack wiki content under the OpsGuide section now includes the full, complete, rendered copy of the “ops-guide,” ready for the real-world operators to own, manage, update and maintain going forward.
It’s our hope that the wiki begins to take on a life of its own, and grows organically like the rest of the OpenStack ecosystem.
Author David A. Desrosiers works at Canonical, US Support.
Superuser is always interested in community content – get in touch at editorATopenstack.org
- Taking the OpenStack ops manuals to the next level - February 13, 2018
