Documentation! It’s unfortunately the neglected “red-headed step child” of developers. Most people don’t get it: a product is useless without solid documentation. Even to this day, most of our developers avoid documenting. “Dude, the code documents everything,” they say. To change the mindset, we need to avoid barriers and make authoring content a pleasure.
Documentation Dilema
I personally love documenting. Even before I start working on a new idea or writing code. It helps get your thoughts together. Authoring documents should be pleasant and effortless for all authors, whether you’re an engineer writing technical specifications for fellow developers or writing product manual for end users.
Documentation started turning into a thorn in our side when it came time to organize all the documents we haphazardly produced over years of working on the Subutai Platform. It stopped being a joy and quickly turned into a headache. Information about the project was spread all over. Some information silos were not easily accessible especially to those outside of engineering. Management wanted Google Docs and refused to use GitHub where most of our developer community used Markdown on our project wiki’s. Practically no one (minus the die hards) wanted to touch the restructureText in our ReadTheDocs content committed to git repositories (these were even less accessible to management). At least the GitHub Wiki pages could be edited by non-engineering staff using the convenient editor GitHub provides, but still we could not please everyone, and no one was happy about the documentation situation.
To make matters worse, in addition to the aim of unifying and organizing all these documents, we were pressed to provide a smooth user experience between our website and our documentation. Meaning visitors should not have any sense of discontinuity while going back and forth from the website through links between our documentation and the main website’s content.
Bringing it Together
Looking for a solution, we naturally googled around and ran into this sweet blog post from Rob Day, Good documentation, ReadTheDocs and Markdown. It got us thinking and was a pleasure to read while seeing what others were doing to unify their documentation. Rob clearly showed that some custom scripting was unavoidable.
Unlike the Clearwater Project, we wanted to unify our markdown documents in GitHub project wikis and Google Docs. We also realized we’re going to have to modify the ReadTheDocs theme in order to make the user experience more fluid. We wanted a thin website, without requiring many changes to it over time. We wanted users to fluidly go back and forth from the ReadTheDocs content generated from Google Docs and Wiki Markdown content which both our non-technical and engineering staff can easily author and maintain.
Google Drive: /readthedocs Folder
We started by defining a global table of contents and the boundaries where developer documentation crosses over to non-technical content. We also created a root folder in Google Drive for the documentation team called /readthedocs. General project information, community content, and resources were put into their own top level sections: General, Community, and Resources. Folders of the same name under the /readthedocs folder were created for these categories. Word docx documents in these folders are downloaded and converted into restructuredText then added to the toctree automatically based on their position.
GitHub Wiki Submodules
We presumed the technical content to be resident in the GitHub project wikis. We setup the Subutai readthedocs GitHub project to import all the sub-project wikis. See all the modules imported here. The project wiki markdown is converted into restructuredText and added to the Projects of the entire documentation site.
We even added some extra organizational features like automatically generating side bar files (_Sidebar.md) for the GitHub wiki if users tag pages with a PARENT_PATH attribute in HTML comments at the end of the markdown page like so:
<!-- PARENT_PATH: /Guides ORIGIN: github -->
The comment is hidden but parsed by the documentation system to generate the side bar and commit it back to the wiki git repo. This way the content can be neatly organized with minimal effort in the GitHub Wiki which is mirrored also in the organization of the generated ReadTheDocs content: i.e. look at the side bar of the PeerOS Agent Wiki and the ReadTheDocs documentation generated for the Agent Project.
Custom ReadTheDocs Theme
For the finishing touches, we added a custom Subutai RTD Theme to mirror the Subutai Website menus so back and forth navigation was seamless. Notice also, the blog you’re reading this article from follows the same pattern to facilitate back and forth navigation.
Conclusions
It was a headache to organize and script out all the toctree generation, the markup and docs transformations for everything to work. But it was all worth it. Not only is everything really nicely organized. We can now search all our documentation from one place. Plus the fluid visitor experience using custom RTD theme made it pretty.
Changes to any wiki pages or Google document automatically download, convert, and deploy to keep the unified website and documentation synchronized. Take a look and see the seamless experience for yourself on the Subutai Website and the Subutai ReadTheDocs Documentation Site.
Next up our community wants to make product online help use it. Mike and Dilshat already started writing a Slack bot to help people on our community Slack. Toight!
If you’re interested in setting up the same unified scheme for your own project feel free to fork our readthedocs repository. Here’s our readthedocs documentation for the readthedocs project. Yeah, that sounds funny: we know. Happy documenting!
Subutai Blog 