Deploying GitLab through Subutai: the quick & easy way


By Subutai Community Manager Felipe Fonseca and Subutai’s GitLab Blueprint creator Marco Silva

Say hello to the GitLab Blueprint

So, you are concerned about the future of your Git repositories, following the huge movements the market has seen this week. You might have heard of another open source alternative, the GitLab community edition. But you don’t want to spend more money to pay for cloud services or spend more time learning how to install and setup GitLab in your server. Here’s where the Subutai GitLab blueprint comes in handy – enabling you to deploy GitLab on your peer-to-peer cloud.

In a nutshell, blueprints are enhanced templates designed to reduce system administration overheads by simplifying the deployment of cloud environments in the Subutai platform. If you are a Subutai Bazaar member, you might have come across blueprints on the Products page. Instead of setting up an environment from scratch, and then manually downloading and configuring GitLab, blueprints do all of these for you in the backend. Want to learn more about the Subutai platform? More info here: https://subutai.io.

Here’s what you need

Subutai users, with Bazaar accounts and peers, can skip this part and go straight to the walk-through below. First time users have to create an account on Subutai Bazaar. To do so, follow the guidelines on our Getting Started page. Just so you know, we are obsessed with security and privacy, so don’t forget to install the E2E browser plugin (all lightweight and open source, of course), which we developed for managing PGP keys. Registration is free and you even get to earn GoodWill, our internal Bazaar token.

With your Bazaar account set up, you can select the peer where you want to deploy the blueprint. From the Tools menu, go to the Peers page where you can select a peer and add it to your Favorites list. You will use that later during the blueprint set up. If you want to use your own peer, here are the instructions and guidelines, https://docs.subutai.io/Products/PeerOS_toctree.html.

Let us walk you through it

Once logged in to Bazaar, it’s just a matter of following these simple steps:

1. From the Tools menu, access the Products page.

Browse through the list on the Products page, where you can gain access to blueprints that can be deployed to any peer running our software, anywhere in the world.

2. On the Application Blueprints tab, click the GitLab blueprint icon or name.

If you want to review the inner structure of the blueprint, before deploying it, click View. Published blueprints come with full specifications so that they can be scrutinized by the community.

3. Click Build.
4. Set your variables.

The variables are quite straightforward, starting with names for the environment and container. Be sure to choose unique names to differentiate from other ones that you have or might want to build later. For the domain name, choose the subdomain through which you will access your GitLab instance. Free subdomains of “envs.subutai.cloud” are available for now. You can enter a new subdomain by clicking Add new. If you want to specify a container size, you may do so. Those who plan on hosting several projects may choose either the Large or Huge size.

The next screen about Ports only informs you that the system will automatically point your selected subdomain to the proper container. You can click Next to continue.

5. Select the peer where you want to run the environment.

Now is the time to decide where to deploy the blueprint. Select the peer that meets the requirements for your environment in terms of resources, geography, performance, and price (in GoodWill).

6. Click Finish.

Wait for the build to finish. You are redirected to the Environments page where you can monitor the progress, as the contents of the blueprint are downloaded automatically and deployed to the container.

It might take a few minutes, but it’s worth it!

7. Access and set up your GitLab admin account.

Want to see it live? Use the domain name that you have set up earlier in the Variables screen. You will see that as a link in the Domain Name column on the Environments page.

When you click the link, you will see the GitLab page where you must set the password for the administrator account or the root user. Once logged in, you may register other accounts.

So, there you go – a functional instance of GitLab deployed on a P2P cloud. You can start transferring your projects and look forward to creating new ones. Enjoy it!

Some important post installation notes:

We recognize that due to constraints about the way the Internet works nowadays, you cannot run your own email server in a sustainable and effective way. Your GitLab instance will work fine without an email server, but email will probably be needed for certain tasks. For instance, when one of your users forgets her password and needs a password recovery link to be sent. Or when the user wants to receive email notifications about changes in repositories they follow. What we recommend is that you create an account on a commercial SMTP server, where you can set up the credentials afterwards. GitLab provides documentation on how to do that here.

GitLab is only one among the various open source software that has been integrated into a blueprint for easy deployment on P2P cloud infrastructure. Pretty cool, right? If you want, you can contribute to our growing blueprint collection. Check out our Blueprint Hackathon. (Hint: We’re awarding GoodWill in exchange for coming up with quality blueprints.)

The Horde Is Coming: Subutai 7.0

by OptDyn CTO and Founder Alex Karasulu

 

The Horde Is Coming: Subutai 7.0

We’re proud to announce the latest 7.0 major version release of the Subutai Platform. We just finished the migration from 6.0 to update all components today, May 10th 2018, on schedule. Please note that 7.0 is not backwards compatible with 6.0. Environments (clouds) running on 6.0 peers will continue to run. Everyone should however upgrade their peers to take advantage of all the benefits of 7.0 and avoid compatibility issues.

Subutai is composed of several different software components. We first started introducing 7.0 features into 6.0 that did not impose compatibility issues. We left the last few changes breaking compatibility towards the end for the last switch over to 7.0.

Subutai 7.0 is the best ever! It’s more stable and performant than 6.0 and well suited for mission critical production environments. Lucky 7, as we call it, is the major version intended to take the community through its token distribution event. It has backend changes to facilitate all the blockchain features needed to enable the dual token scheme use by the platform involving the GoodWill and KHAN tokens.

Here’s a list of the major improvements and features in 7.0:

Subutai PeerOS

  • Abandoned the use of Snap package to use Debian Packages
  • Switch default local CoW file system to ZFS instead of BTRFS
  • Template versioning and concurrent versions support
  • Subutai Agent exposes new REST API for Resource Hosts
  • Desktop as a Service Support for Containers
  • Using hard ZFS quotas instead of soft quotas
  • P2P fast warm up and other performance improvements
  • P2P tunneling and reconnection improvements
  • New environment REST API with CLI (similar to AWS meta api)

Subutai Control Center

  • Lazy installs software components when needed
  • Manages peer lifecycle from cradle to grave
  • Usability enhancements and several bug fixes
  • Using proper LFS paths
  • Desktop as a Service (x2go) Support

Subutai Vagrant Plugin

  • New disk growth command
  • Multiple Vagrant provider boxes for VM peers on several hypervisors
    • Libvirt
    • Virtualbox
    • Parallels
    • VMWare Fusion
    • VMWare Desktop
    • Hyper-V
  • Peer registration commands with Bazaars
  • Resource host registration commands to peers
  • Provisioning support for Subutai Blueprints

Subutai Bazaar

  • Applications management tab for environments
  • Blueprint enhancements to specify cloud distribution topology
  • Usability improvements for several workflows including BP wizard
  • Service Level Agreements as Ethereum Smart Contracts

Subutai E2E Security Plugin

  • Enabled operation with new Firefox API
  • New Microsoft Edge Plugin

Subutai Website

  • New website design
  • Integrated with new readthedocs site

Besides these new features and improvements above (in existing projects) some entirely new projects were emerged:

  • Subutai Ansible Module
  • Subutai Read The Docs
  • Subutai Container Utils
  • Various Debian package projects

The Subutai Ansible Module has been completed and submitted to the Ansible community for inclusion in the distribution: see https://github.com/ansible/ansible/pull/37659. The module can invoke peer CLI commands remotely, provision new peers and provision blueprints. It’s functionality mirrors the commands and functionality present in the Vagrant Subutai Plugin but is geared towards system administrators managing hosted, or cloud peers.

The new documentation project based on read the docs amalgamates documentation stored in github repository wikis as well as product documents kept in Google Docs for the project. It builds an entire table of contents for all the information associated with Subutai, its sub products and its 20+ projects.

The container CLI utilities project manages client tools used to query environment (cloud), peer, and container information. The CLI tool therein handles communicating with a REST API inside each network segment of the virtual private cloud to get information about the hosting peer, and the environment configuration.

As we switched from using a snap to regular vanilla Debian packages, we needed to decompose the snap into them. This produced a few projects just to hold the configuration for each of these new packages

Overall 7.0 opens Subutai to several future enhancements on our roadmap geared towards the Subutai token distribution event. It’s the lucky 7 release that will amass the hordes needed to conquer the cloud. Get started today!

Subutai Engineering Team members Timur Zhamakeev, Sydyk Akhmataliev, Adilet Zholdoshbekov, Aizhan Taalay kyzy, Emil Sulaimanov, Zubaidullo Niimatullo uulu, Eldar Tursunbaev, Azret Kenzhaliev, Bakhtiar Kukanov, Alina Penkina, Ibraghim Muhamedzhar, along with interns Amirkhan Ahetov and Bakhtiar Ayubov, enjoy a moment in the sun after releasing “Lucky 7”. Not pictured: Dilshat Aliev, Mikhail Savochkin, Abdysamat Mamutov, Erkin Matkaziev, Almanbet Sherov, Lars B. Thomsen, Aaron Xu, Fernando Silva, Marco Silva, and Burak Metin.

Unified ReadTheDocs Documentation

AlexKarasulu
SubutaiWarDog is OptDyn CTO and Founder Alex Karasulu

Documentation! It’s unfortunately the neglected “red-headed step child” of developers. Most people don’t get it: a product is useless without solid documentation. Even to this day, most of our developers avoid documenting. “Dude, the code documents everything,” they say. To change the mindset, we need to avoid barriers and make authoring content a pleasure.

Documentation Dilema

I personally love documenting. Even before I start working on a new idea or writing code. It helps get your thoughts together. Authoring documents should be pleasant and effortless for all authors, whether you’re an engineer writing technical specifications for fellow developers or writing product manual for end users.

Documentation started turning into a thorn in our side when it came time to organize all the documents we haphazardly produced over years of working on the Subutai Platform. It stopped being a joy and quickly turned into a headache. Information about the project was spread all over. Some information silos were not easily accessible especially to those outside of engineering. Management wanted Google Docs and refused to use GitHub where most of our developer community used Markdown on our project wiki’s. Practically no one (minus the die hards) wanted to touch the restructureText in our ReadTheDocs content committed to git repositories (these were even less accessible to management). At least the GitHub Wiki pages could be edited by non-engineering staff using the convenient editor GitHub provides, but still we could not please everyone, and no one was happy about the documentation situation.

To make matters worse, in addition to the aim of unifying and organizing all these documents, we were pressed to provide a smooth user experience between our website and our documentation. Meaning visitors should not have any sense of discontinuity while going back and forth from the website through links between our documentation and the main website’s content.

Bringing it Together

Looking for a solution, we naturally googled around and ran into this sweet blog post from Rob Day, Good documentation, ReadTheDocs and Markdown. It got us thinking and was a pleasure to read while seeing what others were doing to unify their documentation. Rob clearly showed that some custom scripting was unavoidable.

Unlike the Clearwater Project, we wanted to unify our markdown documents in GitHub project wikis and Google Docs. We also realized we’re going to have to modify the ReadTheDocs theme in order to make the user experience more fluid. We wanted a thin website, without requiring many changes to it over time. We wanted users to fluidly go back and forth from the ReadTheDocs content generated from Google Docs and Wiki Markdown content which both our non-technical and engineering staff can easily author and maintain.

Google Drive: /readthedocs Folder

We started by defining a global table of contents and the boundaries where developer documentation crosses over to non-technical content. We also created a root folder in Google Drive for the documentation team called /readthedocs. General project information, community content, and resources were put into their own top level sections: General, Community, and Resources. Folders of the same name under the /readthedocs folder were created for these categories. Word docx documents in these folders are downloaded and converted into restructuredText then added to the toctree automatically based on their position.

GitHub Wiki Submodules

We presumed the technical content to be resident in the GitHub project wikis. We setup the Subutai readthedocs GitHub project to import all the sub-project wikis. See all the modules imported here. The project wiki markdown is converted into restructuredText and added to the Projects of the entire documentation site.

We even added some extra organizational features like automatically generating side bar files (_Sidebar.md) for the GitHub wiki if users tag pages with a PARENT_PATH attribute in HTML comments at the end of the markdown page like so:

<!--
PARENT_PATH: /Guides
ORIGIN: github
-->

The comment is hidden but parsed by the documentation system to generate the side bar and commit it back to the wiki git repo. This way the content can be neatly organized with minimal effort in the GitHub Wiki which is mirrored also in the organization of the generated ReadTheDocs content: i.e. look at the side bar of the PeerOS Agent Wiki and the ReadTheDocs documentation generated for the Agent Project.

Custom ReadTheDocs Theme

For the finishing touches, we added a custom Subutai RTD Theme to mirror the Subutai Website menus so back and forth navigation was seamless. Notice also, the blog you’re reading this article from follows the same pattern to facilitate back and forth navigation.

Conclusions

It was a headache to organize and script out all the toctree generation, the markup and docs transformations for everything to work. But it was all worth it. Not only is everything really nicely organized. We can now search all our documentation from one place. Plus the fluid visitor experience using custom RTD theme made it pretty.

Changes to any wiki pages or Google document automatically download, convert, and deploy to keep the unified website and documentation synchronized. Take a look and see the seamless experience for yourself on the Subutai Website and the Subutai ReadTheDocs Documentation Site.

Next up our community wants to make product online help use it. Mike and Dilshat already started writing a Slack bot to help people on our community Slack. Toight!

If you’re interested in setting up the same unified scheme for your own project feel free to fork our readthedocs repository. Here’s our readthedocs documentation for the readthedocs project. Yeah, that sounds funny: we know. Happy documenting!