We’re proud to announce the latest 7.0 major version release of the Subutai Platform. We just finished the migration from 6.0 to update all components today, May 10th 2018, on schedule. Please note that 7.0 is not backwards compatible with 6.0. Environments (clouds) running on 6.0 peers will continue to run. Everyone should however upgrade their peers to take advantage of all the benefits of 7.0 and avoid compatibility issues.
Subutai is composed of several different software components. We first started introducing 7.0 features into 6.0 that did not impose compatibility issues. We left the last few changes breaking compatibility towards the end for the last switch over to 7.0.
Subutai 7.0 is the best ever! It’s more stable and performant than 6.0 and well suited for mission critical production environments. Lucky 7, as we call it, is the major version intended to take the community through its token distribution event. It has backend changes to facilitate all the blockchain features needed to enable the dual token scheme use by the platform involving the GoodWill and KHAN tokens.
Here’s a list of the major improvements and features in 7.0:
Abandoned the use of Snap package to use Debian Packages
Switch default local CoW file system to ZFS instead of BTRFS
Template versioning and concurrent versions support
Subutai Agent exposes new REST API for Resource Hosts
Desktop as a Service Support for Containers
Using hard ZFS quotas instead of soft quotas
P2P fast warm up and other performance improvements
P2P tunneling and reconnection improvements
New environment REST API with CLI (similar to AWS meta api)
Subutai Control Center
Lazy installs software components when needed
Manages peer lifecycle from cradle to grave
Usability enhancements and several bug fixes
Using proper LFS paths
Desktop as a Service (x2go) Support
Subutai Vagrant Plugin
New disk growth command
Multiple Vagrant provider boxes for VM peers on several hypervisors
Peer registration commands with Bazaars
Resource host registration commands to peers
Provisioning support for Subutai Blueprints
Applications management tab for environments
Blueprint enhancements to specify cloud distribution topology
Usability improvements for several workflows including BP wizard
Service Level Agreements as Ethereum Smart Contracts
Subutai E2E Security Plugin
Enabled operation with new Firefox API
New Microsoft Edge Plugin
New website design
Integrated with new readthedocs site
Besides these new features and improvements above (in existing projects) some entirely new projects were emerged:
Subutai Ansible Module
Subutai Read The Docs
Subutai Container Utils
Various Debian package projects
The Subutai Ansible Module has been completed and submitted to the Ansible community for inclusion in the distribution: see https://github.com/ansible/ansible/pull/37659. The module can invoke peer CLI commands remotely, provision new peers and provision blueprints. It’s functionality mirrors the commands and functionality present in the Vagrant Subutai Plugin but is geared towards system administrators managing hosted, or cloud peers.
The new documentation project based on read the docs amalgamates documentation stored in github repository wikis as well as product documents kept in Google Docs for the project. It builds an entire table of contents for all the information associated with Subutai, its sub products and its 20+ projects.
The container CLI utilities project manages client tools used to query environment (cloud), peer, and container information. The CLI tool therein handles communicating with a REST API inside each network segment of the virtual private cloud to get information about the hosting peer, and the environment configuration.
As we switched from using a snap to regular vanilla Debian packages, we needed to decompose the snap into them. This produced a few projects just to hold the configuration for each of these new packages
Overall 7.0 opens Subutai to several future enhancements on our roadmap geared towards the Subutai token distribution event. It’s the lucky 7 release that will amass the hordes needed to conquer the cloud. Get started today!
Subutai Engineering Team members Timur Zhamakeev, Sydyk Akhmataliev, Adilet Zholdoshbekov, Aizhan Taalay kyzy, Emil Sulaimanov, Zubaidullo Niimatullo uulu, Eldar Tursunbaev, Azret Kenzhaliev, Bakhtiar Kukanov, Alina Penkina, Ibraghim Muhamedzhar, along with interns Amirkhan Ahetov and Bakhtiar Ayubov, enjoy a moment in the sun after releasing “Lucky 7”. Not pictured: Dilshat Aliev, Mikhail Savochkin, Abdysamat Mamutov, Erkin Matkaziev, Almanbet Sherov, Lars B. Thomsen, Aaron Xu, Fernando Silva, Marco Silva, and Burak Metin.
Easily manage your Subutai Cloud from your desktop with the Control Center!
The Control Center is an operating system tray application that integrates Subutai natively onto your desktop. It provides a single menu to access all of Subutai’s major features.
From within the Control Center, you can start a virtual desktop session that works with Subutai’s Desktop-as-a-Service (DaaS) feature or SSH into one of your environments’ containers.
Control Center works with your Bazaar account to integrate Subutai onto your desktop. You can manage your peers, and environments natively with Control Center while tracking who’s paying you for your resources and how much GoodWill you’ve earned.
We are pleased to introduce a new feature with the latest update: Peer control management from within the app. With this cool new feature you can create your own Subutai Peer and manage its lifecycle to keep earning Subutai™ GoodWill.
How it works
Control Center uses Vagrant which is a tool for building and managing virtual machines. Subutai supports many hypervisors thanks to Vagrant. Presently only VirtualBox has been integrated into Control Center but it works across all popular operating systems. In the coming weeks we will be adding support for the following hypervisors:
VMWare Fusion (only Mac)
VMWare Desktop and Workstation (Linux, and Windows)
Congratulations! Now you have your own Subutai Peer installed by Control Center with help of Vagrant. Now you can set the price for your peer, change configuration, or destroy it (and install a new one or several new peers).
Even before Bitmain confirmed the availability of their Ethereum ASIC Miners in July, community developers had already started working on modifications to their Ethash algorithms to render the product dead on arrival.
These ASIC miners will be utterly useless before the first customer takes the device out its packaging, if it even ships at all. This will go down in the annals of crypto-mining history as one of the most expensive blunders of all time. How could Wu and Zhan not see this coming?
GPU Miners Live On
The “Bitmain Stillbirth” will prove, once and for all, that hardware flexibility to adapt to algorithmic changes is just as important as raw hash rates. GPU based miners with the ability to load code that is executed by thousands of SIMD processors live on. The threat of ASIC based mining is finally going to be over. The Monero developers are even planning on mutating their algorithm every six months to prevent the chance of ASIC miners gaining any foothold whatsoever.
Stock market traders may be interested in buying back into NVIDIA and AMD. After Analysts reduced their outlook for GPU producers, their stocks dropped slightly. Once the realization of ASIC miner hopelessness permeates, the stock price of various GPU manufacturers should go back up, but with a vengeance. This does not bode well for the environment since GPU’s are notorious power guzzlers. As the number of enthusiast miners grow, at geometric rates, we’re looking at a serious ecological problem.
The “Middle Way” – FPGA Miners
ASIC miners are fast and power efficient, yet inflexible. GPU miners are slower than ASIC miners and flexible, but they consume much more power: several orders more per hash verification.
If Buddha were around and into crypto, he would probably advise, “Take the Middle Way, use FPGA miners”. An FPGA is essentially an ASIC that can be reprogrammed. Performance wise, there’s a slight penalty for being programmable. Approximately 20-50% performance is sacrificed for this flexibility. In terms of power consumption, ASICs and FPGAs are pretty much the same.
FPGA Feasibility: Hash Rates
Most FPGA platforms possess sufficient computing power. Up until recently, the problem with FPGA miners resided in memory bandwidth. Ethereum’s Ethash algorithm is memory hard, meaning the upper bound of hash calculations require a certain amount and bandwidth of RAM. Then new kinds of memory started to emerge making the endeavor completely feasible.
Hybrid Memory Cube chips emerged three years ago to boast a bandwidth of 160 GB/s. The memory technology let’s an FPGA handle up to 20 MH/s on a single interface with less pins reducing the routing complexity. It’s impressive yet certain problems arise because the chips reside outside of the FPGA. Chip to chip communication requires several gigabit transceivers which increases the cost of the FPGA. Regardless the technology is prevalent and experiments show promising results.
Just about a year ago Xilinx introduced in-chip High Bandwidth Memory on their Virtex FPGA series with a default capacity of 8 GB. HBM has a bandwidth of 460 GB/s and a single FPGA can theoretically support hash rates of up to 58 MH/s. The most beautiful aspect to all this is that it requires no high speed (gigabit) transceivers or even controllers with access mechanisms built into the FPGA.
FPGA Feasibility: Cost
Everyone that knows anything about FPGAs knows they’re not cheap. This is due to a limited market, and the need to recover investment and set up costs (research, development, and manufacturing) to produce them. The materials and operating costs required to produce a single chip is relatively small. Hence making 100 chips costs pretty much the same as producing 1,000 or 10,000 chip batches.
Prices shift when markets grow and marginal product revenue plays a big part in these pricing decisions. If the N-th unit of product produced is of negligible cost, then a producer can lower prices to accommodate a wider consumer base and dramatically increase revenues. As an example, perhaps 10,000 consumers buy the product if it costs $1,200 and pays for itself in 10 months. If the price drops to $320 and the product pays for itself in 2 months, then you could have 10,000,000 or more consumers buying the product. These numbers are completely feasible and the difference is $12M vs. $3.2B in revenue. Our relationships and partnerships show the right price points will come with the emergence of FPGA mining thanks to the Subutai Blockchain Router.
The Subutai Blockchain Router v2.0 offers advanced P2P cloud router and IoT gateway functions using the world’s most powerful FPGA MSoC, the Xilinx Ultrascale, packed with four 64-bit A53 ARM cores with up to 16GB of DDR4 RAM capacity.
History and Vision
It’s very important to understand the past, the forces shaping our direction, the reasons behind the decisions made and how they shaped our vision for the router. The past and the future are both intertwined together. This is a chronological breakdown of how it all began and why.
Back in 2013, while developing the Subutai P2P Cloud Software, now together referred to as the PeerOS, we began leveraging software defined networking in addition to other less appealing precursor technologies, like Universal Plug and Play to try to “coordinate” with the hardware. Various optimizations required working with networking hardware on the edge. Synergy between the networking hardware and the fog software was extremely promising and the emergence of various network virtualization efforts gave us a glimmer of hope.
Our problem was unique however. Because of the sheer diversity of the network hardware devices found on the edge, we did not know what to expect, and realized that at most a common denominator of capabilities could be relied on. This was limiting and especially disappointing when some common functionality like uPnP is essentially insecure and dangerous.
We quickly realized that our grand unified vision for fog computing and IoT demanded more advanced functionality above and beyond what the average router and gateway offered on the edge. For starters cloud multi-tenancy imposed stringent security requirements along with the need for secure boot infrastructure that starts with a trusted platform environment. The multi-tenancy rabbit hole goes deep to include things like network QoS requirements across the environments of tenants running across the nano centers we call peers. On the IoT gateway side, enabling the connectivity of the “Things” requires not only wireless but wired connectivity and that entails being able to attach various peripheral connect buses between them and the gateway. The advanced security measures for mission critical systems also required hardware acceleration to even be remotely feasible.
Around 2014, we decided to make a daughter board for an existing platform, the Parallella, to build an advanced cloud router and IoT gateway reference device upon which we could develop a specification to guide hardware vendors. In 2015, we manufactured a few engineering units of this board that we called the Subutai Liquid Router. Liquid because it was flexible with programmable logic along side a processing system using a Zynq-7000 series SoC.
The images above show the daughter board for the Parallella. Yes the daughter board looks more like a mother board instead, being much larger than the Parallella that attaches to it. Notice the switch and ethernet ports along with the Artix 7 FPGA which communicated with the Parallella through it’s LVDS lines via four Samtec connectors. The Parallella had no viable high speed disk interfaces so we added a eMMC RAID controller and multiple eMMC module slots visible in the upper left hand side of the first board image above.
What is an FPGA? MPSoC?
For those not familiar with FPGA technology, the acronym stands for Field Programmable Gate Array. These chips contain a mesh of electronic parts that can be re-programmed to perform different operations. FPGA devices may even be re-programmed after being purchased and put into the field, hence the “Field Programmable” part of the acronym.
An FPGA MPSoC is hybrid multi-processor chip with an FPGA combined. It usually has 2 or more processors with high speed in chip interfaces like AXI to communicate with circuit designs in the FPGA.
Approaching End of Life
All-in-all we had fun and were able to conduct several tests and experiments using this daughterboard design. Many experiments led us to conclusions that helped mould our security framework which is now being published. The Liquid Router was however severely limited in terms of IO bandwidth and the total amount of resources available. Several flaws resulted from a daughter board design. We knew the Liquid Router did what it could for our research, but we had to move on to a new board design without being dependent on another board as a add on.
Some of our research into hardware security mechanisms required using a MiniITX Zynq-7100 to have enough FPGA resources when a fun side project by our engineers turned into a discovery with vast potential. They essentially implemented a miner in the FPGA to operate on a private Ethereum blockchain where they were able to fit the DAG into the limited 1GB of RAM available on this device. It was able to mine at 3-4MH/s due to bandwidth limits but it only consumed approximately 10-watts of power.
At around the same time, we started to see the prices of the next family of Xilinx SoC’s, the Ultrascale+ MPSoC chips, start to drop dramatically. We started designing a new router based on the Ultrascale+ hoping prices would drop considerably to support industrial and commercial applications by the time our design was ready.
Meanwhile working with our partners we realized the availability of more transceivers combined with new memory controller designs using hybrid memory cube serial DRAM with 160GB/s bandwidth which dramatically increases mining hash rates. Tests and theoretical limits now showed hash rates of 20-21 MH/s using HMC memory in the upcoming Residential Edition of the Blockchain Router.
HMC memory is available today, however we’re setting our sights on a new kind of High Bandwidth Memory (HBM) DRAM soon to be integrated directly into the MPSoC family. A standard 8GBs of HBM at 460 GB/s bandwidth is already being packaged inside early FPGA chips themselves. It’s just a short period of time before we see HBM inside the MPSoC family and this could result in hash rates of 58 MH/s.
These possibilities influenced our design directions on both the Industrial and Residential editions of the new router design which we code named the Blockchain Router family of OptDyn products.
Industrial Edition Production Begins
The industrial edition of the Blockchain Router is designed specifically to target high end mission critical IoT applications. The FPGA is used to host hardware security circuitry to augment the expected broadband router functions. We’ve released the security architecture white paper to coincide with the availability of the router for the development of industrial process control applications requiring the highest degree of security and reliability.
Flexibility at its Core
Although more information about the features of the router is available on the specification page, I wanted to stress the compatible expansion headers and shields supported. Specifically 4 PMod headers, a Raspberry Pi header, and an Arduino hat is available.
Almost every detector/sensor, actuator, working with shields for these platforms found at SparkFun Electronics is compatible with the Subutai Blockchain Router. Virtually any new peripheral connect bus could be added to it. In this respect, the Blockchain Router truly surpasses any other IoT gateway on the market. It’s a laboratory and IoT development platform in its own right. Note that this was one of our primary goals when starting out on the design and we’re really excited about it.
Differences Between Editions
The Subutai Blockchain Router’s industrial grade configuration differs from the residential configuration by using FPGA MSoCs operating between extreme temperatures (-40C to 100C) and having a modified memory architecture on the programming logic (PL) side based on HMC DRAM instead of DDR4. The model of the MPSoC’s differs with more transceivers and cross section bandwidth to handle the requirements of Ethereum’s memory hard mining Ethash function. Most importantly though, the FPGA fabric on the industrial grade MPSoC is used for IoT security functions rather than cryptocurrency mining. The Subutai Mission Critical IoT Hardware Security Architecture defines these security functions and their use in mission critical industrial automation applications.
In October 2017, ARM published its Platform Security Architecture which serves as a guidance with primitives to be used. The PSA goes a long way towards fixing the IoT security dilemma coming with the tsunami of devices and data. OptDyn’s framework compliments these primitives and proposes concrete mechanisms for hardware based real time intrusion detection and prevention systems facilitated by IoT gateways like the Subutai Blockchain Router.
Ready for Production Use
OptDyn’s Subutai Blockchain Router takes its place as the high end IoT gateway device in the Caninos Loucos project serving as the foundation of Brazil’s national IoT standard. Professor Zuffo of the University of Sao Paulo spearheads the project,
“We’re really excited about the Subutai Blockchain Router and its industrial configuration with its mission critical security subsystems,” said Dr. Marcelo Zuffo, who spearheads the project at the University of São Paulo, and leads the national IoT effort for Brazil. “The Subutai Blockchain Router, as a secure IoT gateway, has the ability to enable everything from Industry 4.0 to Smart Cities in Brazil. We’re very excited to complete this last part of the puzzle thanks our collaboration with OptDyn, Inc.”
“Mission critical IoT systems don’t have the option of learning from failures where lives and property are at stake,” said Professor Miguel Matrakas, manager of Celtab, one of the Research Centers of the Itaipu Technological Park (PTI; a foundation responsible for promoting scientific, technological and innovation development linked to Itaipu Binacional, the largest clean energy generator in the world), who surveyed the Subutai Blockchain Router v1.0 at Latinoware with Hall and OptDyn CTO and Founder Alex Karasulu. “The Subutai Blockchain Router and its industrial configuration provide the crucial hardware security infrastructure needed for IoT applications in the utility sector.”
“We’re impressed with the innovation and level of performance possible for numerous Smart City IoT projects using the Subutai Blockchain Router,” said Bruno Freitas, Head of the Innovation Unit for the State of São Paulo, Brazil. “The benefits of using this technology —which is being manufactured in São Paulo as part of the Brazilian national IoT program— are apparent. We look forward to considering the Subutai Blockchain Router as a viable option to meet the demands for IoT projects that build an intelligent and prosperous state and cities.”
The Residential Edition of the Subutai Blockchain Router is slated to go into production later this year after the Industrial Edition enters the market and we’re very excited about it’s ecological impact as an alternative to GPU based mining. It’s coming at just the right time, and offers greater flexibility over hardwired ASIC solutions. We also see the Blockchain Router opening the cryptocurrency mining market to FPGA vendors as well as ISP’s interested in providing them to their customers.
The primary cost of production for FPGA/SoC manufacturers is the startup costs of R&D and setting up manufacturing facilities. The difference between making 10 million versus 100 million chips is negligible. The high costs of FPGA SoC’s on the market is to control prices to recoup these initial startup costs. With a larger market for their chips FPGA manufacturers will naturally produce more and drop prices to earn more in the end. It’s all about marginal product revenue.
The main winner however is the environment. Thankfully only a small minute fraction of the world’s population is mining as we speak. Imagine the strain on the power grid when even 5% of the population starts to mine for cryptocurrency? If we can reduce the power consumption by even 100% the environment wins. We know we can reach higher levels of power efficiency up to 1083% when compared to existing GPU cards.
Tomas Evensen, Xilinx CTO, sees a great future for low power FPGA based mining to prevent the needless waste of electricity from power hungry GPU mining devices, “With the proper FPGA design, custom hash generating cores in the SoC can calculate hashes faster with fewer clock cycles and require much less power than using software running on mainboard CPUs or GPUs in graphics cards,” said Tomas Evensen, CTO Embedded Software at Xilinx. “We’re excited to see the Subutai Blockchain Router use our FPGAs for the consensus layer of distributed ledger technologies —while also protecting the environment.”
We intend to work closely with Xilinx and our partners to develop even more efficient editions of the Subutai Blockchain Router’s Residential Edition with a healthy pricepoint to compete with the costs of ASIC and GPU based mining products. We’re especially looking forward to the introduction of HBM DRAM in the MPSoC family of Xilinx products to raise the hash rates to a jaw dropping 58 MH/s.
What about ASIC Miners?
If you’ve ever felt bad for crumpling a clean sheet of paper after a mistake with indelible ink, then you can imagine the waste for mistakes when designing ASICs. Once burnt onto the silicon there’s no going back. The algorithm design cannot be improved, new blockchain algorithms cannot be supported.
For these and other reasons the process of designing and mass producing ASICs are still very high. Using already manufactured FPGAs does not incur that startup cost while FPGAs can be reprogrammed to improve efficiency even after consumers purchase the product. The first consumers are going to have to absorb the start up costs of these ASICS. We’re already seeing suggested prices of two to three thousand dollars for Bitmain products.
In terms of power efficiency FPGAs and ASICs are comparable. In terms of performance ASICs exhibit 20-50% higher performance. This is offset by the differences in flexibility which is a central theme of the Subutai Blockchain Router.
Also remember there’s no router functionality or multi-purpose function to these devices. The Blockchain Router is a cloud router and IoT gateway first and foremost. Our aim is not to just mine crypto. In fact, it’s never was our primary objective. That’s something the router can do on the side while handling its traditional router and gateway functions. Even if new blockchains with different mining algorithms emerge, the Blockchain Router can be reprogrammed to work with them so consumers will not be stuck with yet another useless electronic device that costs too much. Even if mining capabilities are disabled altogether the router can still use the fabric for its advanced security features.
What is the problem with the CLOUD Act, and how Subutai is the answer …
Before the CLOUD Act
Before the CLOUD Act the US government could ONLY get its hands on the data US companies kept on servers inside US borders. If a US company hosted data outside, say in the EU, the US government was unable to access this information using a warrant or subpoena. Accessing data from servers in other countries required treaties and understandings with these governments.
After the CLOUD Act
The CLOUD Act allows ALL the data hosted on servers inside and outside of the US by American corporations to be accessible to US authorities via warrant or subpoena. Additionally this information, associated with US and non-US citizens can be shared with other governments without congressional approval which bypasses national privacy laws in the US and of other nations.
What does this mean for Cloud Computing?
It’s ironic that they used a title for the act whose acronym (almost) spells out CLOUD: the CLarifying Overseas Use of Data Act. Whether or not the data being hosted by US companies outside of the US is stored technically in a cloud does not matter. The data could just be sitting on a disk drive of a server in a foreign state. However the reason for trying to force the CLOUD acronym is clear, the act impacts the cloud industry more than any other since their cloud resources are used predominantly to host applications and data overseas.
This is really bad news for cloud providers with centralized control and administration of their cloud. This should scare companies using their services in the US and outside even more than ever. Let’s take each case separately with the largest fake cloud provider “AcmeZoo, Inc.” which has regions all over the world. Up until now AcmeZoo was able to get business abroad because of laws preventing the US government from accessing data in regions outside of the US. The government of Brazil might have even considered hosting applications in AcmeZoo’s São Paulo region. They might have even seen it as OK, to pass privacy laws that have Brazilian companies hosting the information of Brazilian citizens in AcmeZoo data centers as long as they were in Brazil. With the CLOUD Act all this changes. It no longer matters where that data is stored, AcmeZoo being a US based company is required to hand the data of Brazilian citizens over to US authorities which can basically share it with any other nation now.
What does this mean for P2P Cloud Computing and Subutai?
This is great news for Subutai and couldn’t have happened at a better time. Subutai is the solution that everyone now certainly needs after the CLOUD Act has passed.
Subutai is a Peer-to-Peer (P2P) Cloud platform. It lets anyone buy or sell resources from anyone else to create private cloud environments across edge computers or data center servers. It is completely in the control of the cloud owner. The owner provides governance rules to specify who and from where they are willing to purchase resources and at what cost. There are several parameters that can be used to control your private cloud as it perpetually looks for resources to maintain your desired state for it.
Can’t US authorities get cloud data from OptDyn, a US based company?
Short answer NO.
OptDyn is a US-based company and is the creator of Subutai. OptDyn runs the Subutai Bazaar which is a marketplace with a reputation system and tracker for peer resources. It provides services to help clouds find resources satisfying governance rules around reputation and parameters such as uptime etc.
If the US government required this information it could get it without a subpoena. The best they can do with a warrant is get the names of people and the environments they run. They cannot access data in your P2P cloud environments. First of all OptDyn does not own those servers, others do in the ecosystem. Furthermore you keep your keys to your cloud. It cannot be accessed through OptDyn without doing those operations with your keys when interacting with the Bazaar. These keys are kept and managed in your browser using the Subutai E2E Security plugins.
When using Subutai, you can protect yourself from the CLOUD Act by running on peers owned and operated on servers owned by non-US companies. Subutai has all the governance rules to allow you to restrict exactly where you cloud will run. This is why it was created in the first place as a first class Infrastructure as a Service Cloud and IoT platform.
If you’re a Brazilian Government Agency you should stop using AcmeZoo, Inc. for cloud services from their São Paulo region. Your data is no longer safe there and can be traded to other governments thanks to the CLOUD Act. You can use Subutai with local Brazilian hosting providers under Brazilian law and jurisdiction instead to get an even more resilient government cloud or also leverage your own data centers in combination. This will guarantee that your data will never get into the hands of others you don’t want it in.