Thick-provisioned IOPS in the Public Cloud

How can thick and thin provisioned IOPS help with cloud capacity? Public cloud providers like Amazon Web Services (AWS) and Microsoft Azure are well-known for their ability to deploy fast and elastic infrastructure at the drop of a hat. Compared to the capital expenditure, equipment lead time, and work that it can take to build an analog on-premises solution, the public cloud is the only feasible option for many customers.

As businesses shift selected workloads or move entirely to the public cloud, a few key differences between the familiar on-premises paradigm and a cloud-based environment become apparent. Here are a few of these concepts, as well as how Pure Storage® can help mitigate—or even eliminate—some of these problematic differences. 

Pure Storage and cloud storage paradigms have continued advancing, with updates that further enhance flexibility and efficiency for IOPS management in the public cloud:

• Dynamic IOPS Performance Scaling in Azure Native Cloud
  Azure now allows users to seamlessly adjust provisioned IOPS and bandwidth on-demand—via the Azure Portal slider—all without downtime—making it easier to align performance and cost with workload needs.
• Cloud Block Store Adoption and Thin‑Provisioning Gains
  Cloud Block Store from Pure Storage continues to bridge cloud limitations, enabling thin provisioning not only of capacity but also of IOPS and bandwidth—eliminating the traditional need to over‑provision native cloud volumes and helping customers optimize spend and efficiency.

These enhancements reflect the evolution toward more agile, cost-effective storage performance provisioning for enterprise workloads in the public cloud.

Thick vs. Thin Provisioning in the Cloud

One condition of the agility provided by the public cloud is that resources are charged when allocated regardless of how much or how little is actually consumed. Virtual machines and native cloud block storage options are cloud data center resources that are dedicated to the requestor, so it makes sense that payment for the reservation is expected. Storage administrators are used to this idea via thick vs. thin provisioning from a capacity perspective.

A thin-provisioned volume will have a maximum limit of data that can be stored, but space is only utilized/reported when data is actually written. Thick provisioning, on the other hand, means that the entire volume’s capacity is allocated upon creation and data can then be written to the empty volume until it fills up.  

Leveraging that concept, there is a key difference for the public cloud: IOPS and bandwidth are usually thick provisioned. When we think of an on-premises SAN array, both storage IOPS and bandwidth are typically consumed on demand, are shared by workloads, and generally max out based upon QoS limits or the hardware capabilities of the SAN array or some other artificially set value. Conversely, for the public cloud, many native block offerings need to provision IOPS and bandwidth as set values during creation (see screenshot below) or pick from pre-set disk tiers with those quantities rigidly defined.

For bursty applications or workloads, where IOPS/bandwidth consumption is high but only for a brief amount of time, this leads to wasted spend.

Figure 1: Provisioning Capacity as well as IOPS and throughput for an Azure Ultra Disk.

To illustrate this concept, consider an example scenario where you have two virtual machines that have been recently migrated to the cloud from an on-premises data center. One of the workloads needs 20,000 IOPS for about two hours a day and the other workload needs 18,000 IOPS for around the same amount of time to execute some larger-scale batch jobs. The challenge of this admittedly extreme thought experiment is that the customer needs to provision 20,000 IOPS for VM1 and 18,000 IOPS for VM2 for each of these VM’s native block storage offerings. The actual usage vs. provisioned amount of IOPS is visually depicted in the diagram below.

Figure 2: Daily IOPS requirements and consumption for two example VMs. 

Everything in the patchwork red in the diagram above is provisioned IOPS that the customer is paying for but not utilizing. In fact, these IOPS which aren’t being utilized at all add up to about 75% of the monthly spend. AWS or Azure simply don’t have the ability to dynamically burst IOPS to their native block storage for certain offerings. This example scenario is only for two VMs as well, so imagine how difficult and wasteful it can be for dozens or even hundreds of workloads with bursty IOPS or bandwidth behavior.

This is where Pure Storage Cloud can provide some truly unique value to the public cloud. Pure Storage Cloud is built on top of native AWS and Azure compute and storage and has been purpose-built to more closely emulate what customers have traditionally seen from on-premises SAN arrays. That is, not only thin-provisioned capacity but also thin-provisioned IOPS and bandwidth. Any single Pure Storage Cloud volume has 100% of the underlying array performance available to it and many different VMs and workloads can be attached for further consolidation.  

Pure Storage Cloud has nearly all of the same features as our on-premises FlashArray™:  space-efficient snapshots, the ability to copy a snapshot to a volume almost instantaneously (no need for multiple copies of the same volume to ensure durability), thin provisioning, always-on compression, deduplication, encryption of your data, and highly efficient replication technologies that will make your ingress and egress charges between regions and availability zones less. The value proposition of Pure Storage Cloud is tough to ignore: We provide a rich set of enterprise features while also saving our public cloud customers money as we’re able to better utilize and allocate underlying AWS and Azure resources, particularly at scale.

Check out the following video to learn more about Pure Storage Cloud. 

For further documentation and technical demos, visit the Pure Storage Cloud page.