Back to top


Introduction

In this post, we will talk for one of the most used services on Azure, the Storage Services. Throughout this post, we will learn what a Storage Account is , its architecture, the Azure Storage Services, what are the limitations for this service.

 


Azure Storage Account

The storage account MUST be a globally unique name because this account is accessed from the internet and we can access the data by using public HTTP URL. Under the storage account we can select to use among four services, these are Blob, Files, Tables, and Queues, and for these services, there are three different types of storage accounts we can choose, Blobs, General-Purpose v1, General-Purpose v2.

Architecture

We can access and use the Azure storage services via the Azure Portal or using Microsoft/3rd party tools. But, what happens behind the scene?

As the image below shows there are four layers before the final service.

Notice: Any data object like Blob, Table, Files, Queues is organized in a partition and every partition has a partitioning key. 

The partition key for every object is:

  • Blobs – ContainerName + BlobName
  • Entities – TableName + PartitionKey
  • Messages - QueueName

Layers

  1. Distributed Replication (Partition) Layer: Data replicated across the facility or other facilities in other regions.
  2. ScaleOut And Auto Load Balancing Index (DFS) Layer: This layer offers auto load balancing for the partitions across the servers.
  3. Disk & Blob, Table, Files, Queue (Front End) Layer: This is the Top layer which receives the requests from every service and routes them to the partitions.

 

Back to top


Performance

There are two tiers for storage performance, Standard and Premium.

Standard

This is the most known and usable storage tier for a storage account, where magnetic disks are used at the back end. The cost Per GB for this tier is very low.

Premium

This tier of disks can only be used with Virtual Machines Disks. Premium Disks are backed by SSD (Solid State Disks) and offer consistent and low latency performance. Of course, the cost is higher than a standard tier disk.

Standard SSD

Standard SSD disks were introduced to improve the performance of the Standard HDD disks for an affordable price. Standard SSD Disks can be used with all Azure VM SKUs. For more details on the scalability and performance of the Standard SSD Disks, click here.

Ultra SSD

Ultra SSD (Preview) disks are here to provide all the goodies we need from a storage which are high IOPS, high Throughput and Low latency. This offering is still in Preview and if we want to test it with our deployments, we should fill in a request access survey.

Back to top


Type Of Storage Accounts

Storage v2 (General purpose v2)

General purpose v2 support all the latest features. It supports Blobs, Tables, Files, Queues, and Disks. The general purpose v2 delivers the lowest Per GB prices and industry competitive transaction prices.

Storage (General purpose v1)

General purpose v1 is the standard storage account and provides access to Blobs, Tables, Files, Queues, and Disks. This kind of storage account may not have the latest features or the lowest cost per gigabyte.

Blob Storage

This kind of storage account is the best offer for solutions that need unstructured storing of data, as block blob. It supports block blobs, appends blobs but no page blobs.

 

Back to top


Blob Tiers

There are three available Blob Tiers.

Cool Tier

The Cool Tier is optimized for a large amount of data which are accessed infrequently. It's more cost effective than Hot Tier.

Hot Tier

The Hot Tier is optimized for frequently access, the cost is higher than the Cool Tier and when we create a new storage account this is the default value for this setting.

Archive Tier

The Archive Tier is the most cost-effective option for storing and access the data rarely. When accessing those data the cost is higher than Hot or Cool Tiers.

 

Back to top


Replication

All data in Azure storage accounts are replicated to ensure high resiliency and availability. We can choose the replication technique that suites us, based on our needs and budget.

Locally redundant storage (LRS)

LRS provides high availability by keeping 3 copies of the data in separate nodes, inside a single data-center. LRS provides resiliency for node failures.

Zone-redundant storage (ZRS)

ZRS provides high availability by keeping 3 copies of the data, in separate zones inside a single region. ZRS provides resiliency in case of data-center failures.

Geo-redundant storage (GRS)

GRS provides high availability by keeping 6 copies of the data. 3 replicas are kept in the region where the storage account resides, and the other 3 are in a paired region. 

GRS provides resiliency even in case of a whole region outage.

Read-access geo-redundant storage (RA-GRS)

RA-GRS is similar with GRS in terms of resiliency, but offers read-access to the data in the paired region, so the data remain available even in case of regional outage.

 

Back to top


Disks Type

Unmanaged

Unmanaged disks are managed by Microsoft Cloud Services. In this case, we have a storage account and under this account one or more blobs (e.g vhd's).

For unmanaged disks there is a capacity limit of 500 TB per Storage Account.

Managed

Managed disks became known after the ARM was introduced. With managed disks, the disks management is more simplified than the unmanaged. Also, we can deploy cloud-scale VM scale sets more easily.

Limitations

At the table below we can see the storage account limitations.

Resource  Default Limit 
Number of storage accounts per region, per subscription, 
including both standard and premium accounts
200
Max Storage Account capacity 500TB 
Max number of blob containers, blobs, file shares, tables, queues,entities, of messages per storage account  No limit
Maximum request rate per storage account  20.000 requests per second
Max ingress per storage account (US Regions) 10 Gbps if RA-GRS/GRS enabled / 20 Gbps for LRS/ZRS 
Max egress per storage account (US Regions) 20 Gbps if RA-GRS/GRS enabled / 30 Gbps for LRS/ZRS 
Max ingress per storage account (Non-US regions) 5 Gbps if RA-GRS/GRS enabled / 10 Gbps for LRS/ZRS 
Max egress per storage account (Non-US regions) 10 Gbps if RA-GRS/GRS enabled / 15 Gbps for LRS/ZRS

 

Back to top


Azure Storage Services

There are four Azure Services available:
  • Blob
  • Files
  • Tables
  • Queues

Azure Blob

The word Blob means "Binary Large Object". Using this service we are able to store a large amount of unstructured data. For example, we can store types of files like images, videos, text files, log files, vhd, iso files, etc.

 

Type Of Blobs

There are three types of blobs :
  • Page Blobs: Page blobs are used for frequent read-write operations. Every Page Blob consists of pages of 512 bytes each. The maximum size for a page blob is 8 TB. Page Blobs are used in Azure Virtual Machines, OS and Data Disks.
  • Block Blobs: Block blobs are used for images, video files, etc. Each Block Blob is composed from blocks of data which can be managed individually. Each Block Blob has a maximum size of 4.7 TB.
  • Append Blobs: Append Blobs are similar to Block Blobs, with the difference that are the best choice for append operations like Log Files.

 

Limitations

At the table below we can see the Azure Blob Scale Targets.  

Resource  Target
Max size of the single blob container  Same as max storage account capacity
Max number of blocks in a block blob or append blob 50.000 blocks
Max size of a block in a block blob 100 MB 
Max size of a block blob 50.000 x 100 MB (approx. 4.75 TB)
Max size of a block in an append blob 4 MB
Max size of an append blob 50.000 x 4 MB (approx. 195 GB)
Max size of a page blob 8 TB
Max number of stored access policies per blob container
5
Targe throughput for a single blob
Up to 60 MB  per second, or up to 500 requests per second

Default Endpoint: http://mystorageaccount.blob.core.windows.net

 

Back to top


Azure Files

Azure Files supports the Server Message Block protocol standard (SMB) and gives the capability to create a file share in Azure that can be mounted to an on-premises workstation as a mapped network drive. The Azure Files can be accessed from multiple points.

 

Limitations

At the table below we can see the Azure File Share Scale Targets.  

Resource Standard file shares  Premium file shares (preview)
Minimum size of a file share (no minimum; pay as you go) 100 GB
Max size of a file share 5 TB 5 TB
Max size of a file in a file share 1 TB 1 TB
Max number of files in a file share No limit No limit
Max IOPS per share 1000 IOPS
5120 IOPS baseline / 5.360 IOPS with burst
Max number of stored access policies per file share
5 5
Target throughput for a single file share
Up to 60 MB/sec Up to 612 MB/sec (provisioned)
Maximum open handles per file 2.000 open handles 2.0000 open handles
Maximum number of share
snapshots
200 share snapshots 200 share snapshots

Default Endpoint: http://mystorageaccount.file.core.windows.net

 

Back to top


Azure Tables

Azure Tables is a solid cloud NoSQL storage which is massively scalable. Azure Tables can be accessed via REST API. It is a great solution for cases with a schemaless design. 

Limitations

At the table below we can see the Azure Tables Scale Targets.

Resources Target
Max size of a single table 500 TB
Max size of a table entity 1 MB
Max number of properties in a table entity 255 (including 3 system properties: PartitionKey, RowKey, and Timestamp)
Max number of stored access policies per table 5
Maximum request rate per storage account 20.000 transactions per second (assuming 1 KB entity size)
Target throughput for single table partition (1 KB entities) Up to 2000 entities per second

Default Endpoint: http://mystorageaccount.table.core.windows.net

 


Azure Queues

The Azure Queues storage service is appropriate for storing and exchange messages using HTTP/HTTPS calls. This is a 'low latency with high throughput' system. A message size can be 64 KB and the maximum number of messages is defined from the storage account capacity.

 

Limitations

At the table below we can see the Azure Queues Scale Targets.

Resource Target
Max size of a single queue  500 TB
Max size of a message in a queue  64 KB
Max number of stored access policies per queue  5
Maximum request rate per storage account 20.000 messages per second assuming 1 KB message size
Target throughput for the single queue (1 KB messages) Up to 2000 messages per second

Default Endpoint: http://mystorageaccount.queue.core.windows.net

 

Back to top


Azure Storage Cost

In the following links, we can find all the useful information on how to calculate Azure Storage consumption.

Managed Disk

The formula: Managed Disk Cost = Fixed Cost (Per Disk Size) +  Operations Cost

The Example: Managed Disk Cost = S10 128GB (€4.97) + (200.000 Transaction units * €0.0003 Per unit) = €65.68

Unmanaged Disk

The formula: Unmanaged Disk Cost = (Storage Usage In GB * Cost Per GB) + Operations Cost

The Example: S10 128GB (€4.86) +  (200.000 Transaction units * €0.0005 Per unit) = €91.08 *

 

  Note
*Unmanaged disks billing is based on the used storage space, the example supposes that all 128 GB are used.

The formulas apply on the Standard HDD and Standard SSD disks, while no transaction costs are applied for the Premium SSD disks.

The examples are calculated for Operations Cost = 200.000 Transaction units. Each Transaction unit is equal to 10.000 transactions.

 

Back to top


Useful Links For Storage Cost

 

Back to top


Azure Storage Tools

There are several tools (Microsoft or 3rd Party) that help us to manage Azure Storage Services.

Microsoft Tools

3rd Party Tools

 

Back to top


Conclusion

After reading this post we have a good grasp of Azure Storage Services. We are in a position to know how important is to implement a strategy for the Azure Storage Services usage. Microsoft offers many options that can suit all business needs in terms of performance, data replication and budget, and each of them adheres to its respective service limits and benefits.

 

Back to top


See Also

 

Back to top