Azure Storage Deployment Models

First, we need to understand what is Azure Storage Service and how we can use it.

Azure storage is a cloud storage systems that enables organizations the ability to store seemingly and limitless amounts of data for any duration of time. On Azure the data is stored durably using both local and geographic replication to improve and facilitate disaster recovery.

Azure storage is a service that can be used to store partially structured and unstructured data. There are many ways to use Azure Storage as a service. Usually, and what I have seen using the majority of the times is to host App Service, PaaS cloud services or even to facilitate data exchange between components of Azure based solutions. On the other hand, for IaaS is heavily used to store the Azure VM disks or even to host network file shares. Besides these examples exist a lot more scenarios that you can leverage the Azure Storage Service. Although, Azure Storage offers four types of storage services:

  • Blobs – These typically represent unstructured files such as media content, virtual machine disks, backups, or logs. Blobs facilitate locking mechanism, ensuring exclusive file access that IaaS virtual machines require. There are three types of blobs.
    • Block blob, is optimized for sequential access, which is ideal for media content.
    • Page blob, offers superior random access capabilities, which is best suited for virtual machine disks.
    • Append blob, applies to data append operations, without the need to modify existing content. This works best with logging and auditing activities.
  • Tables – These host nonrelational and partially structured content, which consists of multiple rows of data with different sets of properties. In the context of Azure Table storage, these rows are referred to as entities. Developers frequently implement table storage as the backend data store for App Service or PaaS cloud services.
  • Queues – These are temporary storage for messages that Azure services commonly use to asynchronously communicate with each other. In distributed applications, a source component sends a message by placing it in a queue. The destination component works though the messages in the queue one at a time.
  • Files – Like blobs, these provide storage for unstructured files, but they offer support for file sharing in the same manner as traditional on-premises Windows file shares.

Storage Accounts

To start to use the Azure Storage service you need to create a storage account. There are two tiers of storage accounts: Standard and Premium. Premium Storage accounts offers superior performance because of its reliance on solid-state drive (SSD) technology. Although, these storage account are strictly for page blob storage. Standard Storage account uses traditional hard disk drives.

Storage accounts can be created for different purposes.

  • Dedicate a storage account for maintenance of master images to be deployed to other storage accounts.
  • Dedicate a storage account for backup, separate from production data and in a separate region than the primary data.

Storage Account Design

There is no magical formula to design the storage account in Azure. Although, there are some considerations that you are good idea to take in count:

  • Storage accounts have many variables
  • Establish a Storage “Stamp” model
  • Determine initial storage account footprint based on workload requirements
  • Determine process and policy where a new stamp would be created or extended
  • Typical “stamp” would include these storage accounts:
    • LRS
    • GRS
    • Premium Storage

Beside all the points above to take in consideration, there are some good practices when it comes to design the storage solution that you want to use in Azure.

  1. Storage Costs – Type of data will be cost driver; critical data might need GRS whereas less critical data may suffice with just LRS. Quick Access Data with high IOPS will drive the usage of Premium Storage, while lower requirements may accept Standard Storage.
  2. Containers for Organizing Data – Use containers like how a folder structure in a file server would be used. By default, all VHDs will be put into the “vhds” container in a storage account. However, this destination can be changed.
  3. Concurrency Settings – Blob service allows you to specify optimistic or pessimistic concurrency to manage access to blob and containers. Last write winds by default. For IaaS, concurrency settings do not need to be modified. While for PaaS, developers need to take concurrency into consideration depending on the app and user base data types
  4. Storage Account Limits
    1. Primary Constrain: Amount of VHD files a single storage account allows and storage account requests limits.
    2. Basic Tier VMs: No more than 66 highly used VHDs in a storage account. Only allowed 20,000/300 request rate limit.
    3. Standard Tier VMs: No more than 40 highly used VHDs in a storage account. Only allowed 20,000/500 request rate limit.

Convention Names versus Tags in Azure

Sometimes when I am working on an Azure Foundation project, usually they ask me “What is the difference between Convention Names and Tags?” followed by “When we should use each one?”

So, I decide to write this post to try to help to clarify what is the difference and when you should use each one.

Names

For more information about Naming Convention on Azure, see this previous post (here). Although naming the resource that you are creating is mandatory. You should have a very clear strategy to easy identify the resource and address all the requirements that a resource name must have.

  • Names are the resource/object name
  • They are required on the creation (it’s a mandatory field)
  • Names are surfaced at the top level of the portal
  • Names are used in PowerShell.

Tags

Regarding Tags, it’s not mandatory when you create a resource, but is highly recommend that you combine both. If you want more information about Tags, see my previous post (here).

  • Tags are metadata for the resource/object
  • Names/Tags can be used for billing drilldown
  • Name/Tags can be used for data analysis
  • Tags can be used for lot more scenarios then names, the reason why is each resource/object can have up to 15 different tags
  • Tags are used to provide context that a name cannot

When to use Resource Tags in Azure

First, Tags in Azure, and almost in any other platform/application, are used to help you organize in a logical way, all the resources. So, if you have a model of subscriptions like Tier 0 (see this post) and then you have different subscriptions, where you will have all the resources (for example Production, Test and Development subscriptions) how can you organize in a logical way? For example, by department or even by application?

OK, you might think using Resource Groups. But that doesn’t have the some “power” than tags. One good example I frequently see the use of tags in Azure is for automation or for chargeback reports. I know that there is an obscene amount of way to use tags, I just mention two.

When you apply tags, you will able to retrieve the resources in your subscription with a tag key and the value. You can have the same tag across different resource groups. So, with tags in azure you can:

  • Name-value pairs assigned to individual resources or to resource groups
  • Subscription-wide taxonomy
  • Each resource can have up to 15 tags

Although tags in Azure have some limitations. These are the limitations applied to tags:

  • Each resource or resource group can have a maximum of 15 tags.
  • The tag name is limited to 512 characters.
  • The tag value is limited to 256 characters.
  • Tags applied to the resource group are not inherited by the resources in that resource group.

So, when to use tags in Azure? Please see some of the Tagging Tips

  • Notes: Simple note for VM
  • Creator: track the “owner” of a VM
  • Department/Cost center: who pays
  • Environment: production vs. pre-production vs. test

Resource Tags Tips

  • Tag by environment (e.g., dev/test/prod)
  • Tag by role (e.g., web/cache/DB)
  • Tag by department
    (e.g., finance/retail/legal)
  • Tag by responsible party (e.g., Bob)
  • Tag for automation (e.g., autoshutdownschedule)

Naming Conventions in Azure

Why having a good name convention in Azure have a huge importance?

Because describes type of resource in the subscription. Placing the naming pattern in an order that allows easier application level grouping for potential showback/chargeback billing and it help when you decide to use Azure Automation.

There isn’t, so far at my best knowledge, the perfect naming convention. Although here are some considerations that might help you creating one:

  • Constrained unique across entire Azure – Example: SQL Server Name, Storage Account Name, etc. must be unique across Azure not just on the subscription
  • Constrained by length – Example: Search Service is constrained 2 to 15 characters
  • Constrained to alpha-numeric – Example: Storage Account Name cannot have dash, dots, etc.
  • Constrained unique within account – Example: Storage Table Name must be unique within Azure subscription account
  • Cannot include upper case characters – Example: Storage account names must be all lower case
  • Cannot contain offensive or forbidden sub-strings.

So, when you create a naming convention for all the Azure resources, these are the requirements that you need to ensure that convention will address:

  • Unique Azure naming
  • Case sensitivity requirements
  • Application association
  • Environment association
  • Region association
  • Instance association
  • Object association

Azure Subscriptions Best Practices – Part 2

On the previous blog post (see here), we saw the characteristics of an Azure subscriptions. Now lets see how we can architect the Azure Subscription model, using best practices.

First we must understand the limits that each subscription has. See the picture bellow to see the limits. Although because azure is always change, please visit the link bellow to see the most up-to-date Azure subscription limits.

http://azure.microsoft.com/en-us/documentation/articles/azure-subscription-service-limits/

Subscription Design

Frequently a lot of costumer ask me “what is the best model to design the Azure subscription?”. I don’t have just one model to answer the question, because there are a lot of factors that influence the model. What I usually do is start with the Tier 0 subscription model.

The Tier 0 subscription model, allow you have an architecture that we solution architect are used to have on premise infrastructures, by having the core services segregated from the different environments.

In this case the Tier 0 Subscription will have all the ExpressRoute or Site-to-Site (S2S) VPN, the domain controllers and all other services that are considered core services. Then you will create the environment subscriptions, like production, test and development (see picture below), if applicable.

By using this model, the Global Admins will control all the access and configuration for this subscription. This subscription should not have much change along the time, only when you are adding another core service or need to adjust something.

The other subscriptions should connect to the Tier 0 to be able to use the core services. On each subscription, you can have different Resource Groups (RG) in each region, if that is applicable. This mean, if you have 2 datacenters (one is US West and the other in US East) you can have the ExpressRoute connecting the both datacenter to the respective Azure regional datacenter.

With this model, you are minimizing the risk of anyone on either the production, test or dev subscription to “mess” with the core services. This apply as well for the Active Directory domain controllers running on Azure VMs.

If you want to use a different model, start with this one and evaluate what is the best solution for your organization. These are some points that might help design the right Azure subscription for you:

  1. Multiple Azure subscriptions means:
  2. Multiple Subscriptions = Complexity
    1. Duplicate provisioning and management: IP Address space, network circuits, gateways, vNets, Subnets, NSGs, routing
    2. More connectivity to manage
    3. More security to manage
    4. More identities to manage
    5. Potentially more ExpressRoute circuits to buy
  3. Multiple subscriptions are going to happen though, so design for it

Containers and Resources

It’s not because you have only one big subscriptions (for example production) that you can control the access to the resources. You can give access to the RG using RBAC. To help you with that you need to understand that:

  • Subscription is the top-level container
  • Create Resource groups in the subscription
  • Place resources within the resource groups

When I design the Subscription model I always tried to take in consideration the following:

  1. Management approach
    1. Single team or distributed
    2. RBAC
  2. Security requirements
    1. Data or network security
    2. Environments – Sandbox, Dev, Test, UAT, Pre-Prod, Prod
  3. Connectivity requirements
    1. Single point of ingress?
    2. Multiple regions?
  4. Application requirements
    1. Data flow
    2. Compliance