Deconstructing JSON: Anatomy of a VM

While the Azure VM on has two official dependencies, the storage account and the network interface, there are a variety of properties that you need to gather or determine ahead of time in order for your deployment to be successful.  If you were doing the manual creation of the VM in the Portal, the properties would be the parts you’d have to enter while shuffling through all the blades.  Let’s shuffle less blades….

With the VM template, we start out like every other resource, requiring a type, name, API version and location for deployment.  You can see the two lines that call the two dependencies.

{
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Compute/virtualMachines",
      "name": "[variables('vmName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]",
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
      ],

When you get into the properties, you can see where everything comes together with a variety of profiles for hardware, the operating system, storage, networking and diagnostics.  When deploying via PowerShell, you’ll get prompted to enter the Admin username and password manually.

To fill in the variables for the image reference part of the storage profile, you can use some handy PowerShell to query Azure for the existing image names.  (I’ll put that at the bottom of the post.)

      "properties": {
        "hardwareProfile": {
          "vmSize": "[variables('vmSize')]"
        },
        "osProfile": {
          "computerName": "[variables('vmName')]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "[variables('imagePublisher')]",
            "offer": "[variables('imageOffer')]",
            "sku": "[parameters('windowsOSVersion')]",
            "version": "latest"
          },
          "osDisk": {
            "name": "osdisk",
            "vhd": {
              "uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('OSDiskName'),'.vhd')]"
            },
            "caching": "ReadWrite",
            "createOption": "FromImage"
          }
          },
            
          "networkProfile": {
          "networkInterfaces": [             
        {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
            }
          ]
        },
          "diagnosticsProfile": {
          "bootDiagnostics": {
             "enabled": "true",
             "storageUri": "[concat('https://',parameters('newStorageAccountName'),'.blob.core.windows.net')]"
          }
        }
        }

The PowerShell for the image references: these lines will return the current images available for Windows Server. Be sure to pick the location you intend to use, because some image skus have limited regions and change the publisher name as needed if Windows Server isn’t what you are looking to deploy.

Get-AzureRmVMImagePublisher -Location "West US"

Get-AzureRmVMImageOffer -Location "West US" -PublisherName MicrosoftWindowsServer

Get-AzureRmVMImageSku -Location "West US" -PublisherName MicrosoftWindowsServer -Offer WindowsServer

You’ll find the full template to deploy this one VM here: https://github.com/techbunny/Templates/tree/master/smallcloudlab

Advertisement

Deconstructing JSON: Bring on the NICs

One of the final dependencies for deploying a server on a network is a network card.  For Azure, networking is crucial for VMs so you can remotely access and control your VM after deployment.

Now here’s the thing about JSON… The dependencies.  The resource dependencies allow you to basically pack all your resources, parameters and variables into one file and the deployment sorts out what needs to be created in what order.  This differs from writing a complete deployments in PowerShell, where you have to account for the dependencies yourself, by ensuring things are placed in the proper order and account for any necessary wait times.

So far in this blog series, I’ve done deployments that can stand alone: just the network, just storage, etc.  But now that I’m getting closer to my goal of being able to deploy VMs, the dependency requirements start to appear.

NICs are dependent on a public IP address and the virtual network.  While you can deploy NICs without having a VM to “plug” them into, the template that is used to deploy them does require those other two resources.  So even though I’ve already deployed the virtual network and would be deploying the NIC into that existing resource group and network, the template won’t be considered valid unless the networking resources are included.

So at this point, I’ve started adding the new resources to my existing template.  Upon deployment, ARM will verify the existence and configuration of all the resources an only add or change the ones that are missing or configured differently.

For the public IP address I’ve added:

{
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[variables('publicIPAddressName')]",
      "location": "[parameters('location')]",
      "properties": {
        "publicIPAllocationMethod": "[variables('publicIPAddressType')]",
        "dnsSettings": {
          "domainNameLabel": "[variables('dnsNameForPublicIP')]"
        }
      }
    },

For the network interface I’ve added:

{
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[variables('nicName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
      ],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
             "privateIPAllocationMethod": "Dynamic",
             "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]"
              },
              "subnet": {
                "id": "[variables('subnetRef')]"
              }
              }
          }
        ]
      }
    },

We also needed to add a few variables, but you could make these parameters instead, depending on your needs.  Take note of vnetID and subnetRef variables, they are more complex expressions.

   "variables": {
       "virtualNetworkName": "SmallNet",
       "subnetName": "Subnet20",
       "addressPrefix": "192.168.0.0/16",
       "publicIPAddressName": "smallcloudpubip",
       "publicIPAddressType": "Dynamic",
       "dnsNameForPublicIP": "smallcloudlab",
       "nicName": "smallcloud_nic1",
       "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
       "subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]"
        
     },

To see the full template up until this point, find it in my GitHub Repository.

Deconstructing JSON: Storage Wars

The last place I left off with my deconstruction of JSON, I had deployed a basic network as the basis for my future home for some virtual machines.  The next key dependency for Virtual Machine is storage.  In this case, I’ll need a storage group in the same regional location (and ideally in the same resource group) as my network.

For the sake experimentation, this template example creates two different storage accounts and takes advantage of both parameters and variables to determine the location and the storage account name.

The default parameters set the storage account as “wrongnamestor” and the location in the East US. These settings will apply to the first instance of the storage account resource.  In the variable section, the storage account name is set as “labweststor2” with a location in the West US region.

{
   "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
   "contentVersion": "1.0.0.0",
   
   "parameters": {
    "newStorageAccountName": {
      "type": "string",
      "defaultValue": "wrongnamestor",
      "metadata": {
        "description": "Unique DNS Name for the Storage Account where the Virtual Machine's disks will be placed."
      }
    },
   
    "location": {
      "type": "string",
      "defaultValue": "East US",
      "allowedValues": [
        "West US",
        "East US"
      ],
      "metadata": {
        "description": "Restricts choices to where premium storage is located in the US."
      }
    }
   },
      
   "variables": {
       "location": "West US",
       "newStorageAccountName": "labweststor2"
           
    },
   
   "resources": [ 
     {
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[parameters('newStorageAccountName')]",
      "apiVersion": "2015-06-15",
      "location": "[parameters('location')]",
      "properties": {
        "accountType": "Standard_LRS"
      }
     },
  
      {
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[variables('newStorageAccountName')]",
      "apiVersion": "2015-06-15",
      "location": "[variables('location')]",
      "properties": {
        "accountType": "Standard_LRS"
      }
     }   
   
    ]
}

Because I would probably like both my storage accounts to be in the same region with similar naming, I’ve also created the additional parameters file named “azuredeploy.parameters.json”.  This short file will override the default parameters in the deployment template and set the desired storage account name of “labweststor1” in the West US.

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "newStorageAccountName": {
      "value": "labweststor1"
     },
    "location": {
      "value": "West US"
    }

  }
}

I’ll deploy this template using the following PowerShell:

New-AzureRmResourceGroupDeployment -ResourceGroupName 'BigLab' -TemplateURI <templateFileURI> -TemplateParameterURI <parameterFileURI> -Verbose 

This line of PowerShell is exactly the same as the line I used to deploy the networking template. The only thing that would need be changed is the actual details of the file location.

As a bonus, the default behavior of Azure Resource Manager deployments is incremental. Since I’m deploying to the same Resource Group that was created with my networking template, the result will be the addition of the two storage accounts to the resource group that contains the network.

If I wanted the template to remove anything not specified, I would have to add the “-mode complete” switch.  Template deployments in the portal only run in incremental mode.

Now that I know my storage template is working as expected, I’ll integrate those resources and parameters into one template as I build out the complete deployment.

Deconstructing JSON: Networking 101

This leads me to really needing to deconstruct a JSON template so I can really understand what’s going on.  I think it’s great that we can use the Azure Quickstart Templates as resources, as well as export some templates directly from Portal configurations, but I need to break it down to really feel like I understand what I’m working with.

To relate this to real life, I started to map out a simple, yet ambitious improvement to my Imperfect Lab (the subject of prior posts).  Still, this proved to be too ambitious for my “getting started with JSON” goal, so I simplified it.

Within Azure, ARM templates are awesome for doing the equivalent of “racking and stacking” in the real world.  So I went small… let’s get a network deployed.  A small network; the physical equivalent of a switch with a couple VLANs. Let the JSON “unboxing” begin!

A basic JSON template has this format:

{
   "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
   "contentVersion": "",
   "parameters": {  },
   "variables": {  },
   "resources": [  ],
   "outputs": {  }
}

The sections for $schema and resources are required.  Parameters, variables and outputs are optional, depending on your needs.   So let’s check out the resources I’m using and take a few notes:

  1.  The contents of the resources section are flanked with [square brackets].
  2. All resources need to include the apiVersion, type and name.  All the other details vary based on resource.
  3. All these resources in my sample use parameters to determine the properties, not variables. This means they can all be overridden with an additional “azuredeploy.parameters.json” file.

"resources": [
    {
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/virtualNetworks",
      "name": "[parameters('vnetName')]",
      "location": "[parameters('location')]",
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "[parameters('vnetAddressPrefix')]"
          ]
        },
        "subnets": [
          {
            "name": "[parameters('subnet1Name')]",
            "properties": {
              "addressPrefix": "[parameters('subnet1Prefix')]"
            }
          },
          {
            "name": "[parameters('subnet2Name')]",
            "properties": {
              "addressPrefix": "[parameters('subnet2Prefix')]"
            }
          },
          {
            "name": "[parameters('subnet3Name')]",
            "properties": {
              "addressPrefix": "[parameters('subnet3Prefix')]"
            }
          }
        ]

As you can see, this resources deploys a single virtual network with three subnets.  Because there are no hard coded names and IP addresses, I use the parameters section to set those. (In the sample below, I’ve only shown the parameters for subnet 1, but the parameters for the other two subnets would be similar.)

  1.  The “defaultValue” will allow the deployment to be repeatable, but can be overridden with values provided in a separate parameters file.
  2. The “allowedValues” section used in the location parameter is an optional example.  This would drive selections from the drop down menus using the Azure portal or limit the possibilities that can be passed from a parameters file.

  "parameters": {
    "location": {
      "type": "string",
      "defaultValue": "West US",
      "allowedValues": [
          "West US",
          "East US"
       ],
      "metadata": {
        "description": "Deployment location"
      }
    },
    "vnetName": {
      "type": "string",
      "defaultValue": "SmallNet",
      "metadata": {
        "description": "VNet name"
      }
    },
    "vnetAddressPrefix": {
      "type": "string",
      "defaultValue": "192.168.0.0/16",
      "metadata": {
        "description": "Address prefix"
      }
    },
    "subnet1Prefix": {
      "type": "string",
      "defaultValue": "192.168.20.0/24",
      "metadata": {
        "description": "Subnet 1 Prefix"
      }
    },
    "subnet1Name": {
      "type": "string",
      "defaultValue": "Subnet20",
      "metadata": {
        "description": "Subnet 1 Name"
      }

As none of my resources require variables or outputs, I won’t need those section in my file.  The final version of the file is here.

Now to actually deploy this template, you have a few options:

  1. Copy and paste the code into a blank “Template Deployment” in the Azure Portal.  You’ll be shown the blades for the parameters to edit or accept the default values.
  2. Deploy the template using PowerShell either from a local file or a web location.  Not the different parameters used to determine the location and replace the placeholders with the appropriate paths.  (Because I’m not using a parameters file, I’ll leave off the “-TemplateParameterURI” part.)

For a web location source (like Github) use:

  New-AzureRmResourceGroupDeployment -ResourceGroupName 'BigLab' -TemplateURI <templateFileURI> -TemplateParameterURI <parameterFileURI> -Verbose

 

 

For a local source (on your pc) use:

New-AzureRmResourceGroupDeployment -ResourceGroupName 'BigLab' -TemplateFile <$templateFileURI> -TemplateParameterFile <parameterFileURI> -Verbo

 

Once this template deploys, which only takes a few moments, you’ll be able to review the results in the Azure Portal.

Deconstructing JSON: When is VM Not Just a VM?

One of the cool things about deploying infrastructure in Azure is all the interesting ways you can do it  – portal, PowerShell or ARM templates.  I like using the portal to help clarify what I need to set up, how I want it configured and to learn about new Azure offers quickly.  But let’s face it, we are going to have to live and die by the ARM template.

This week I decided to buckle down and really start to get my head around those JSON templates for Azure.  But looking at some of the Quickstart Templates for even the most basic VM deployments seemed like a big mess of parameters and resource dependencies. What happened to the days where a VM needed just a storage account and a cloud service?

I’ll tell you what happened: Azure Resource Manager.  ARM makes it possible to solve a lot of the design challenges that cropped up with Azure Service Manager, particularly giving cloud administrators a lot more flexibility and access to the networking functions.  This gives organizations the flexibility to design the networking in Azure to align more with what they may already have on-prem, or to optimize the cloud network for specific needs.  With greater flexibility comes greater complexity.

That “simple” VM has a few dependencies:

Virtual Machine (depends on)

  • Storage Account
  • Network Interface

Now you might look at those two things and think that it’s not much different than the dependencies from the ASM days.  You wouldn’t be far from wrong.  A storage account is still a storage account. But that network interface is a bit more work.

In the “before time” with Cloud Services, the fabric did the work of automatically creating the connection to the network, particularly if you opted to “quick create” the VM.  And even if you took the step to create a virtual network beforehand, the cloud service is what connected the VM to both the private and public networks.

Now, with ARM, you have to create and connect all the bits that used to be part of the cloud service.  If this was a physical machine, you’d be installing a NIC and running a cable to a switch port.

Virtual Machine (depends on)

  •  Storage Account
  •  Network Interface (depends on)
    •  Public IP Address
    •  VNet (depends on)
      •  Subnet

Thus, in order to deploy one Virtual Machine, you need to design and account for all the dependencies and sub-dependencies in one template.  In my next blog post, I’ll take on that very bottom item – the Virtual Network.

Arm in ARM with VM Backups

With all the interesting announcements with BUILD this week, you might have missed this which, to me, signals that the end is near for the wait for Azure Backup and DR related features to be available in “Azure v2” aka Azure Resource Manager.

Now in public preview: Azure Backup for ARM VMs!  Check out the full post with instructions on how to get started – https://azure.microsoft.com/en-us/documentation/articles/backup-azure-vms-first-look-arm/

Seeing some of the these features being ported over means that organizations looking to start out using the cloud to supplement there on-prem backup and recover plans aren’t locked into the Azure Service Manager model.

No word yet on the big guns like Azure Site Recovery, but being able to snapshot an Azure VM for backup purposes making moving some on-prem workloads to the cloud much easier!

 

Today I Learned: Azure Site Recovery Mobility Service

Today I learned that if you are switching from the legacy version of ASR to the new version of ASR, the Mobility Service version has a new revision, which would be expected.  However, you have to manually uninstall the older version from the servers you wish to protect.  Otherwise, the “push” installation just fails and doesn’t report anything really useful in the Azure portal.

When I went to manually install it on the server, I got better error messages from the application noting that a previous version of the service was installed and needed manual removal.

Perhaps I’ve just saved you some time one day.

Every Week in the Microsoft Reactor

If you haven’t discovered the Microsoft Reactor in San Francisco, you are in for a treat.  This new community space near the Moscone Center is often available to use for meet ups and training workshops.  You can check out the event calendar for details of upcoming events.  You can also watch the new Reactor Weekly show by Tim Reilly – in this week’s episode you can learn about the pending installation of a Surface Hub and hear about some of the value of the community user groups like Pacific IT Professionals.

Some of the events I’m looking forward to this month are:

Notable Notes: Certification, Azure VMs and Linux!

It’s February, so if you are looking for a way to continue some momentum you might have started with the new year, consider the Know It, Prove It challenge.  Follow the steps, commit an hour a day and by the end of the month you’ll have a new tech “super power”.

When it comes to certification, don’t get stuck thinking Azure is all about Windows.  Over 25% of workloads in Azure are Linux, so if that’s your passion you can become Microsoft certified as well.  Check out the new MSCA: Linux on Azure.

Also, if you’ve got 7 minutes, check out this episode of Tuesdays with Corey – Troubleshooting and Diagnostics for VMs in Azure.

Microsoft Cloud Networking – The Poster!

If there’s one thing I’ve learned while doing IT, it’s that nothing is really simple.  And with the addition of the “cloud” when it comes to providing services for your business or your customers, things just keep getting more complicated.  Remember the days where you had one nice T1 coming into your office and all was right in the world?  (Ah… the good ol’ days….)

Anyway, sometimes when you are looking at changing the networking for your company to support more cloud offerings like Office365 or moving workloads into Azure, you could really use a cheat sheet to get you pointed in the right direction and give you some insight into what you might need to research further as you work on your design.  Enter the “Microsoft Cloud Networking for Enterprise Architects” poster!

This printable, 8-page poster (it fits on legal paper) gives you the skinny on the design components and considerations for networking in the cloud age.  Topics cover are:

  • Evolving your network for cloud connectivity  Cloud migration requires changes to the volume and nature of traffic flows within and outside a corporate network.
  • Common elements of Microsoft cloud connectivity  Integrating your networking with the Microsoft cloud provides optimal access to a broad range of services.
  • Designing networking for Microsoft SaaS (Office 365, Microsoft Intune, and Dynamics CRM Online)  Optimizing your network for Microsoft SaaS services requires careful analysis of your Internet edge, your client devices, and typical IT operations.
  • Designing networking for Azure PaaS  Optimizing networking for Azure PaaS apps requires adequate Internet bandwidth and can require the distribution of network traffic across multiple sites or apps.
  • Designing networking for Azure IaaS  Optimizing networking for IT workloads hosted in Azure IaaS requires an understanding of Azure virtual networks (VNets), address spaces, routing, DNS, and load balancing.

There are several other posters that might be of interest as well, focusing on identity, security and storage. Find them with some other handy resources for IT architecture.