Deconstructing JSON: Super Secrets

“Psst… Hey buddy, want to keep your plain text password in your source control?”

Hopefully, you answer is “No”, but if you’d been following along with this JSON/ARM template series of posts, you will have noticed that I’ve defaulted to deployments with straight up, plain text, username/passwords in my template examples. So before we go any further, lets secure this up a bit!

Goal: Deploy a Windows Nano VM using an ARM template in GitHub without having to put the Admin username or password anywhere in plain text.

To do this, we’ll need a place to hide the secrets. In this case, we’ll use Azure Key Vault. With Key Vault, you create a resource group and vault (using PowerShell or an ARM template) and then load up some secrets or keys. You can hide basic passwords or load certificates.

I’m going to create my Key Vault using PowerShell. I’m purposely creating my vault in its own Resource Group. You can create the Key Vault in the same Resource Group as the rest of your deployment, but then if you delete the whole group for during testing, you’ll have to recreate the vault each time.

  1. Log into Azure
  2. Get your Subscription details, copy the GUID of the subscription you want to use for the key vault into the $VSSubID variable.
  3. Customize the variables for the Resource Group name, Vault name and deployment region.
  4. Once the vault is created, customize the “SuperSecretPassword” and “FancyAdminName” to your liking and add those secrets to the vault.
Login-AzureRmAccount
Get-AzureRmSubscription
$VSSubID = {subcription_guid}  
Set-AzureRmContext -SubscriptionID $VSSubID

$RGName = "KeyVaultRG"
$VaultName = "MyVaultName"
$Location = "West US"

New-AzureRmResourceGroup -Name $RGName -Location $Location

New-AzureRmKeyVault -VaultName $VaultName -ResourceGroupName $RGName -Location $Location

Set-AzureRmKeyVaultAccessPolicy -VaultName $VaultName -EnabledForTemplateDeployment -ResourceGroupName $RGName

$secretvalue = ConvertTo-SecureString 'SuperSecretPassword' -AsPlainText -Force
$secret = Set-AzureKeyVaultSecret -VaultName $VaultName -Name 'ServerAdminPassword' -SecretValue $secretvalue

$secretvalue = ConvertTo-SecureString 'FancyAdminName' -AsPlainText -Force
$secret = Set-AzureKeyVaultSecret -VaultName $VaultName -Name 'ServerAdminName' -SecretValue $secretvalue

Now that my vault is crated, I’ll need to edit my ARM template to grab the secrets from the Key Vault when my VM is deployed. Make the change to your parameters.json file, not your deployment template. This way you can use the same template deployment with different subscriptions and vaults easily. Instead of using the basic string type for your adminUsername and adminPassword, reference the secrets you added to the vault.

"adminUsername": {
      "reference": {
      "keyVault": {
        "id": "/subscriptions/{guid}/resourceGroups/KeyVaultRG/providers/Microsoft.KeyVault/vaults/ImperfectKeyVault"
      }, 
      "secretName": "ServerAdminName" 
       } 
    },
   "adminPassword": {
      "reference": {
      "keyVault": {
        "id": "/subscriptions/{guid}/resourceGroups/KeyVaultRG/providers/Microsoft.KeyVault/vaults/ImperfectKeyVault"
      }, 
      "secretName": "ServerAdminPassword" 
       } 
    }

Now, when you run your deployment, the admin username and password will be pulled from the vault and passed to your VM during creation. If you go to log in that server directly, you’ll need to provide the plain text versions of those secrets just like usual.  It’s worth noting that not anyone can pull keys out of your vault.  By default only the subscription admin will be able to access the contents of the vault, but you can add additional permissions to a vault to allow for other users and automation tools to access the same secrets.

You can find the example of this template in my GitHub repo.

Learn more about Azure Key Vault at https://azure.microsoft.com/en-us/documentation/articles/key-vault-get-started/

 

Getting Started with Azure Automation

As a regular use of an MSDN account with Azure credits, I often find myself running out of credit before I expect and it’s often my own fault. It’s really easy to spin up a server or start experimenting with a service and forget to turn it off or delete it when done. If only a selection of my VMs would turn on during the day and off during the night…. Enter Azure Automation.

I looked at Azure Automation when it first came onto the scene, but admittedly struggled with getting it going. The certificate management related to the credentials was challenging, my PowerShell was less than stellar… and I just got frustrated and moved on to something else.

But after watching my subscription accidentally drain down to nothing 2 weeks before it reset, I had to revisit Azure Automation. The credential management is much easier now and the PowerShell required to start and stop VMs is minimal.

You’ll need two scripts… one to start and one to stop the desired VMs. You set a schedule for each one and sit back and relax. I’ve also been using GitHub a lot for source control and you can tie Azure Automation into that as well.

My scripts list out each VM I want on the nightly shut down schedule, because I didn’t want to have every VM in my subscription shut down automatically, in case I do have a need to keep something running for more than 24 hours. You may want to customize your scripts differently.

To get started, I recommend following the tutorial in the Azure documentation for “My First Runbook” . It walks you through all the steps of getting your automation account going as well as giving some sample scripts.

If you like to see the basic scripts I’m using for a few of my VMs, you can find them in my GitHub repo.

Automate Away!!

Happy Bastille Day!

Bastille Day is the name given in English-speaking countries to the French National Day, which is celebrated on July 14th each year. The French National Day commemorates the beginning of the French Revolution with the storming of the Bastille on July 14, 1789.
Now, 227 years later, Systems Administrators everywhere might remember this day as the day support ended for Windows Server 2003 in 2015.  The successor to Windows 2000 Server it included features from Windows XP that were well loved by IT Pros and consumers alike, leading to it’s wide-spread adoption.
Released on April 24, 2003, Windows Server 2003 was very security minded for the time, reducing the attack surface by limiting the number of features installed by default. It also included several compatibility modes to allow older applications to run with more stability and continued to support Window NT 4.0 networking. Improvements were also made to ease the transition from the NT 4.0 directory to Active Directory.
Windows Server 2003 was also the first operating system released by Microsoft after the announcement of its Trustworthy Computing initiative, and as a result, contains a number of changes to security defaults and practices. Some of the cutting edge features of the time included Internet Information Services (IIS) v6.0, the “Manage Your Server” administrative tool to help with server configuration and improvements to Active Directory and Group Policy administration.
Here are some other great milestones and inventions that we saw in 2003:
  • iTunes Music Store also opens in April 2003.
  • FluMist by MedImmune released as an alternative to the flu shot.
  • The Intelligent Oven (www.tmio.com) keeps your food refrigerated until the programmed cooking time. Can be remotely controlled via the Internet or by telephone.
  • Because the CD-ROM was the primary storage medium in the early 2000s, the CD-ROM shredder came onto the scene in 2003.
  • Java Logs (http://www.pinemountainbrands.com/products/pine-mountain-java-log-4-hour-firelogs) – for those of you who love the open fire but feel bad about burning wood.
If you are still running Windows Server 2003 in your datacenter (a solid year after support has ended), you might want to check out some current training on Microsoft’s latest server offerings. Enjoy!

Deconstructing JSON: Counting On

So here I am, in JSON-land, working on my deployment for VMs. But so far, I’ve only deployed one VM in my network. But one VM is boring. I want to be able to rack and stack more than one machine at a time! :-)  Enter the copy object and the copyIndex function!  You can read the official documentation about it, but let see how this works within my existing template. Things to note:

  • You can only reference the copyIndex function in RESOURCES. Don’t try to build variables with them, you’ll just be sad.
  • You can’t use a copy loop in a nested resource (this blog series hasn’t encountered this yet).
  • You can’t use them to loop items that are defined as properties within a resource, like multiple data disks.

So in my example, let’s extend the template to deploy 3 VMs in the same subnet and put them all in the same NSG (network security group). First, you must declare in the parameters that you’d like count something:

   "count": { 
      "type": "int", 
      "defaultValue": 3 
    },

Now, whenever you want to reference the “current iteration” of the loop elsewhere in your template, you’ll use copyIndex(). Another thing to note – computers like to count starting at zero, humans like to count starting at 1. If we stick with copyIndex(), we’ll get VMs numbered 0, 1, 2. If you’d like them to be numbered 1, 2, 3 use copyIndex(1), which will offset the numbering by 1.

Since this template is going to create 3 VMs, we will need to make sure to account for other resources that VMs have a one to one relationship with and be sure to loop those as well. The only items that we are required to loop are the network interfaces and the public IP addresses (which have individual DNS names).  We don’t have to loop things like the storage account (which can be used by multiple VMs) or the network, since we only need one in this case.

In the resource for the publicIPAddresses, you’ll see the reference to the copyIndex() in the name, and in the DNS domain label. In each case, the incremented number will be tacked on at the end of each. Then finally, you’ll see the specific reference to looping this resource added, with the name of “pubipcopy”, pulling the count from the parameter you set early.

     {
      "type": "Microsoft.Network/publicIPAddresses",
      "apiVersion": "2015-06-15",
      "name": "[concat(parameters('publicIPAddressName'), parameters('dnsNameForPublicIP'),copyIndex())]", 
      "location": "[parameters('location')]",
      "properties": {
        "publicIPAllocationMethod": "[variables('publicIPAddressType')]",
        "idleTimeoutInMinutes": 4,
        "dnsSettings": {
          "domainNameLabel": "[concat(parameters('dnsNameForPublicIP'),copyIndex())]"
        }
      },
       "copy": {
        "name": "pubipcopy",
        "count": "[parameters('count')]"
      }
     },

One of the reasons you need to name each resource copy object is so they can be used in places where dependencies are required.

For the network interface, you’ll see I’ve made similar references to the copyIndex() in the name. Also a network interface is dependent on the existence of a public IP address, so you’ll see “pubipcopy” referenced under “dependsOn” as well as in the IP configuration properties.

   {
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(parameters('nicName'), copyIndex())]",
      "location": "[parameters('location')]",
    
      "dependsOn": [
        "pubipcopy",
        "[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]",
        "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSecurityGroupName'))]"
      ],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
             "privateIPAllocationMethod": "Dynamic",
             "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses', concat(parameters('publicIPAddressName'),parameters('dnsNameForPublicIP'),copyIndex()))]"
              },
              "subnet": {
                "id": "[variables('subnetRef')]"
              }
              }
          }
        ],
       "dnsSettings": {
         "dnsServers": []
       },
       "enableIPForwarding": false,
       "networkSecurityGroup": {
               "id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSecurityGroupName'))]"
            }
      },
       "copy": {
        "name": "niccopy",
        "count": "[parameters('count')]"
      }
    },

This all comes together when you get to the VM resource where it refers to the NIC in the network profile and using the copyIndex() when creating the VM name, computer name and OS disk.

    {
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Compute/virtualMachines",
      "name": "[concat('NanoVM-', copyIndex())]", 
      "location": "[parameters('location')]",
      "copy": {
        "name": "vmcopy",
        "count": "[parameters('count')]"
        },
      "dependsOn": [
        
        "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]",
        "niccopy"
      ],
      "properties": {
        "hardwareProfile": {
          "vmSize": "[parameters('vmSize')]"
        },
        "osProfile": {
          "computerName": "[concat('NanoVM-', copyIndex())]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "[variables('imagePublisher')]",
            "offer": "[variables('imageOffer')]",
            "sku": "[parameters('windowsOSVersion')]",
            "version": "latest"
          },
          "osDisk": {
            "name": "[concat('osdisk_', concat('NanoVM-', copyIndex()))]" ,
            "vhd": {
              "uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/NanoVM-',copyIndex(),uniqueString(resourceGroup().id),'-osdisk.vhd')]"
            },
            "caching": "ReadWrite",
            "createOption": "FromImage"
          }
          },
            
          "networkProfile": {
          "networkInterfaces": [             
        {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',concat(parameters('nicName'), copyIndex()))]"
            }
          ]
        },
          "diagnosticsProfile": {
          "bootDiagnostics": {
             "enabled": "true",
             "storageUri": "[concat('https://',parameters('newStorageAccountName'),'.blob.core.windows.net')]"
          }
        }
        }

      }

You can find my full template here: https://github.com/techbunny/Templates/tree/master/multiple_nano_server

ARM Templates: What Error?!?

There is more than one way to deploy an ARM template once you’ve created or edited on to your liking. The first way is to copy the code into a blank template deployment in the Azure portal. This has a few benefits, primarily that it gives you the option to edit any parameters in the template to your liking before deployment and saves you from having to write a separate “azuredeploy.parameters.json” file.

However the deployment blade in the portal doesn’t really give you much in the way of error messages when the template fails. Often, you are give something along the lines of “error exists on line 1: position 10,374” or some such information. But your template is clearly more than one line in length and I have no intention of counting out thousands of characters to find the error location. The full text of the message may or may not be helpful depending on what point in the deployment the error occurred.

This is where deploying the template via PowerShell shines. As much as the red text is heartbreaking, it will give you the proper line number and position count to line up with your code. Also included is some often useable error text to point you on your way.

This single line of PowerShell will deploy a template that is hosted online (like in GitHub) and the -Verbose switch will report on the status of each of your resources as they start and complete deployment.

New-AzureRmResourceGroupDeployment -ResourceGroupName "<YourResourceGroupName>" -TemplateURI $templateFileURI -TemplateParameterURI $parameterFileURI -Verbose

If that doesn’t solve your problem or point you in the right direction, it’s time to dig deeper in the Azure Audit Logs from within the portal. They are chronological, so after a few clicks, you can drill down into the details of the most recent error. You can also access those logs via PowerShell:

Get-AzureRmLog -Status Failed -ResourceGroup "YourResourceGroupName" -DetailedOutput

You can also access these logs by digging down into the deployment in the portal. Just to “Audit Logs” and you’ll find all the activity in your subscriptions. Looked for the ones related to failed deployments or validations of deployments and as you drill into the details you’ll find what’s going wrong. Usually it’s an incorrect location of a parameters file, maybe a storage account name that’s already in use, or a missing resource group.

Happy Hunting!

How Microsoft IT does the Cloud

One of the things that is fascinating about working at Microsoft is watching how we function internally as our own “customer”. This is very apparent when it comes to integrating with cloud solutions like Azure. We have the same challenges as any other external customer, with the added bonus of developing, testing and implementing our own solutions. Sometimes people will ask me something like “You work for Microsoft, isn’t Azure just free for you?”

Alas, it is not. Every bit of Azure I use to learn, test and demo is accounted for somewhere, and I’m just lucky my manager hasn’t come after me holding an embarrassing bill to explain!

The MSIT Showcase has a host of documents, case studies and webinars to peel back the curtain and give you an idea of what it’s like to move many critical workloads into the cloud. Here are three links to get you started:

The cloud security mindset (Article) –
https://www.microsoft.com/itshowcase/Article/Content/703/The-cloud-security-mindset

Managing resource efficiency in a hybrid cloud (Webinar) –
https://www.microsoft.com/itshowcase/Article/Content/702/How-Microsoft-IT-manages-resource-efficiency-in-Azure-and-onpremises

Architecting your network for successful Azure adoption (Case study) –
https://www.microsoft.com/itshowcase/Article/Content/648/Architecting-your-network-for-successful-Azure-adoption

Enjoy!