I’ve been working to get more familiar with both Terraform and Azure Devops recently and working on understanding how both can be used in ‘real world’ situations. One example I came across was the need to extract values about deployed Azure resources from Terraform for later use within the Azure Devops Pipeline.
For example, Storage Accounts must have unique names throughout Azure, so you might generate a unique string in Terraform and dynamically build a name for your Storage Account. The following Terraform configuration would create a Resource Group and then create a Storage Account with name that ends with a random string generated by Terraform.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
resource "random_id" "random_suffix" {
byte_length = 4
}
resource "azurerm_resource_group" "site_rg" {
name = "rg-website"
location = "uksouth"
}
resource "azurerm_storage_account" "storageaccount" {
name = "storacc${random_id.random_suffix.hex}"
resource_group_name = azurerm_resource_group.site_rg.name
location = azurerm_resource_group.site_rg.location
account_tier = "Standard"
account_kind = "StorageV2"
account_replication_type = "LRS"
}
Azure DevOps provides a method for setting variables from within the command line. If we were to run the following command within a “script” resource in the Terraform configuration, it would set the value of the STORAGE_ACCOUNT_NAME variable to the name of the Storage Account that the Terraform configuration above creates:
1
##vso[task.setvariable variable=STORAGE_ACCOUNT_NAME]${azurerm_storage_account.storageaccount.name}
The local-exec provisioner with Terraform allows you to run code on the same box that’s running your ‘terraform apply’. You can nest this within an existing resource, or have it separately within a “null_resource” as in the example below.
In this case I’m running it within PowerShell Core, as I’m running the Pipeline on Ubuntu, and using Write-Host to run the command. You could use bash as well, you just need to run the ##vso…. but on the host running the Pipeline. In this example, the trigger used will make sure the code runs everytime we do a ‘terraform apply’.
1
2
3
4
5
6
7
8
9
10
11
12
resource "null_resource" "terraform-to-devops-vars" {
triggers = {
// always execute
uuid_trigger = uuid()
}
provisioner "local-exec" {
command = <<EOT
Write-Host "##vso[task.setvariable variable=STORAGE_ACCOUNT_NAME]${azurerm_storage_account.storageaccount.name}"
EOT
interpreter = ["pwsh", "-Command"]
}
}
If I want to use the values in those variables in another task, you just reference when as you would a Pipeline Variable. As an example, the following would upload files into the Storage Account specificed in the $STORAGE_ACCOUNT_NAME variable.
1
2
3
4
5
6
7
8
9
10
11
- task: AzureCLI@2
displayName: Copy data to storage account
inputs:
azureSubscription: AzureDevOpsWebsite1
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
az storage blob upload-batch \
--destination \$web \
--account-name $STORAGE_ACCOUNT_NAME \
--source "$(System.ArtifactsDirectory)/site"
Wait! Variables have scope!
The above works fine if all your tasks are within the same “Job” within Azure DevOps. You will run into problems the moment you want to use the variables from a Task that is within a different Job in your Pipeline. This is because the variables you set within a Job are scoped to that Job only.
The solution to this is to mark them as “output variables” by including “isOutput=true” in the command that creates the variable. This this case I had the following being run by the local-exec provisioner in my Terraform configuration:
1
Write-Host "##vso[task.setvariable variable=STORAGE_ACCOUNT_NAME;isOutput=true;]${azurerm_storage_account.storageaccount.name}"
In the Job where you want to use the value in a variable from a previous Job, you have to create a local Variable within that Job and reference the Variable from the other Job that contains the value.
In this example, within definition of the Job where I want to use the Variable, the variables block would look like this:
1
2
variables:
STORAGE_ACCOUNT_NAME: $[ dependencies.deploy_azure_resources.outputs['terraformApply.STORAGE_ACCOUNT_NAME'] ]
Where:
- deploy_azure_resources = Name of the Job where the Varuable is created
- terraformApply = Name of the Task where the Variable is created
- STORAGE_ACCOUNT_NAME = Name of the Variable
You also need to make sure that the Job that is “consuming” the variable has the originating job set as a dependency.
The whole thing looks like this, with the variable created within the ‘terraform apply’ run in the deploy_azure_resources Job. The variable then used within the deploy_jekyll_site Job.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
- job: deploy_azure_resources
displayName: 'Deploy Azure Resources (Terraform)'
pool:
vmImage: 'ubuntu-16.04'
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: '0.14.8'
displayName: 'Install Terraform'
- task: TerraformTaskV1@0
name: terraformInit
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(Build.Repository.LocalPath)/tf-azure-deploy'
backendServiceArm: 'AzureDevOpsWebsite1'
backendAzureRmResourceGroupName: 'rg-tfstate'
backendAzureRmStorageAccountName: 'stortfstate7h76f54'
backendAzureRmContainerName: 'tfstate-blog'
backendAzureRmKey: 'terraform.tfstate'
- task: TerraformTaskV1@0
name: 'terraformApply'
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
environmentServiceNameAzureRM: 'AzureDevOpsWebsite1'
command: 'apply'
workingDirectory: '$(Build.Repository.LocalPath)/tf-azure-deploy'
- job: deploy_jekyll_site
displayName: 'Deploy Jekyll Site'
pool:
vmImage: 'ubuntu-16.04'
dependsOn: deploy_azure_resources
condition: succeeded()
variables:
STORAGE_ACCOUNT_NAME: $[ dependencies.deploy_azure_resources.outputs['terraformApply.STORAGE_ACCOUNT_NAME'] ]
steps:
- task: AzureCLI@2
displayName: Copy data to storage account
inputs:
azureSubscription: AzureDevOpsWebsite1
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
az storage blob upload-batch \
--destination \$web \
--account-name $STORAGE_ACCOUNT_NAME \
--source "$(System.ArtifactsDirectory)/site"
References: