In a previous post back in 2021, I described a method for getting Terraform outputs into Azure DevOps pipeline variables. I’ve been doing some work on Infrastructure as Code deployments in pipelines recently and have come across a much beter method.
A recap of the previous method I used
It used a local provisioner in Terraform, which ran a PowerShell command during the terraform apply process, which write to the name and value of a specific output to the command line. Using the task.setvariable logging command, Azure DevOps automatically interpreted this as a pipeline variable.
The local provisioner looked like this:
1
2
3
4
5
6
7
8
9
10
11
12
resource "null_resource" "terraform-to-devops-vars" {
triggers = {
// always execute
uuid_trigger = uuid()
}
provisioner "local-exec" {
command = <<EOT
Write-Host "##vso[task.setvariable variable=STORAGE_ACCOUNT_NAME]${azurerm_storage_account.storageaccount.name}"
EOT
interpreter = ["pwsh", "-Command"]
}
}
In hindsight, this isn’t a brilliant solution. While it did the job for the piece of work I was doing at the time, it has some limitations:
- It makes the Terraform code less reusable. It will always try and execute the PowerShell command to write the variable to the command line, even if it’s not running in Azure DevOps.
- If the host running the Terraform doesn’t have PowerShell installed, then it will likely fail.
- It’s not very flexible. You can only write one variable at a time, and you have to know the name of the variable you want to write.
A more flexible method!
The terraform output command can output the values of all the outputs in a Terraform configuration as JSON. For example, if you run terraform output --json on a Terraform configuration that’s just been deployed, you might get something like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
PS C:\git\variable-test> terraform output --json
{
"afw_private_ip_address": {
"sensitive": false,
"type": "string",
"value": "172.16.100.4"
},
"hub_virtual_network_id": {
"sensitive": false,
"type": "string",
"value": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-hub/providers/Microsoft.Network/virtualNetworks/vnet-hub"
},
"localadminpassword": {
"sensitive": true,
"type": "string",
"value": "Pa55w.rd123!!"
}
}
PowerShell is my go-to scripting language and it can easily parse JSON, so I used that as the basis for a solution.
The following script will run terraform output --json and capture the output. It then cycles through all the properties in the JSON object and writes them to the command line using the task.setvariable logging command, so Azure DevOps will interpret them as pipeline variables.
Any variable marked in the Terraform output as “sensitive” will be marked as a secret variable in Azure DevOps and masked in the logs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
$tfoutput = (terraform output --json) | ConvertFrom-Json
if ($tfoutput -eq $null) {
Write-Host("No Terraform output variables found")
} else {
foreach($outputVar in $tfoutput.psobject.properties) {
if ($outputVar.value.sensitive -eq "true") {
Write-Host("##vso[task.setvariable variable=$($outputVar.name);issecret=true]$($outputVar.value.value)")
Write-Host("$($outputVar.name)-***SECRET***")
} else {
Write-Host("##vso[task.setvariable variable=$($outputVar.name);]$($outputVar.value.value)")
Write-Host("$($outputVar.name)-$($outputVar.value.value)")
}
}
}
The script can be run as an inline script in the pipeline after the terraform apply runs. A simple example of this being used in a pipeline that has 3 tasks that:
- Run
terraform initandterraform apply - Run the PowerShell script to write terraform outputs to pipeline variables
- Run a PowerShell command to output one of the variables to the command line to test it’s been set correctly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- script: |
terraform init
terraform apply --auto-approve
displayName: 'Apply Terraform configuration'
- task: PowerShell@2
displayName: 'Capture Terraform output variables and push to pipeline variables'
inputs:
targetType: 'inline'
script: |
$tfoutput = (terraform output --json) | ConvertFrom-Json
if ($tfoutput -eq $null) {
Write-Host("No Terraform output variables found")
} else {
foreach($outputVar in $tfoutput.psobject.properties) {
if ($outputVar.value.sensitive -eq "true") {
Write-Host("##vso[task.setvariable variable=$($outputVar.name);issecret=true]$($outputVar.value.value)")
Write-Host("$($outputVar.name)-***SECRET***")
} else {
Write-Host("##vso[task.setvariable variable=$($outputVar.name);]$($outputVar.value.value)")
Write-Host("$($outputVar.name)-$($outputVar.value.value)")
}
}
}
- task: PowerShell@2
displayName: 'Test a pipeline variables'
inputs:
targetType: 'inline'
script: |
Write-Host "afw_private_ip_address = $($env:AFW_PRIVATE_IP_ADDRESS)"
To try this out, I’ve put together a Terraform config that doesn’t actually deploy anything, but just has a few hard coded outputs. You can find it in my terraform-adf-var-capture repository on Github.
Running this in an Azure DevOps Pipeline, we can see that it does what we expected - the outputs have been saved as pipeline variables and can be used by another task in the pipeline.
This works fine for tasks running within the same job, but the variables have scope. If you need them to be available in a different Job or Stage in the pipeline, you will need to modify the logging command to include isOutput=true like this:
1
Write-Host("##vso[task.setvariable variable=$($outputVar.name);isOutput=true;]$($outputVar.value.value)")
I wrote some more about that in my original post (see the “Wait! Variables have scope” section). Microsoft’s documentation on pipelines variables is pretty good and Adam the Automator has a great blog post on the subject too. I’ve included links to both below:
References: