How To Build Infrastructure as Code With Pulumi And Golang - Part 2

When I started this series on creating infrastructure as code on AWS with Pulumi, I knew the team was actively improving Go support. What I didn’t expect was how quickly those improvements would land and how much cleaner the code would get. This post revisits some of the earlier code and updates it to the new SDK.

The complete project is available on GitHub.

The core programming model in Pulumi has a lot of features, but not all of them were available in the Go SDK yet. The work the Pulumi team was doing included more strongly typed structs (no more interface{}, yay! 🥳), better Input and Output type support, and feature parity with other language SDKs. With that many changes, breaking changes were inevitable. Pulumi recommends that Go developers pin their SDK versions.

TL;DR I had to make a bunch of changes to make everything work again (like adding pulumi.String() wrappers around strings) but I could also replace all my custom structs with ones from the Pulumi SDK. There are a few rough edges, but it’s a work in progress and the direction is great.

Configuration

#

One of the new features — or maybe I just missed it before — is the ability to read and parse configuration directly from the YAML file into typed structs.

Previously, I wrote my own helper to pull config values from the context:

// getEnv searches for the requested key in the pulumi context and provides either the value of the key or the fallback.
func getEnv(ctx *pulumi.Context, key string, fallback string) string {
	if value, ok := ctx.GetConfig(key); ok {
		return value
	}
	return fallback
}

That helper and its related functions can now be replaced. The snippet below defines a VPCConfig struct and populates it using RequireObject(), which reads directly from the YAML file. If the config key doesn’t exist, Pulumi throws an error. You also get to model your structs with proper types — arrays instead of comma-separated strings.

// VPCConfig is a strongly typed struct that can be populated with
// contents from the YAML file. These are the configuration items
// to create a VPC.
type VPCConfig struct {
	CIDRBlock   string `json:"cidr-block"`
	Name        string
	SubnetIPs   []string `json:"subnet-ips"`
	SubnetZones []string `json:"subnet-zones"`
}

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		// Create a new config object with the data from the YAML file
		// The object has all the data that the namespace awsconfig has
        conf := config.New(ctx, "awsconfig")

        var vpcConfig VPCConfig
		conf.RequireObject("vpc", &vpcConfig)

		vpcArgs := &ec2.VpcArgs{
			CidrBlock: pulumi.String(vpcConfig.CIDRBlock),
			Tags:      pulumi.Map(tagMap),
        }

        ... // snip
    })
}

The configuration file itself also gets cleaner. Here’s what it looked like before:

vpc:name: myPulumiVPC
vpc:cidr-block: "172.32.0.0/16"
vpc:subnet-zones: "us-east-1a,us-east-1c"
vpc:subnet-ips: "172.32.32.0/20,172.32.80.0/20"

And here’s the new version with proper YAML structure:

awsconfig:vpc:
	cidr-block: 172.32.0.0/16
	name: myPulumiVPC
	subnet-ips:
		- 172.32.32.0/20
		- 172.32.80.0/20
	subnet-zones:
		- us-east-1a
		- us-east-1c

A few more lines, but being able to see which config items are arrays is worth it. One thing that struck me as a little odd: you write config in YAML but the struct tags use JSON.

Type Safety

#

In the previous version, I created my own structs to add type safety when creating DynamoDB tables. Here’s what the old approach looked like:

// DynamoAttribute represents an attribute for describing the key schema for the table and indexes.
type DynamoAttribute struct {
	Name string
	Type string
}

// DynamoAttributes is an array of DynamoAttribute
type DynamoAttributes []DynamoAttribute

// ToList takes a DynamoAttributes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (d DynamoAttributes) ToList() []map[string]interface{} {
	array := make([]map[string]interface{}, len(d))
	for idx, attr := range d {
		m := make(map[string]interface{})
		m["name"] = attr.Name
		m["type"] = attr.Type
		array[idx] = m
	}
	return array
}

// Create the attributes for ID and User
dynamoAttributes := DynamoAttributes{
    DynamoAttribute{
        Name: "ID",
        Type: "S",
    },
    DynamoAttribute{
        Name: "User",
        Type: "S",
    },
}

The updated SDK introduced strongly typed structs for Table Attributes and Global Secondary Indices, which eliminates the need for those helper types and methods. What was roughly 33 lines of code is now 10:

// Create the attributes for ID and User
dynamoAttributes := []dynamodb.TableAttributeInput{
    dynamodb.TableAttributeArgs{
        Name: pulumi.String("ID"),
        Type: pulumi.String("S"),
    }, dynamodb.TableAttributeArgs{
        Name: pulumi.String("User"),
        Type: pulumi.String("S"),
    },
}

The same benefits apply to Lambda deployments. More strongly typed structs with clear field definitions means the code is easier to write and easier to read. No helper methods needed.

Going forward

#

It’s not all perfect — the Pulumi engineers still have work to do, and they’re doing it out in the open. There are some rough edges, like documentation gaps for inputs and outputs of certain methods. But the progress over just a few weeks was impressive, and I’m looking forward to seeing where the Go SDK goes from here.

Cover image by Martin Vorel from Pixabay