Creating a personal cloud: Terraform

For a long time, I've run a few random VPSes on the internet, some for experimentation, some for apps that I actually use (Syncthing) and need to continue to run. I admired products like those of Hashicorp from afar while continuing to manage my snowflake servers by hand, logging in to do an apt upgrade and mess around with configs every few months.

Recently, though, I got a new job and it seems like ops will heavily feature in my life, so I decided enough was enough and it was time to completely overengineer my personal setup. This ended up being surprisingly fun (well, maybe it was just fun relative to using AWS's web UI) so I wrote it up.

For the first part, we'll explore creating a server on AWS with Terraform, but with the added caveat that this will go in a security group that only allows SSH connections via key. In further parts of this, we'll explore adding a VPN to that instance and using it as an access point for other applications and AWS services.

Note that actually applying these terraform commands will cost you cash money. emoji-money_with_wings

Terraform

Terraform is a tool for "building, changing and versioning infrastructure safely and efficiency." Essentially the idea is that you define your infrastructure as code, making it easy to tear down, recreate and inspect from one place. If you've ever spent a joyful afternoon setting up some AWS workflow only to have no clue how to repeat what you just did, the value proposition is very obvious.

Terraform's docs are pretty excellent, but I wanted to write down how to swing a few moving parts together, so starting from the top, in a file called aws.tf (or whatever-you-like.tf):

provider "aws" {
  profile = "default"
  region = "us-east-2"
}

First we pull in the provider we're using and since it's AWS, specify a region. Terraform's provider list is quite fascinating and contains the obvious cloud and VPS providers, but also interesting things like configuring a Postgres server or managing SAAS accounts such as Pagerduty.

At any point in this article, it's great to run terraform apply to see what Terraform wants to do. Go ahead and try it -- it'll download the AWS provider plugin, and save the state of the Terraform world in a local file.

variable "az" {
  type = string
  default = "us-east-2c"
}

variable "ubuntu_ami" {
  type = string
  default = "ami-0d36f68a8c544bbe"
}

Then we'll define a few variables for use later, an availability zone to put our servers in (the high availability version of this post will come out...well probably never, but we'll punt to a later article) and an image to use for our server, the 64-bit Ubuntu 19.04. Incidentally I struggled to find this through Amazon's interface until a Google search turned up Ubuntu's EC2 image locator.

Next thing we'll do is define a key pair to provision our servers with.

resource "aws_key_pair" "login_key" {
  key_name = "login_key" 
  public_key = "your key here"
}

We've just created our first resource, e.g. something that will actually result in something in AWS being created, or updated or destroyed if you make further changes to it after creation. You'll also notice that login_key is repeated twice. The first is the id within Terraform, meaning this resource will be accessible as aws_key_pair.login_key when we need to reference it later. The second is the name in AWS, which allows us to name some things and not others. emoji-anguished

If you don't have an SSH key already, you can generate one with

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_aws

and get the value you need with

cat ~/.ssh/id_rsa_aws.pub

Then we'll define a security group to put our VPN in. We'll allow public SSH access only, and since these AMIs only allow logging in with a key, that should be secure enough for educational purposes.

# This odd-seeming line tells Terraform to pull in AWS's default VPC, allowing us to attach new
# security groups to it. I actually started off without noticing this option, and started creating
# an entirely new VPC which meant I needed gateways, ACLs... if you need to do that, you probably
# don't need to read this article :p

resource "aws_default_vpc" "default" {}

resource "aws_security_group" "public_ssh" {
  name = "public_ssh" 
  description = "Allow public SSH to VPN"
  vpc_id = "${aws_default_vpc.default.id}"

  ingress {
    from_port = 22
    to_port = 22
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Okay, finally we get to the payoff

resource "aws_instance" "gate" {
  ami = var.ubuntu_ami
  instance_type = "t3.nano"
  key_name = "${aws_key_pair.login_key.id}"
  availability_zone = var.az
  vpc_security_group_ids = ["${aws_security_group.public_ssh2.id}"]
}

# I also recommend giving this instance an Elastic IP, so you can destroy and recreate the server
# itself if needed

resource "aws_eip" "gate_ip" {
  instance = "${aws_instance.gate.id}"
  vpc = true
}

Finally, run a terraform apply and then a terraform show. If everything went correctly, you should be able to SSH to that EC2 instance using your key! You'll see the IP under a line like this:

resource "aws-eip" "gate-ip" {
  # ...
  public_ip = "12.34.56.78"
  # ...
}

(if you created one using the commands above, ssh -i ~/.ssh/id_rsa_aws <ip> will connect you to the instance)

To see the power of terraform in action, we can try recreating just the server with terraform taint aws_instance.gate and terraform apply. After waiting a minute or so, you should be able to connect to a new EC2 instance on the same IP.

Now celebrate by running terraform destroy to avoid racking up your AWS bill, or heading to CloudWatch and creating a billing alert if you plan to continue down this road like I do. emoji-tada


Feel free to send questions, comments and suggestions to phil@upvalue.io

If you enjoy this sort of thing, you should come be my manager or check out our other job openings.