Ilya Ivanov

Ilya Ivanov

DevOps eXtraordinaire

Terraform, Lambda, source_code_hash

Oh, no! Terraform deploys Lambda again!

~ resource "aws_lambda_function" "myfunc" {
      id                             = "myfunc"
      ...
    ~ source_code_hash               = "eqFDS7dB1KoQEAC6CNQ2ZAY1tV0ghf836eCQZwnWlpc=" -> "vnjjxeDfV108KVscceyBBxsRGYin4CPOWqgWvgWTVOo="

Problem

Terraform and AWS Lambda have a long history of bad blood: 1, 2. The problem is that Lambda appears to be perpetually in dirty state, due to source_code_hash differences.

The root cause of is this is difference in packaging on different machines and bad documentation. Well, and an asinine design choice on AWS part.

  1. source_code_hash gets overwritten by AWS-provided data upon response.
  2. The documentation for source_code_hash (aka output_base64sha256, filebase64sha256) lies:

    (String) The base64-encoded SHA256 checksum of output archive file.

Why would you even want to base64-encode a hash? The purpose of base64 encoding is to do away with non-printable chars, which a hash doesn’t have.

Turns out, what they actually do is compute sha256, then take the resulting text string and treat its characters as binary values, then base64 that: sha256sum lambda.zip | xxd -r -p | base64.

The problem is, recent zip versions store file permissions, and different umask values on different machines result in different permissions, which in turn produces different archives with different hashes.

Solution

To alleviate this, set output_file_mode in the corresponding archive_file:

data "archive_file" "myfile" {
  type             = "zip"
  source_file      = "${path.module}/src/myfunc/index.js"
  output_path      = "myfunc.zip"
  output_file_mode = "0644"
}

resource "aws_lambda_function" "myfunc" {
  description                    = "Example description"
  function_name                  = "myfunc"
  filename                       = "myfunc.zip"
  source_code_hash               = data.archive_file.myfile.output_base64sha256
  ...

If it doesn’t seem to help, add a newline to each of your lambda functions and deploy again. Sometimes AWS seems to return old source_code_hash if the new version only differs in file permissions (or not deploy the new version at all). The newline will force re-deploy, and after that deployment from any machine should be working as intended.

Comments

You May Also Enjoy