Since the protos for uploading an SBOM for a build have been changed to
use an enumeration instead of a plain string with the latest revisions
to the HCP Packer SBOM support feature, we update how we reference those
values for the SBOM format to use that enum instead.
In the current state, a Packer build that succeeds but fails to push its
metadata to HCP for reasons other than a lack of artifact will always
succeed from the perspective of a user invoking `packer build`.
This can be a bit misleading, as users may expect their artifacts to
appear on HCP Packer if their build succeeded on Packer Core, so this
commit changes this behaviour, instead reporting HCP errors as a real
error if the build failed, so packer returns a non-zero error code if
this happens.
When a build cannot be completed without errors, the build state was
left as running, unless the build explicitly failed, which meant that
HCP Packer would be responsible for changing the status after the
heartbeats for the build stopped being sent for two 5m periods.
This commit changes this behaviour, by explicitly marking the build as
failed if something did not work while trying to complete a build on HCP
Packer, even if the local Packer core build succeeded before that.
Since packer now supports keeping track of SBOMs produced during a
build, we add the code to integrate those changes into the internal/hcp
package, so we do upload them on build completion.
The hcp-sbom provisioner is a provisioner that acts essentially like a
download-only file provisioner, which also verifies the file downloaded
is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to
HCP Packer later on.
With the draft to support both gob and protobuf as serialisation formats
for Packer, along with the SDK changes that propel them, we add a series
of tests that make sure the logic that picks which protocol is solid and
functional.
These tests rely on building several versions of the tester plugin, with
and without protobuf support, to then install them in the tests as
needed to test the logic of Packer using packer build with them, and
templates that require multiple plugins.
As we're trying to move away from gob for serialising data over the
wire, this commit adds the capability for Packer to pick dynamically
between gob or protobuf for the serialisation format to communicate with
plugins.
As it stands, if all the plugins discovered are compatible with
protobuf, and we have not forced gob usage, protobuf will be the
serialisation format picked.
If any plugin is not compatible with protobuf, gob will be used for
communicating with all the plugins that will be used over the course of
a command.
As the SDK now supports it in the context of legacy templating engine,
we add support in HCL2 for the aws_secretsmanager_raw function, which
gets the raw value of a secret from aws secrets manager.
Since the version 0.6.0 of the plugin SDK was released yesterday, we
bump packer to use this version.
This change adds new functions to use within Packer regarding AWS
secretsmanager, along with changes to how plugins can communicate
over-the-wire (e.g. using protobuf/msgpack for serialising
configurations instead of gob).
# Description
The output of the example on [uuidv4 Function](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/functions/uuid/uuidv4) is not a valid RFC compliant UUIDv4. It indicates the usage of the `uuidv4()` function and outputs `b5ee72a3-54dd-c4b8-551c-4bdc0204cedb` which is not a valid UUIDv4.
I've corrected the example to output a UUIDv4 conforming to the RFC as such `xxxxxxxx-xxxx-4xxx-Nxxx-xxxxxxxxxxxx`, where:
- The 13th character is always `4` (indicating version 4).
- The 17th character must be either `8`, `9`, `a`, or `b` (indicating the first character of the variant).
# Changes
```diff
- b5ee72a3-54dd-c4b8-551c-4bdc0204cedb
+ 9fc99a70-7cd5-482d-bb2b-03af016e4e94
```
Replaces the old UUID output with a valid RFC compliant UUIDv4.
# References
- [RFC 4122](https://datatracker.ietf.org/doc/html/rfc4122)
- [RFC 9562](https://datatracker.ietf.org/doc/html/rfc9562)
# Misc.
To make sure this wasn't an issue with the `uuidv4()` function within Hashicorp's [packer](https://github.com/hashicorp/packer) I tested the function in the following way:
### Command executed:
```ps
> .\packer.exe inspect .\uuid.pkr.hcl
```
### Contents of the _uuid.pkr.hcl_ file:
```hcl
locals {
uuid_0 = uuidv4()
uuid_1 = uuidv4()
uuid_2 = uuidv4()
uuid_3 = uuidv4()
uuid_4 = uuidv4()
uuid_5 = uuidv4()
uuid_6 = uuidv4()
uuid_7 = uuidv4()
uuid_8 = uuidv4()
uuid_9 = uuidv4()
}
```
### Output:
```ps
Packer Inspect: HCL2 mode
> input-variables:
> local-variables:
local.uuid_0: "90877db8-5519-46ea-ae15-7dfb92594064"
local.uuid_1: "fe6a4649-97d9-4686-b981-3295175f941a"
local.uuid_2: "9944d83d-dab2-4cfb-a1db-572d19271e7a"
local.uuid_3: "547cddb7-c979-4b87-90d0-2bd9b68858b5"
local.uuid_4: "c13dc47a-552c-4dfb-a75d-2f63bb248b41"
local.uuid_5: "3db1ce29-bdde-4642-b010-1a41d47c22a3"
local.uuid_6: "4a020460-edd1-471d-b8a2-5956c0c68257"
local.uuid_7: "1845bf87-6908-4fc0-8f11-b5b4f36c60a7"
local.uuid_8: "f5c7e552-b799-45f3-8172-46162eadfd89"
local.uuid_9: "057c2eaf-6769-4a8d-90c8-775aec80496a"
> builds:
```
`packer validate` would output the same error message four times per
unsupported root block type found in a template (e.g., 'src' instead of
'source'). This behavior was due to a function being called four times
for each file on each stage of the parsing.
Version 5.11.0 of the go-git library is vulnerable to two CVEs as
reported by our scanners.
Both are not impacting Packer since we only use go-git to read values
from a local Git repository, but still we upgrade our version to 5.13.0
so those reports don't apply to us.
The reported AWS S3 vulnerability was inherited from the go-getter
module that Packer uses for downloading files from external sources.
This vulnerability only impacts S3 uploads, therefore Packer is not
vulnerable itself as go-getter only downloads such blobs.
Since the change required to fix this advisory would be to bump the AWS
SDK to v2, this being a major change, is not something to do lightly, so
we opted to ignore this advisory for now so it doesn't block upcoming
releases.
The version of golang.org/x/net that we're using (v0.25.0) is vulnerable
to GO-2024-3333, a DoS vulnerability.
Packer is not particularly vulnerable to this, as we are not a hosted
service, but since security scanners report this, and the fix is rather
simple, we address it today.
The hcl2_upgrade command transforms a JSON template into an HCL2
template for use with Packer.
The command is quite fragile already, but given that this is the last
remaining fragment that causes Packer to depend on the AWS SDK directly,
we can do away with it.
This commit therefore imports the definitions for AWS access config, so
we can extract this information from the JSON template, and include it
in the definition of the output source for AWS, since we manage this one
differently from other sources.
This allows us to not depend on the AWS plugin directly, which in turn
makes Packer not need to link with the AWS plugin when compiling the
executable.
We are still depending on the AWS SDK for now since the SDK exposes a
aws_secretsmanager function that can be used for interpolation (legacy
JSON interpolation to be clear), so this cannot be removed from now, but
we should consider some form of remediation in the future.
The crypto experimental module that the SDK depends on was detected
vulnerable on v0.23.0 to an authentication bypass attack through the
ServerConfig for SSH.
This is only used for locally proxying SSH communications, typically for
use with Ansible to provide an SSH server with which to interact, in
order to provision an instance of an OS that uses a communicator other
than SSH, therefore the vulnerability does not necessarily impact us.
However, in order to resolve the automatic report of that vulnerability,
we bump the crypto module's version to 0.31.0, where that problem was
addressed.
Compiling plugins was originally intended to be an idempotent operation.
This however starts to change as we introduce build customisations,
which have the unfortunate side-effect of changing the state of the
plugin directory, leading to conflicts between concurrent compilation
jobs.
Therefore to mitigate this problem, this commit changes how compilation
jobs are processed, by introducing a global compilation queue, and
processing plugins' compilation one-by-one from this queue.
This however makes such requests asynchronous, so test suites that
require plugins to be compiled will now have to wait on their completion
before they can start their tests.
To this effect, we introduce one more convenience function that
processes those errors, and automatically fails the test should one
compilation job fail for any reason.
When building a plugin, we may want some customisation capabilities
beyond changing the version/pre-release/metadata, and instead run
commands or change files on the filesystem.
To do so, we introduce functions under the BuildCustomisation type,
which have two responsabilities: changing the current state of the
plugin's directory, and cleaning up afterwards.
These customisations are passed as parameters to the BuildSimplePlugin
function, and are called one-by-one, deferring their cleanup after the
build process is finished.
A first implementation of such a customisation is added with this
commit, in order to change the version of a module that the plugin
depends on, which we'll use to change the version of the plugin SDK in
order to test how Packer behaves with different versions of the SDK for
a single plugin.
When trying to validate that a particular file exists after a run of
Packer in a test suite, we can use the FileExists checker that we
provide as part of the gadgets we added for the acceptance test suites.
This approach works well, but only if we can extract a file name
reliably from the output of Packer core, or if we know what to look for
exactly beforehand. For other cases with a generated name however, the
FileExists checker is not enough, and therefore to accomodate for those
cases, we are introducing a new checker for this purpose: FileGlob.
FileGlob, as its name suggests, runs a glob expression on the
filesystem, and returns an error if no match was found regarding this
glob expression.
Some tests will create files and directories as part of the execution
path for Packer, and we need a way to check this, so this commit adds a
new file gadget to do those checks after a command executes.
When a user defines a `hcp_packer_registry` block in their `build`
without a `bucket_name`, but they define it in their environment, Packer
should not report the bucket_name being wrong.
The strcontains function check if a sub string is a indeed a subset of a
given string.
hcl2template: add strcontains function
The strcontains function check if a sub string is a indeed a subset of a
given string.
The acceptance tests for the HTTP datasource had a `Outputs` attribute
for checking the output of the command for a specific regexp pattern.
As pointed out, given the expectative nature of the attribute, naming it
`Outputs` did not make the intent clear, so we rename it to
`ExpectedOutputs` with this commit.
Since the expected error to look for in the output is compiled to a
regexp, the `[]` from the error message were interpreted as a set of
characters, which made the regexp not match the expected output from the
command.
So to avoid this problem, we escape them so they are expected verbatim
in the command output.
Since the DAG package was lifted from Terraform, its contents are more
than what we need for now, so this commit cleans-up the package to keep
only the currently needed parts of code.
If we need to support more in the future, we can revert this commit, or
pickup the changes again from Terraform.
