* bump golang.org/x/crypto to 0.43.0
CVE-2025-47913 GO-2025-4116
* fixed go.sum
* fixed multiple warnings that prevented test runs
* make generate
* fix lint errors, update linter version
* fix go vet issues
* hcl2template: err on malformed local/data dep
When introducing the DAG for locals and datasources, we forgot to handle
one limit case: if a dependency for a local or data is malformed, we
didn't check that a vertex was associated to it, leading to the final
DAG being malformed, and the DAG library would crash in this case.
This commit fixes this problem by checking that the dependency does
exist before attempting to add it to the graph as an edge for a vertex,
so that it is reported accurately, and do that Packer doesn't crash.
* error message change
---------
Co-authored-by: anshul sharma <anshul.sharma@hashicorp.com>
With the draft to support both gob and protobuf as serialisation formats
for Packer, along with the SDK changes that propel them, we add a series
of tests that make sure the logic that picks which protocol is solid and
functional.
These tests rely on building several versions of the tester plugin, with
and without protobuf support, to then install them in the tests as
needed to test the logic of Packer using packer build with them, and
templates that require multiple plugins.
Compiling plugins was originally intended to be an idempotent operation.
This however starts to change as we introduce build customisations,
which have the unfortunate side-effect of changing the state of the
plugin directory, leading to conflicts between concurrent compilation
jobs.
Therefore to mitigate this problem, this commit changes how compilation
jobs are processed, by introducing a global compilation queue, and
processing plugins' compilation one-by-one from this queue.
This however makes such requests asynchronous, so test suites that
require plugins to be compiled will now have to wait on their completion
before they can start their tests.
To this effect, we introduce one more convenience function that
processes those errors, and automatically fails the test should one
compilation job fail for any reason.
When building a plugin, we may want some customisation capabilities
beyond changing the version/pre-release/metadata, and instead run
commands or change files on the filesystem.
To do so, we introduce functions under the BuildCustomisation type,
which have two responsabilities: changing the current state of the
plugin's directory, and cleaning up afterwards.
These customisations are passed as parameters to the BuildSimplePlugin
function, and are called one-by-one, deferring their cleanup after the
build process is finished.
A first implementation of such a customisation is added with this
commit, in order to change the version of a module that the plugin
depends on, which we'll use to change the version of the plugin SDK in
order to test how Packer behaves with different versions of the SDK for
a single plugin.
When trying to validate that a particular file exists after a run of
Packer in a test suite, we can use the FileExists checker that we
provide as part of the gadgets we added for the acceptance test suites.
This approach works well, but only if we can extract a file name
reliably from the output of Packer core, or if we know what to look for
exactly beforehand. For other cases with a generated name however, the
FileExists checker is not enough, and therefore to accomodate for those
cases, we are introducing a new checker for this purpose: FileGlob.
FileGlob, as its name suggests, runs a glob expression on the
filesystem, and returns an error if no match was found regarding this
glob expression.
Some tests will create files and directories as part of the execution
path for Packer, and we need a way to check this, so this commit adds a
new file gadget to do those checks after a command executes.
Since we introduce the DAG with this series of commits, only on locals
and data sources, we need to make sure that the behaviour is what we
expect.
Therefore, this commit adds a basic test with Packer build, and packer
validate, to evaluate a template with both locals and data sources
depending on one another.
This is rejected with the sequential evaluation methods, as we process
the different types one-by-one, whereas the DAG allows us to mix the
order between the two, while still rejecting circular dependencies (and
doing that before they even get evaluated), and self-references.
In order for the creation of a temporary directory to install plugins
into to be simpler to understand and use, we change how the directory is
created, cleaned-up, and installs plugins into.
Now, instead of a tuple of a string (path) and a cleanup function, we
return a structure that comprises the test suite, and the temporary
directory, along with methods to handle those steps independently.
When using a PackerCommand, the Run function was made public as a way to
access the contents of an execution.
This was clumsy as it had too many responsabilities, and was not needed
strictly as Assert was performing the executions, as many times as
required.
This could introduce cases in which one run as spent by the caller, then
the remainder were executed through Assert.
Therefore, we change this convention.
Now, run is private to the type, and only through Assert can a command
be executed.
If a test needs access to a command's output, stderr, or error, it can
do so through the Output function, which requires Assert to be called
first.
When a command asserts its output with checkers, by default it will
register errors through a t.Errorf.
While this works, in some cases we would want to stop execution
immediately if a function's Assert fails, as the rest of the test may
depend on the assertion being valid.
In the current state, this means either getting the result of the run to
check if an error was returned (not fully reliable as if the command was
run multiple times, and the last run succeeded, we won't get an error),
or relying on t.IsFailed() (completely reliable).
Instead, we introduce a new function on packerCommand, that lets users
change how Assert behaves, so that if an error was reported, instead of
logging the error and flagging the test as failed, we can use t.Fatalf,
so that the test immedately fails and stops execution.
The lib name for the common components for writing packer_test suites
was not clear, and did not follow the convention established in Packer
core and plugins.
Therefore this commit does two things: first the lib is renamed into
common as to follow this convention, and clearly document which
components are common to all tests.
Also checkers are placed in a subpackage of common, common/check, so
that it is clearer what is meant to be used as checks for a command's
execution status after it's been run, as part of Assert.
The function's name was the same as the plugins test suite runner/init
function, making it impossible to differentiate between the two when
looking at the logs, or when attempting to filter which tests to run
using the suite function's name.
The interface for building a plugin through the test suite was
confusing, as it would build the plugin and return its path, cache the
path for the version built, and return the path regardless if something
was built or not.
While in the current state this is harmless as builds are idempotent,
since the state of the plugin package/module does not change, this will
in the future as we introduce customisation techniques on the plugin's
directory and files, making this double-use potentially dangerous.
Furthermore, the current behaviour is unclear, as the function hides
that caching mechanism, which could come as a surprise for users
attempting to build a plugin for the duration of a test, while the built
plugin is linked to the test suite being run, and not the unit test
being evaluated.
Therefore this commit changes the sequence in which plugins are built
and used. Now the `CompilePlugin` function builds a plugin, and does not
return its path anymore, instead terminating the tests immediately if
they fail.
In normal test usage, a new `GetPluginPath` function is introduced,
which looks-up the path in the suite's cache, failing immediately if
invoked before the plugin is built.
With this change, it is heavily advised to build plugins when
initialising the suite, then in the tests, the GetPluginPath function
should be used to get a plugin's path for interacting with packer
commands.
The compiledPlugins map with a mutex was essentially a glorified
sync.Map, and therefore did not need to be defined as such.
Instead, this commit replaces its uses by a normal sync.Map, and keeps
it in the base test suite.
When testing a packer build or validate, one common use case is to check
which plugins are loaded and used by Packer to run a command.
This is generally done through Grep, but repeating the same pattern can
be redundant, and if the output changes, all those need to be updated.
Therefore, this commit introduces a PluginsUsed gadget, which can be
orchestrated to ensure a plugin is used, or not used, allowing to check
for multiple plugins at once.
As a small reorganisation attempt, this commit moves some file-system
oriented functions from plugin.go into its own file so they are more
logically grouped.
In terms of organisation, we keep functions that interact with plugins
into its own file, therefore the function that build plugin versions
should really be in plugin.go, not in suite.go.
The compiledPlugins map used to be a global variable, which can be
problematic as we move to independent test suites, since those test
suites run in the same process space, the global variable could point to
now deleted plugin versions/paths in separate suites, which would make
tests fail with random errors.
To avoid this, the map is now scoped to the test suite, and a new copy
is created lazily if used by the test suite.
Having only one test suite for the whole of Packer makes it harder to
segregate between test types, and makes for a longer runtime as no tests
run in parallel by default.
This commit splits the packer_test suite into several components in
order to make extension easier.
First we have `lib`: this package embeds the core for running Packer
test suites. This ships facilities to build your own test suite for
Packer core, and exposes convenience methods and structures for building
plugins, packer core, and use it to run a test suite in a temporary
directory.
Then we have two separate test suites: one for plugins, and one for core
itself, the latter of which does not depend on plugins being compiled at
all.
This sets the stage for more specialised test suites in the future, each
of which can run in parallel on different parts of the code.
When a command is run, it is the expectation that no test should make
Packer panic. If it did, something is wrong and Packer should be fixed
so it doesn't panic anymore in that situation.
The way we did the check before was adding a PanicCheck after the
command ran, so we could make sure of that during `Assert`.
However, since we introduced the possibility to have multiple runs,
having this addition as part of the run loop meant that the PanicCheck
would be run as many times as there were runs.
While this worked, this implied that we'd do the same check multiple
times on a single command output, which is not optimal.
Instead, this commit moves the check to within the `Run` function, this
way for each run of the command we do the check once, and then we can
assert the results of the command on what output it produced.
Previously duplicate detection for local variables happened during
`Initialise`, through a call to `checkForDuplicateLocalDefinition`.
This works in a majority of cases, but for commands like `console`, this
was not detected as the return diagnostics for `Initialise` are ignored.
That check can be done as early as during parsing however, as the names
of blocks are not dynamic in the slightest (no interpolation possible),
so we move that detection logic into `Parse`, so that the behaviour is
coherent between all commands.
The logic for evaluating local variables used to rely on their
definition order, with some edge cases. Typically `locals` blocks define
multiple local variables, which don't necessarily appear in the same
order at evaluation as within the template, leading to inconsistent
behaviour, as the order in which those are added to the list of local
variables is non-deterministic.
To avoid this problem, we change how local variables are evaluated, and
we're adopting a workflow similar to datasources, where the local
variables first build a list of direct dependencies. Then when
evaluation happens, we evaluate all the dependencies recursively for
each local variable, which takes care of this issue.
As with Datasources, we add a cap to the recursion: 10. I.e. if the
evaluation of a single variable provokes an infinite recursion, we stop
at that point and return an error to the user, telling them to fix their
template.
The SkipNoAcc function on PackerTestSuite allows to mark a test run as
not to be run every time we run `make test`, but only when PACKER_ACC=1
is set in the environment.
This allows us to skip executing tests that are either long-running, or
that depend on external dependencies (typically Github), which have a
higher potential to fail on a normal run of Packer tests.
When a test fails to exert its assertions on the command-line output, a
test fails, but we don't necessarily can troubleshoot what happened,
especially when this happens in a CI environment.
Therefore, for convenience, we add the faculty for packerCommand.Assert
to automatically dump a command's output (both stdout and stderr) if a
test fails.
Some commands need to have an input in order to work.
For those, we add the capability for the packerCommand struct to have
their stdin defined from a string, which is then fed to the command
being executed.
When a Packer command is created for testing the tool, we generally run
it once, then the command is essentially nooping.
This change allows us to run Packer multiple times with the same
parameters, and make sure all runs conform to a specific list of checks.
This allows us to more reliably test non-deterministic behaviours.
