Commit graph

722 commits

Author SHA1 Message Date
Lucas Bajolet
68ddb2f89d
hcl2_upgrade: add share_image_gallery workaround (#12087)
In azure templates, the shared_image_gallery was mistakenly considered
an attribute while this is supposed to be a block.

This is due to a heuristic we use for deciding whether a JSON object is
to be translated to an attribute or a block that fell short as the
shared_image_gallery does not contain complex types.

This cannot be fixed trivially for the general case, so we add this
entity to the list of workarounds until we can implement something more
robust.
2022-10-27 18:09:40 -04:00
Jenna Goldstrich
0bac9b6f63
Remove a skip for darwin arm64 (m1/m2 macs) for exec tests. (#12088) 2022-10-27 17:52:26 -04:00
Lucas Bajolet
296562614f hcl2_upgrade: fix liniting issues with fmt.Sprintf
Some literal strings without format-string directives nor arguments were
littered around the hcl2_upgrade command's code, which is pointed out by
the linters.

Since they serve no purpose, we remove them.
2022-10-26 15:35:02 -04:00
Lucas Bajolet
3c249c0d42 hcl2_upgrade: replace passthroughs by pre-visit
In previous versions of hcl2_upgrade, we referenced a list of
passthrough statements from the original JSON template to be replaced by
their equivalent Go-template in HCL2.

This works but requires us to explicitely accept every possible
templated variable in order to not ever encounter <no value> in our
generated HCL2 templates.

This is suboptimal, hence this commit changes approach by pre-visting
the AST from the original Go Template, escaping every sequence that is
not covered by the list of functions we can migrate. Pipes are also
excluded from this function.
2022-10-26 15:35:02 -04:00
Wilken Rivera
7cb0c98925
Merge pull request #12033 from hashicorp/error_messages_rework
[HPR-575] command/build: rework HCP-related error messages
2022-10-21 11:25:15 -04:00
Zach Shilton
879fd9035c
fix: update redirected and broken links (#12060)
* website: fix broken links on /docs/templates

* fix: redirected install-plugins link

* fix: debugging link

* fix: secrets manager link in docs

* fix: secrets manager link in source

* fix: amazon ami plugin link in docs

* fix: amazon ami plugin link in source

* fix: extending plugins link

* fix: plugins/builders/amazon links

* fix: various builders links

* fix: various amazon builder links

* fix: redirected terminology link

* fix: custom-provisioners link

* fix: docker-push redirected plugin link

* fix: googlecompute plugin links

* fix: hyperv iso plugin links

* website: update link to hcl upgrade guide

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
2022-10-21 11:00:58 -04:00
Lucas Bajolet
61c810e720 command: rework HCP-related error messages
The HCP error messages were sometimes a bit terse in terms of the
information it bubbles up to the user.
This made them useful for people who already knew what to look for, but
newcomers would almost certainly be lost because of the lack of
information in those.

To improve on this situation, we reword most of those error messages in
this commit.
2022-10-21 10:32:45 -04:00
Lucas Bajolet
35496e533d packer: include builder type in HCP name for JSON
JSON templates used only to report the builder's type in HCP builds,
even if the name was specified.

This commit changes this behaviour to include both if they're available,
in a similar fashion as what is done on the HCL2 templates.
2022-10-20 17:12:33 -04:00
Wilken Rivera
12e9316f94 Add documenation for HCPConfigMode type 2022-10-13 15:13:29 -04:00
Wilken Rivera
148d5e3a2d Add initialize method to Iteration
Fingerprint initialize was previously occurring during the creation of a
bucket. We want to be able to initialize a bucket and defer the setting
of a fingerprint to a later point.

* Update test to reflect new function signatures for Bucket and Iteration
2022-10-13 15:13:29 -04:00
Wilken Rivera
7f7f397124 Rename withHCLBucketConfiguration function 2022-10-13 15:13:29 -04:00
Wilken Rivera
a4a97d5c05 Move datasource data into its own bucketConfiguratioOpts func 2022-10-13 15:13:29 -04:00
Wilken Rivera
cf9304266e Consolidate bucket configuration logic 2022-10-13 15:13:29 -04:00
Lucas Bajolet
b3188637bf registry: load bucket slug in command/registry
Loadgin the slug from the environment within the Bucket implied that the
Bucket was successfully created, which may fail outside of a Git
directory when the iteration fingerprint is not set through environment
variables.

To make it possible to print as much information regarding the
environment to the users at once rather than stopping immediately when
this step fails, we move the logic to read the default value from the
environment to the HCP setup code.
2022-10-13 15:13:29 -04:00
Lucas Bajolet
9267d933bc command: report multiple errors in bad HCP config
When HCP is detected to be enabled, but some configuration is missing,
we returned immediately on the first error.

This commit changes this behaviour by reporting every error at once, so
users will know immediately if something is wrong when they invoke
Packer with HCP support, and one or more environment variables is not
defined as we'd expect them.
2022-10-13 15:13:29 -04:00
Lucas Bajolet
ae15ed339c registry: add function to detect explicit HCP
The HCP_PACKER_REGISTRY environment variable had its behaviour changed
recently, as prior versions of Packer expected the attribute to be
forcefully set to a value that is neither "off" nor "0" in order to
get HCP integration to work, or that a "hcp_packer_registry" block was
defined in an HCL template. Now, this environment variable defaults to
not explicitely enable, but instead to explicitely disable HCP
integration, and the feature switch fall upon HCP_PACKER_BUCKET_NAME.

As an extra feature, we keep the prior behaviour alive when it is
explicitely defined as a value to enable it. That way we can report
errors if the rest is not defined, rather than silently ignore it.

This function we add to env is the first stone to enable this behaviour.
2022-10-13 15:13:29 -04:00
Lucas Bajolet
c0e7e7bd3c hcl2: report error on build without sources
When a template describes a build block without a source reference, the
build should be considered invalid as we won't have a CoreBuild produced
as a result of the need to have both.

In current versions of Packer, this will produce an error message
hinting that nothing will happen because of the lack of either build or
source block.

This commit takes the defined block, and points out to it as missing a
source block as being the reason why nothing is happening, making it
clearer what is required for an HCL2 build to be processed.
2022-09-30 15:39:27 -04:00
Lucas Bajolet
61c56e161c hcl2: add example for undefined variable set warn
When a variable is set in a pkrvars file, but isn't defined, an error
message is output, but does not deliver an example of what is expected
by Packer in order to complete a build.

To remedy that, we improve the error message by giving an example of
variable block to include in the build template.
2022-09-28 14:51:28 -04:00
Lucas Bajolet
73af77aeed command: enhance error message on no build to run 2022-09-23 12:07:32 -04:00
Lucas Bajolet
e94b8766cb build: exit immediately if no builds and diags
When a template with some builds to run ends its GetBuilds with an
error, and no builds produced, we can exit immediately without printing
more errors.
2022-09-23 12:07:32 -04:00
Lucas Bajolet
3f6c0a5f23 command: move func to extract stdout/err in tests
In order to test the output from a test command, the commandOutput
function exists in the command_test.go file.

Since its behaviour is linked to the test meta that we produce in the
test_utils.go file, we move it there, and rename it to
`GetStdoutAndErrFromTestMeta' to make it clearer what to expect from it.
2022-09-23 12:07:32 -04:00
Eng Zer Jun
f5ec0f1c57 test: use T.Setenv to set env vars in tests
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-09-20 14:54:20 -04:00
Lucas Bajolet
779a31a25f hcp: use HCP_PACKER_REGISTRY to disable HCP
The current behaviour for HCP integration is based on the presence of
the HCP_PACKER_REGISTRY environment variable, where if it is either not
"0" or "off", the HCP integration is enabled.

This commit changes the behaviour to only use this variable to
explicitely disable HCP, and the HCP_PACKER_BUCKET_NAME variable will
condition if the HCP integration is enabled for a build or not in JSON
templates.

For HCL templates, the integration can also be disabled if the
HCP_PACKER_REGISTRY variable is set to "0" or "off", and will be
considered enabled if the HCP_PACKER_BUCKET_NAME is set, or there is a
"hcp_packer_registry" block in a build.
2022-09-19 11:22:13 -04:00
Wilken Rivera
11e71729f1
Remove Oracle plugin from the list of vendored plugins (#11983)
The latest release of the Oracle plugin for Packer removed
Solaris from its list of supported platforms. Since Packer still
releases binaries for Solaris the Oracle plugin will no longer be
bundled and distributed within the Packer binary. Practitioners relying
on the plugin can continue using the plugin by installing it manually
using either the `packer init` or `packer plugins install` commands.
2022-09-15 12:31:52 -04:00
Lucas Bajolet
a298617ecb command/build: warn and exit when nothing to build
When a `packer build' is run on a config that does not define anything
to be built, the process would run normally, and only state that no
artifacts were produced after builds were run successfully.

This is misleading as we reach the end of the command without having a
proper warning that nothing was even attempted to be built.

This commit adds a warning when that happens, and leaves immediately
after.
2022-09-13 15:58:28 -04:00
Lucas Bajolet
7cd095062a parser: extract HCP logic out of the HCL parser
Before this commit, some HCP-related logic was embedded in the parser,
making all of it common to all commands.

This complexifies the logic as some parts rely on the template being in
a HCP-valid environment, and the datasources need to be evaluated in
order for some operations to be executed.

To simplify this and avoid those pitfalls, we move this logic to its own
set of functions that will be performed after the parsing is done, on
any valid HCL2 or JSON template.
2022-08-12 11:54:21 -04:00
Lucas Bajolet
4be7e38474 command: move GetConfig methods to meta
Since the GetConfig methods are common to all the commands of Packer
that rely on a config to work, we move them from build.go to meta.go in
order to make the intent clearer.
2022-08-12 11:54:21 -04:00
Wilken Rivera
e3ad29d2aa
Update third-party plugin imports (#11912)
* packer-plugin-digitalocean: Update module import path

* packer-plugin-outscale: Remove bundled plugin

* packer-plugin-scaleway: Remove bundled plugin
2022-07-29 11:45:30 -04:00
teddylear
805225a113
feat: add http datasource (#11658) 2022-05-17 19:26:34 +02:00
Wilken Rivera
a7c37ee34d
Add tests for the packer plugins subcommand (#11660)
* Add tests for plugins install command

* Add tests for plugins remove command

* Add PACKER_GITHUB_API_TOKEN to all test env

* Update dir hashes for non-darwin os

* Update tests with feedback from review
2022-03-15 16:32:49 -04:00
Wilken Rivera
1c398a73b4
Set default file extension for binary removal on Windows (#11627)
Related to #11625
2022-03-04 10:50:26 +01:00
Wilken Rivera
ea3c73a480
Set default file extension for binary installations on Windows (#11625)
This change sets the default file extension for binary installations on
Windows.

Results before change
```
$ packer plugins install github.com/hashicorp/amazon
9 errors occurred:
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_freebsd_arm64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_darwin_amd64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_freebsd_amd64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_linux_amd64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_darwin_arm64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_windows_arm64.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_windows_386.zip: wrong system, expected windows_amd64
        * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_linux_arm.zip: wrong system, expected windows_amd64
        * could not find a packer-plugin-amazon_v1.0.8_x5.0_windows_amd64.zip file in zipfile

exit status 1
```

Results after change
```
$ packer plugins install github.com/hashicorp/amazon
Installed plugin github.com/hashicorp/amazon v1.0.8 in "C:/Users/Packer/AppData/Roaming/packer.d/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.0.8_x5.0_windows_amd64.exe"
```
2022-03-03 17:30:27 -05:00
Adrien Delorme
0362a3ec10
all shells: Pass env vars through a key value store (#11569)
* allowing to set shell env vars from a key/value map.
* tests

This new map will work alongside the `environment_vars` but it allows using a
datasource value seamlessly. At validation, and because `environment_vars` was
an array of strings containing `KEY=value`, using datsources would not work,
because these values are not known yet and would evaluate to `<unknown>`. With
this, the value and the key can be unknown and will validate correctly for
datasources.
2022-02-16 12:05:44 +01:00
Adrien Delorme
e03ad29ca0
Sanitize var code (#11566)
* evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs

* Make default input variable type the DynamicPseudoType

This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty.

* do not take Empty types from default value

* Update types.variables.go

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
2022-02-14 11:00:41 -05:00
Adrien Delorme
9f4a1281dd
Add packer plugins command and subcommands to interact with plugins (#11553)
* add basic docs for plugins command

* refactor docs

Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com>

* add plugins command

* add plugins subcommands

they do nothing for now

* add plugins installed command + tests

* add plugins install command

* add remove plugin command

* better docs for the plugins install command

* remove duplicate content

* better output for installed plugins

* add plugins required command

* Update plugins_install.go

* add newline after `Usage:`

* Update plugins_remove.go

* Update plugins_required.go

* Update plugins_remove.go

* Update plugins_installed.go

* Update plugins_install.go

* add docs

* Update plugins_install.go

* fix typos

* Update plugins_test.go

* fix typos

Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com>

* Update core_wrapper.go

Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com>

* Update website/content/docs/commands/plugins/remove.mdx

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>

* Update website/content/docs/commands/plugins/required.mdx

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>

* Update website/content/docs/commands/plugins/required.mdx

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>

* Update plugins_required.go

* Update install.mdx

* Update required.mdx

* plugins requirement, warn when no plugin was found

* Update website/content/docs/commands/plugins/required.mdx

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>

Co-authored-by: Wilken Rivera <1749304+nywilken@users.noreply.github.com>
Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
2022-02-10 16:53:50 -05:00
teddylear
1d01ad3651
feat: print all locals errors when there is a circular error (#11527) 2022-01-27 14:40:45 +01:00
Wilken Rivera
58fa1f96e2
Temporarily revert the use of indirect redirects for external plugins (#11430)
The auto installation of external plugins via packer init only work for
HCL templates. Since we have not officially removed all plugins (slated
for 1.8.0) I am reverting this test change. Once 1.7.9 is released we
  can begin removing plugins and optimizing the redirect experience for
  JSON-only users.
2021-12-06 10:55:49 -05:00
Adrien Delorme
3278c1a817
Packer SVC acc tests (#11274) 2021-12-01 15:58:33 +01:00
Wilken Rivera
4a0fbd911a
Add support for variable interpolation to hcp_packer_registry and build blocks. (#11421)
* Add variable interpolation support to hcp_packer_registry block

* Add tests for variable interpolation in hcp_packer_registry block

* Add variable interpolation support to the build block (#11425)

* Add simple test to build command
2021-12-01 11:35:51 +01:00
Adrien Delorme
710d109c55
HCL2: fix a crash when an HCL2 component is nil after a misconfiguration (#11382)
* add test for crash

* fix

* show more precise error message when parse of a provisioner fails

* Revert "fix"

This reverts commit fc88e89fbb.

* add better severity
2021-11-04 10:08:27 -04:00
Adrien Delorme
c54326b030 remove all non official plugins from vendored plugins 2021-11-03 12:03:08 +01:00
Adrien Delorme
faeef90910 remove ansible plugin from bundled plugins 2021-11-03 11:47:29 +01:00
Adrien Delorme
8acd453065 remove vagrant plugin from bundled plugins 2021-11-03 11:46:06 +01:00
Adrien Delorme
bfe200d920 remove qemu plugin from bundled plugins 2021-11-03 11:46:05 +01:00
Adrien Delorme
d9d96169d8 remove googlecompute plugin from bundled plugins 2021-11-03 11:46:05 +01:00
Adrien Delorme
4efa69cfff remove docker plugin from bundled plugins 2021-11-03 11:46:05 +01:00
Adrien Delorme
7a21f141c8 remove vsphere plugin from bundled plugins 2021-11-03 11:46:05 +01:00
Adrien Delorme
eea27b02d5 remove parallels plugin from bundled plugins 2021-11-03 11:46:04 +01:00
Adrien Delorme
36c30706f1 remove vmware plugin from bundled plugins 2021-11-03 11:46:04 +01:00
Adrien Delorme
eca7b442c1 remove virtualbox plugin from bundled plugins 2021-11-03 11:46:04 +01:00