Commit graph

68 commits

Author SHA1 Message Date
Lucas Bajolet
c0e7e7bd3c hcl2: report error on build without sources
When a template describes a build block without a source reference, the
build should be considered invalid as we won't have a CoreBuild produced
as a result of the need to have both.

In current versions of Packer, this will produce an error message
hinting that nothing will happen because of the lack of either build or
source block.

This commit takes the defined block, and points out to it as missing a
source block as being the reason why nothing is happening, making it
clearer what is required for an HCL2 build to be processed.
2022-09-30 15:39:27 -04:00
Lucas Bajolet
61c56e161c hcl2: add example for undefined variable set warn
When a variable is set in a pkrvars file, but isn't defined, an error
message is output, but does not deliver an example of what is expected
by Packer in order to complete a build.

To remedy that, we improve the error message by giving an example of
variable block to include in the build template.
2022-09-28 14:51:28 -04:00
Lucas Bajolet
e94b8766cb build: exit immediately if no builds and diags
When a template with some builds to run ends its GetBuilds with an
error, and no builds produced, we can exit immediately without printing
more errors.
2022-09-23 12:07:32 -04:00
Eng Zer Jun
f5ec0f1c57 test: use T.Setenv to set env vars in tests
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-09-20 14:54:20 -04:00
Lucas Bajolet
a298617ecb command/build: warn and exit when nothing to build
When a `packer build' is run on a config that does not define anything
to be built, the process would run normally, and only state that no
artifacts were produced after builds were run successfully.

This is misleading as we reach the end of the command without having a
proper warning that nothing was even attempted to be built.

This commit adds a warning when that happens, and leaves immediately
after.
2022-09-13 15:58:28 -04:00
Adrien Delorme
0362a3ec10
all shells: Pass env vars through a key value store (#11569)
* allowing to set shell env vars from a key/value map.
* tests

This new map will work alongside the `environment_vars` but it allows using a
datasource value seamlessly. At validation, and because `environment_vars` was
an array of strings containing `KEY=value`, using datsources would not work,
because these values are not known yet and would evaluate to `<unknown>`. With
this, the value and the key can be unknown and will validate correctly for
datasources.
2022-02-16 12:05:44 +01:00
Adrien Delorme
e03ad29ca0
Sanitize var code (#11566)
* evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs

* Make default input variable type the DynamicPseudoType

This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty.

* do not take Empty types from default value

* Update types.variables.go

Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
2022-02-14 11:00:41 -05:00
Adrien Delorme
3278c1a817
Packer SVC acc tests (#11274) 2021-12-01 15:58:33 +01:00
Wilken Rivera
4a0fbd911a
Add support for variable interpolation to hcp_packer_registry and build blocks. (#11421)
* Add variable interpolation support to hcp_packer_registry block

* Add tests for variable interpolation in hcp_packer_registry block

* Add variable interpolation support to the build block (#11425)

* Add simple test to build command
2021-12-01 11:35:51 +01:00
Adrien Delorme
710d109c55
HCL2: fix a crash when an HCL2 component is nil after a misconfiguration (#11382)
* add test for crash

* fix

* show more precise error message when parse of a provisioner fails

* Revert "fix"

This reverts commit fc88e89fbb.

* add better severity
2021-11-04 10:08:27 -04:00
Adrien Delorme
254e020b08
Allow to use build variables in a post processor (#11323)
* hcl: add test to verify that build.name can be used in a pp

* allow to use build vars in post processors
2021-10-11 10:17:34 -04:00
Adrien Delorme
3135714465
rename all things 'inexist[e|a]nt' to 'nonexistent' & fix tests (#11325)
+ a typo
2021-10-11 15:48:51 +02:00
Adrien Delorme
9d0193ac14
add possibility to use variables everywhere in post-processor blocks (#11094)
feature + tests.

Including in:
* name
* only
* except
* keep_input_artifact

Example file:

```hcl
source "null" "example1" {
  communicator = "none"
}

source "null" "example2" {
  communicator = "none"
}

locals {
  except = "null.example1"
}

variable "only" {
  default = "null.example1"
}

build {
  sources = ["source.null.example1", "source.null.example2"]
  post-processor "shell-local" {
    except = [local.except]
    inline = ["echo first post-processor"]
  }

  post-processor "shell-local" {
    only   = [var.only]
    inline = ["echo second post-processor"]
  }
}
```
Ouput:
```shell-session
$ packer build foo.pkr.hcl
null.example1: output will be in this color.
null.example2: output will be in this color.

==> null.example1: Running post-processor:  (type shell-local)
==> null.example2: Running post-processor:  (type shell-local)
==> null.example2 (shell-local): Running local shell script: /var/folders/3k/2gb5ct4s7cncr52_jh2kz6cw0000gq/T/packer-shell201696062
==> null.example1 (shell-local): Running local shell script: /var/folders/3k/2gb5ct4s7cncr52_jh2kz6cw0000gq/T/packer-shell494781572
    null.example1 (shell-local): second post-processor
Build 'null.example1' finished after 61 milliseconds 432 microseconds.
    null.example2 (shell-local): first post-processor
Build 'null.example2' finished after 111 milliseconds 678 microseconds.

==> Wait completed after 111 milliseconds 714 microseconds
```

close #4895
2021-06-15 18:29:26 +02:00
Adrien Delorme
77a29fc2f8
Allow to have dynamic blocks in a build block + tests (#10825)
This :
* allows to have a `build.dynamic` block
* add tests
* makes sure to show a correct message when a source was not found
  * display only name of source (instead of a weird map printout) 
  * use a "Did you mean %q" feature where possible 


Because dynamic blocks need all variables to be evaluated and available, I moved parsing of everything that is not a variable to "after" variables are extrapolated. Meaning that dynamic block get expanded in the `init` phase and then only we start interpreting HCL2 content.

After #10819 fix #10657
2021-03-30 15:53:04 +02:00
Adrien Delorme
a115b428ac simplify fmt test case a little 2021-03-12 11:07:26 +01:00
Sylvia Moss
ef4afafde9
Fix force flag for hcl2 provisioners and post-processors (#10571) 2021-02-05 10:57:14 +01:00
Adrien Delorme
ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Sylvia Moss
291121dd55
(2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Adrien Delorme
52d2d7fe5f
Copy MapOf... plugin types back into Packer core (#10466)
* move maps of plugins back in core

* go mod vendor

* more fixes

* fix imports

* Update core_test.go

* fix build

* more fixes

* more fixes

* up vendors after fixing sdk

* Update post_processor_mock.hcl2spec.go

* Leave implementatino of MapOf in the sdk for plugi tests

Other wise use the interface

* go mod tidy

* add MapOfDatasource type too
2021-01-13 12:14:06 +01:00
Megan Marsh
88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Megan Marsh
5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh
b69d69095e move postprocessor to sdk, fix generation code 2020-12-01 14:48:55 -08:00
Megan Marsh
be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh
ada91b24e9 extract builder interface to sdk 2020-12-01 13:42:42 -08:00
Megan Marsh
d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Adrien Delorme
6dd06fad14 add command/ tests 2020-11-02 15:50:38 +01:00
Megan Marsh
66b81d9bee
add build.name to hcl code (#10114) 2020-10-16 10:57:42 +02:00
Adrien Delorme
3d7ab9ea43 add an hcl2-json test 2020-09-18 16:38:04 +02:00
sylviamoss
39a8dee4ea add tests and update override example 2020-08-14 11:22:51 +02:00
Chris Gilmer
6e918c4bd3
Update test for builder output 2020-08-11 15:25:33 -07:00
Adrien Delorme
125178d943
core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622)
* core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it.
* HCL2: add path variables + docs & tests

Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-07-23 09:25:07 +02:00
Adrien Delorme
a15ad19411 test builds can be 'excepted' 2020-07-07 11:41:24 +02:00
Adrien Delorme
c1c8b8d22a complete -> recipes 2020-07-07 11:31:19 +02:00
Adrien Delorme
0deaa5d2a5 add a complete tests for builder variables + only/except 2020-07-06 17:01:55 +02:00
Adrien Delorme
7969b690de Revert "Share build info with Provisioner and Post-Processor via HCL2 variables (#9444)"
This reverts commit 99046c9178.
2020-07-02 15:07:29 +02:00
Sylvia Moss
99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
Adrien Delorme
fb337f8867
hcl work on only/except (#9454)
* HCL2: allow to skip a named build block too

* test that excepting a build block works

* test only on a named build block

* add/update docs
2020-06-23 10:53:16 +02:00
Moss
daaeccd8b0 Skip consecutives post-processors 2020-06-09 17:35:53 +02:00
Moss
bfb9df0039 fix except flag for JSON and HCL2 2020-06-09 15:26:45 +02:00
Adrien Delorme
e6aac8cd6f Update build_test.go 2020-05-08 17:50:48 +02:00
Adrien Delorme
efcc6af06c fix tests 2020-05-08 17:46:33 +02:00
Adrien Delorme
42a05e1e80 more refactoring 2020-05-08 16:41:47 +02:00
Tom Dyas
c0a6623ea2
teach HCL mode builds to honor -only and -except options (#8947) 2020-04-28 15:03:24 +02:00
Adrien Delorme
2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Megan Marsh
f5c98a7601
make sure cli vars supercede var files (#8964) 2020-03-30 10:31:59 +02:00
Adrien Delorme
ad8dafa3bd
HCL: add tests and fixes around var-file and var args (#8914) 2020-03-19 13:57:22 +01:00
Sylvia Moss
23fa3107a3
Render variables one time on prepare method (#8727) 2020-02-13 17:35:23 +01:00
Adrien Delorme
0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Moss
7466c4fdca Return exit code 1 when builder type is not found 2019-12-10 18:55:18 +01:00
Megan Marsh
0683bc409b add test for error-cleanup provisioner, and fix tests by fixing null builder to use an actual none communicator instead of skipping communicator generation altogether 2019-09-25 13:39:09 -07:00