upstream/packer
1
0
Fork 0
mirror of https://github.com/hashicorp/packer.git synced 2026-05-28 04:35:38 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config

Browse source
This commit is contained in:
Adrien Delorme 2019-08-21 12:28:34 +02:00
commit 4cb7c30987
1007 changed files with 142028 additions and 5668 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

151
CHANGELOG.md
View file

@ -1,28 +1,167 @@
## 1.4.2 (upcoming)
## 1.4.4 (Upcoming)
### IMPROVEMENTS:
* builder/openstack: Store WinRM password for provisioners to use [GH-7940]
### BUG FIXES:
* core: Fix bug where sensitive variables contianing commas were not being
properly sanitized in UI calls. [GH-7997]
## 1.4.3 (August 14, 2019)
### IMPROVEMENTS:
* **new builder** UCloud builder [GH-7775]
* **new builder** Outscale [GH-7459]
* **new builder** VirtualBox Snapshot [GH-7780]
* **new builder** JDCloud [GH-7962]
* **new post-processor** Exoscale Import post-processor [GH-7822] [GH-7946]
* build: Change Makefile to behave differently inside and outside the gopath
when generating code. [GH-7827]
* builder/amazon: Don't calculate spot bids; Amazon has changed spot pricing to
no longer require this. [GH-7813]
* builder/google: Add suse-byos-cloud to list of public GCP cloud image
projects [GH-7935]
* builder/openstack: New `image_min_disk` option [GH-7290]
* builder/openstack: New option `use_blockstorage_volume` to set openstack
image metadata [GH-7792]
* builder/openstack: Select instance network on which to assign floating ip
[GH-7884]
* builder/qemu: Implement VNC password functionality [GH-7836]
* builder/scaleway: Allow removing volume after image creation for Scaleway
builder [GH-7887]
* builder/tencent: Add `run_tags` to option to tag instance. [GH-7810]
* builder/tencent: Remove unnecessary image name validation check. [GH-7786]
* builder/tencent: Support data disks for tencentcloud builder [GH-7815]
* builder/vmware: Fix intense CPU usage because of poorly handled errors.
[GH-7877]
* communicator: Use context for timeouts, interruption in ssh and winrm
communicators [GH-7868]
* core: Change how on-error=abort is handled to prevent EOF errors that mask
real issues [GH-7913]
* core: Clean up logging vs ui call in step download [GH-7936]
* core: New environment var option to allow user to set location of config
directory [GH-7912]
* core: Remove obsolete Cancel functions from builtin provisioners [GH-7917]
* post-processor/vagrant: Add option to allow box Vagrantfiles to be generated
during the build [GH-7951]
* provisioner/ansible: Add support for installing roles with ansible-galaxy
[GH-7916
* provisioner/salt-masterless: Modify file upload to handle non-root case.
[GH-7833]
### BUG FIXES:
* builder/amazon: Add error to warn users of spot_tags regression. [GH-7989]
* builder/amazon: Allow EC2 Spot Fleet packer instances to run in parallel
[GH-7818]
* builder/amazon: Fix failures and duplication in Amazon region copy and
encryption step. [GH-7870] [GH-7923]
* builder/amazon: No longer store names of volumes which get deleted on
termination inside ebssurrogate artifact. [GH-7829]
* builder/amazon: Update aws-sdk-go to v1.22.2, resolving some AssumeRole
issues [GH-7967]
* builder/azure: Create configurable polling duration and set higher default
for image copies to prevent timeouts on successful copies [GH-7920]
* builder/digitalocean: increase timeout for Digital Ocean snapshot creation.
[GH-7841]
* builder/docker: Check container os, not host os, when creating container dir
default [GH-7939]
* builder/docker: Fix bug where PACKER_TMP_DIR was created with root perms on
linux [GH-7905]
* builder/docker: Fix file download hang caused by blocking ReadAll call
[GH-7814]
* builder/google: Fix outdated oauth URL. [GH-7835] [GH-7927]
* builder/hyperv: Improve code for detecting IP address [GH-7880]
* builder/ucloud: Update the api about stop instance to fix the read-only image
build by ucloud-uhost [GH-7914]
* builder/vagrant: Fix bug where source_path was being used instead of box_name
when generating the Vagrantfile. [GH-7859]
* builder/virtualbox: Honor value of 'Comment' field in ssh keypair generation.
[GH-7922]
* builder/vmware: Fix validation regression that occurred when user provided a
checksum file [GH-7804]
* buildere/azure: Fix crash with managed images not published to shared image
gallery. [GH-7837]
* communicator/ssh: Move ssh_interface back into individual builders from ssh
communicator to prevent validation issues where it isn't implemented.
[GH-7831]
* console: Fix console help text [GH-7960]
* core: Fix bug in template parsing where function errors were getting
swallowed. [GH-7854]
* core: Fix regression where a local filepath containing `//` was no longer
properly resolving to `/`. [GH-7888]
* core: Fix regression where we could no longer access isos on SMB shares.
[GH-7800]
* core: Make ssh_host template option always override all builders' IP
discovery. [GH-7832]
* core: Regenerate boot_command PEG code [GH-7977]
* fix: clean up help text and fixer order to make sure all fixers are called
[GH-7903]
* provisioner/inspec: Use --input-file instead of --attrs to avoid deprecation
warning [GH-7893]
* provisioner/salt-masterless: Make salt-masterless provisioner respect
disable_sudo directive for all commands [GH-7774]
## 1.4.2 (June 26, 2019)
### IMPROVEMENTS:
* **new feature:** Packer console [GH-7726]
* builder/alicloud: cleanup image and snapshot if target image is still not
available after timeout [GH-7744]
* builder/alicloud: let product API determine the default value of io_optimized
[GH-7747]
* builder/amazon: Add new `skip_save_build_region` option to fix naming
conflicts when building in a region you don't want the final image saved
in. [GH-7759]
* builder/amazon: Add retry for temp key-pair generation in amazon-ebs
[GH-7731]
* builder/amazon: Enable encrypted AMI sharing across accounts [GH-7707]
* builder/amazon: New SpotInstanceTypes feature for spot instance users.
[GH-7682]
* builder/azure: Allow users to publish Managed Images to Azure Shared Image
Gallery (same Subscription) [GH-7778]
* builder/azure: Update Azure SDK for Go to v30.0.0 [GH-7706]
* builder/cloudstack: Add tags to instance upon creation [GH0-7526]
* builder/cloudstack: Add tags to instance upon creation [GH-7526]
* builder/docker: Better windows defaults [GH-7678]
* builder/google: Add feature to import user-data from a file [GH-7720]
* builder/hyperv: Abort build if there's a name collision [GH-7746]
* builder/hyperv: Clarify pathing requirements for hyperv-vmcx [GH-7790]
* builder/hyperv: Increase MaxRamSize to match modern Windows [GH-7785]
* builder/openstack: Add image filtering on properties. [GH-7597]
* provisioner/powershell: Fix null file descriptor error that occurred when
remote_path provided is a directory and not a file. [GH-7705]
* builder/qemu: Add additional disk support [GH-7791]
* builder/vagrant: Allow user to override vagrant ssh-config details [GH-7782]
* builder/yandex: Gracefully shutdown instance, allow metadata from file, and
create preemptible instance type [GH-7734]
* core: scrub out sensitive variables in scrub out sensitive variables logs
[GH-7743]
### BUG FIXES:
* builder/alicloud: Fix describing snapshots issue when image_ignore_data_disks
is provided [GH-7736]
* builder/amazon: Fix bug in region copy which produced badly-named AMIs in the
build region. [GH-7691]
* builder/amazon: Fix failure that happened when spot_tags was set but ami_tags
wasn't [GH-7699]
wasn't [GH-7712]
* builder/cloudstack: Update go-cloudstack sdk, fixing compatability with
CloudStack v 4.12 [GH-7694]
* builder/proxmox: Update proxmox-api-go dependency, fixing issue calculating
VMIDs. [GH-7755]
* builder/tencent: Correctly remove tencentcloud temporary keypair. [GH-7787]
* core: Allow timestamped AND colorless ui messages [GH-7769]
* core: Apply logSecretFilter to output from ui.Say [GH-7739]
* core: Fix "make bin" command to use reasonbale defaults. [GH-7752]
* core: Fix user var interpolation for variables set via -var-file and from
command line [GH-7733]
* core: machine-readable UI now writes UI calls to logs. [GH-7745]
* core: Switch makefile to use "GO111MODULE=auto" to allow for modern gomodule
usage. [GH-7753]
* provisioner/ansible: prevent nil pointer dereference after a language change
[GH-7738]
* provisioner/chef: Accept chef license by default to prevent hangs in latest
Chef [GH-7653]
* provisioner/powershell: Fix crash caused by error in retry logic check in
powershell provisioner [GH-7657]
* provisioner/powershell: Fix null file descriptor error that occurred when
remote_path provided is a directory and not a file. [GH-7705]
## 1.4.1 (May 15, 2019)

3
CODEOWNERS
View file

@ -16,7 +16,9 @@
/builder/scaleway/ @sieben @mvaude @jqueuniet @fflorens @brmzkw
/builder/hcloud/ @LKaemmerling
/builder/hyperone @m110 @gregorybrzeski @ad-m
/builder/ucloud/ @shawnmssu
/builder/yandex @GennadySpb @alexanderKhaustov @seukyaso
/builder/osc @marinsalinas
# provisioners
@ -27,5 +29,6 @@
/post-processor/alicloud-import/ dongxiao.zzh@alibaba-inc.com
/post-processor/checksum/ v.tolstov@selfip.ru
/post-processor/exoscale-import/ @falzm @mcorbin
/post-processor/googlecompute-export/ crunkleton@google.com
/post-processor/vsphere-template/ nelson@bennu.cl

7
Makefile
View file

@ -46,7 +46,12 @@ install-build-deps: ## Install dependencies for bin build
install-gen-deps: ## Install dependencies for code generation
@go get golang.org/x/tools/cmd/goimports
@go get -u github.com/mna/pigeon
@./scripts/off_gopath.sh; if [ $$? -eq 0 ]; then \
go get github.com/mna/pigeon@master; \
else \
go get -u github.com/mna/pigeon; \
fi
@go get github.com/alvaroloes/enumer
@go install ./cmd/struct-markdown

2
builder/alicloud/ecs/step_check_source_image.go
View file

@ -28,7 +28,7 @@ func (s *stepCheckAlicloudSourceImage) Run(ctx context.Context, state multistep.
images := imagesResponse.Images.Image
// Describe markerplace image
// Describe marketplace image
describeImagesRequest.ImageOwnerAlias = "marketplace"
marketImagesResponse, err := client.DescribeImages(describeImagesRequest)
if err != nil {

2
builder/amazon/common/access_config.go
View file

@ -253,7 +253,7 @@ func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
}
// Make sure it's obvious from the config how we're getting credentials:
// Vault, Packer config, or environemnt.
// Vault, Packer config, or environment.
if !c.VaultAWSEngine.Empty() {
if len(c.AccessKey) > 0 {
errs = append(errs,

12
builder/amazon/common/ami_config.go
View file

@ -12,6 +12,7 @@ import (
// AMIConfig is for common configuration related to creating AMIs.
type AMIConfig struct {
// The name of the resulting AMI that will appear when
// managing AMIs in the AWS console or via APIs. This must be unique. To help
// make this unique, use a function like timestamp (see [template
@ -96,7 +97,18 @@ type AMIConfig struct {
// conjunction with `snapshot_users` -- in that situation you must use
// custom keys. For valid formats see *KmsKeyId* in the [AWS API docs -
// CopyImage](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CopyImage.html).
//
// This option supercedes the `kms_key_id` option -- if you set both, and
// they are different, Packer will respect the value in
// `region_kms_key_ids` for your build region and silently disregard the
// value provided in `kms_key_id`.
AMIRegionKMSKeyIDs map[string]string `mapstructure:"region_kms_key_ids" required:"false"`
// If true, Packer will not check whether an AMI with the `ami_name` exists
// in the region it is building in. It will use an intermediary AMI name,
// which it will not convert to an AMI in the build region. It will copy
// the intermediary AMI into any regions provided in `ami_regions`, then
// delete the intermediary AMI. Default `false`.
AMISkipBuildRegion bool `mapstructure:"skip_save_build_region"`
// Tags to apply to snapshot.
// They will override AMI tags if already applied to snapshot. This is a
// [template engine](../templates/engine.html), see [Build template

13
builder/amazon/common/block_device.go
View file

@ -55,6 +55,14 @@ type BlockDevice struct {
// The size of the volume, in GiB. Required if not specifying a
// snapshot_id.
VolumeSize int64 `mapstructure:"volume_size" required:"false"`
// ID, alias or ARN of the KMS key to use for boot volume encryption. This
// only applies to the main region, other regions where the AMI will be
// copied will be encrypted by the default EBS KMS key. For valid formats
// see KmsKeyId in the [AWS API docs -
// CopyImage](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CopyImage.html)
// This field is validated by Packer, when using an alias, you will have to
// prefix kms_key_id with alias/.
KmsKeyId string `mapstructure:"kms_key_id" required:"false"`
}
type BlockDevices []BlockDevice
@ -117,6 +125,11 @@ func (b *BlockDevice) Prepare(ctx *interpolate.Context) error {
return fmt.Errorf("The `device_name` must be specified " +
"for every device in the block device mapping.")
}
// Warn that encrypted must be true or nil when setting kms_key_id
if b.KmsKeyId != "" && b.Encrypted != nil && *b.Encrypted == false {
return fmt.Errorf("The device %v, must also have `encrypted: "+
"true` when setting a kms_key_id.", b.DeviceName)
}
return nil
}

29
builder/amazon/common/run_config.go
View file

@ -314,7 +314,8 @@ type RunConfig struct {
WindowsPasswordTimeout time.Duration `mapstructure:"windows_password_timeout" required:"false"`
// Communicator settings
Comm communicator.Config `mapstructure:",squash"`
Comm communicator.Config `mapstructure:",squash"`
SSHInterface string `mapstructure:"ssh_interface"`
}
func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
@ -340,12 +341,12 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
errs := c.Comm.Prepare(ctx)
// Validating ssh_interface
if c.Comm.SSHInterface != "public_ip" &&
c.Comm.SSHInterface != "private_ip" &&
c.Comm.SSHInterface != "public_dns" &&
c.Comm.SSHInterface != "private_dns" &&
c.Comm.SSHInterface != "" {
errs = append(errs, fmt.Errorf("Unknown interface type: %s", c.Comm.SSHInterface))
if c.SSHInterface != "public_ip" &&
c.SSHInterface != "private_ip" &&
c.SSHInterface != "public_dns" &&
c.SSHInterface != "private_dns" &&
c.SSHInterface != "" {
errs = append(errs, fmt.Errorf("Unknown interface type: %s", c.SSHInterface))
}
if c.Comm.SSHKeyPairName != "" {
@ -379,20 +380,6 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
"block_duration_minutes must be multiple of 60"))
}
if c.SpotPrice == "auto" {
if c.SpotPriceAutoProduct == "" {
errs = append(errs, fmt.Errorf(
"spot_price_auto_product must be specified when spot_price is auto"))
}
}
if c.SpotPriceAutoProduct != "" {
if c.SpotPrice != "auto" {
errs = append(errs, fmt.Errorf(
"spot_price should be set to auto when spot_price_auto_product is specified"))
}
}
if c.SpotTags != nil {
if c.SpotPrice == "" || c.SpotPrice == "0" {
errs = append(errs, fmt.Errorf(

14
builder/amazon/common/run_config_test.go
View file

@ -119,7 +119,6 @@ func TestRunConfigPrepare_EnableT2UnlimitedBadWithSpotInstanceRequest(t *testing
c.InstanceType = "t2.micro"
c.EnableT2Unlimited = true
c.SpotPrice = "auto"
c.SpotPriceAutoProduct = "Linux/UNIX"
err := c.Prepare(nil)
if len(err) != 1 {
t.Fatalf("Should error if T2 Unlimited has been used in conjuntion with a Spot Price request")
@ -129,19 +128,14 @@ func TestRunConfigPrepare_EnableT2UnlimitedBadWithSpotInstanceRequest(t *testing
func TestRunConfigPrepare_SpotAuto(t *testing.T) {
c := testConfig()
c.SpotPrice = "auto"
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if spot_price_auto_product is not set and spot_price is set to auto")
}
// Good - SpotPrice and SpotPriceAutoProduct are correctly set
c.SpotPriceAutoProduct = "foo"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
c.SpotPrice = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if spot_price is not set to auto and spot_price_auto_product is set")
// Shouldn't error (YET) even though SpotPriceAutoProduct is deprecated
c.SpotPriceAutoProduct = "Linux/Unix"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
}

2
builder/amazon/common/state.go
View file

@ -35,7 +35,7 @@ type StateChangeConf struct {
}
// Following are wrapper functions that use Packer's environment-variables to
// determing retry logic, then call the AWS SDK's built-in waiters.
// determine retry logic, then call the AWS SDK's built-in waiters.
func WaitUntilAMIAvailable(ctx aws.Context, conn ec2iface.EC2API, imageId string) error {
imageInput := ec2.DescribeImagesInput{

66
builder/amazon/common/step_ami_region_copy.go
View file

@ -21,19 +21,58 @@ type StepAMIRegionCopy struct {
Name string
OriginalRegion string
toDelete string
getRegionConn func(*AccessConfig, string) (ec2iface.EC2API, error)
toDelete string
getRegionConn func(*AccessConfig, string) (ec2iface.EC2API, error)
AMISkipBuildRegion bool
}
func (s *StepAMIRegionCopy) DeduplicateRegions(intermediary bool) {
// Deduplicates regions by looping over the list of regions and storing
// the regions as keys in a map. This saves users from accidentally copying
// regions twice if they've added a region to a map twice.
RegionMap := map[string]bool{}
RegionSlice := []string{}
// Original build region may or may not be present in the Regions list, so
// let's make absolutely sure it's in our map.
RegionMap[s.OriginalRegion] = true
for _, r := range s.Regions {
RegionMap[r] = true
}
if !intermediary || s.AMISkipBuildRegion {
// We don't want to copy back into the original region if we aren't
// using an intermediary image, so remove the original region from our
// map.
// We also don't want to copy back into the original region if the
// intermediary image is because we're skipping the build region.
delete(RegionMap, s.OriginalRegion)
}
// Now print all those keys into the region slice again
for k, _ := range RegionMap {
RegionSlice = append(RegionSlice, k)
}
s.Regions = RegionSlice
}
func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
amis := state.Get("amis").(map[string]string)
snapshots := state.Get("snapshots").(map[string][]string)
intermediary, _ := state.Get("intermediary_image").(bool)
s.DeduplicateRegions(intermediary)
ami := amis[s.OriginalRegion]
// Always copy back into original region to preserve the ami name
s.toDelete = ami
s.Regions = append(s.Regions, s.OriginalRegion)
// Make a note to delete the intermediary AMI if necessary.
if intermediary {
s.toDelete = ami
}
if s.EncryptBootVolume != nil && *s.EncryptBootVolume {
// encrypt_boot is true, so we have to copy the temporary
@ -42,7 +81,11 @@ func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) m
if s.RegionKeyIds == nil {
s.RegionKeyIds = make(map[string]string)
}
s.RegionKeyIds[s.OriginalRegion] = s.AMIKmsKeyId
// Make sure the kms_key_id for the original region is in the map
if _, ok := s.RegionKeyIds[s.OriginalRegion]; !ok {
s.RegionKeyIds[s.OriginalRegion] = s.AMIKmsKeyId
}
}
if len(s.Regions) == 0 {
@ -53,15 +96,18 @@ func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) m
var lock sync.Mutex
var wg sync.WaitGroup
var regKeyID string
errs := new(packer.MultiError)
wg.Add(len(s.Regions))
for _, region := range s.Regions {
var regKeyID string
ui.Message(fmt.Sprintf("Copying to: %s", region))
if s.EncryptBootVolume != nil && *s.EncryptBootVolume {
// Encrypt is true, explicitly
regKeyID = s.RegionKeyIds[region]
} else {
// Encrypt is nil or false; Make sure region key is empty
regKeyID = ""
}
go func(region string) {
@ -96,6 +142,10 @@ func (s *StepAMIRegionCopy) Cleanup(state multistep.StateBag) {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
if len(s.toDelete) == 0 {
return
}
// Delete the unencrypted amis and snapshots
ui.Say("Deregistering the AMI and deleting unencrypted temporary " +
"AMIs and snapshots")

301
builder/amazon/common/step_ami_region_copy_test.go
View file

@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"sync"
"testing"
"github.com/aws/aws-sdk-go/aws"
@ -25,10 +26,14 @@ type mockEC2Conn struct {
deregisterImageCount int
deleteSnapshotCount int
waitCount int
lock sync.Mutex
}
func (m *mockEC2Conn) CopyImage(copyInput *ec2.CopyImageInput) (*ec2.CopyImageOutput, error) {
m.lock.Lock()
m.copyImageCount++
m.lock.Unlock()
copiedImage := fmt.Sprintf("%s-copied-%d", *copyInput.SourceImageId, m.copyImageCount)
output := &ec2.CopyImageOutput{
ImageId: &copiedImage,
@ -38,7 +43,9 @@ func (m *mockEC2Conn) CopyImage(copyInput *ec2.CopyImageInput) (*ec2.CopyImageOu
// functions we have to create mock responses for in order for test to run
func (m *mockEC2Conn) DescribeImages(*ec2.DescribeImagesInput) (*ec2.DescribeImagesOutput, error) {
m.lock.Lock()
m.describeImagesCount++
m.lock.Unlock()
output := &ec2.DescribeImagesOutput{
Images: []*ec2.Image{{}},
}
@ -46,19 +53,25 @@ func (m *mockEC2Conn) DescribeImages(*ec2.DescribeImagesInput) (*ec2.DescribeIma
}
func (m *mockEC2Conn) DeregisterImage(*ec2.DeregisterImageInput) (*ec2.DeregisterImageOutput, error) {
m.lock.Lock()
m.deregisterImageCount++
m.lock.Unlock()
output := &ec2.DeregisterImageOutput{}
return output, nil
}
func (m *mockEC2Conn) DeleteSnapshot(*ec2.DeleteSnapshotInput) (*ec2.DeleteSnapshotOutput, error) {
m.lock.Lock()
m.deleteSnapshotCount++
m.lock.Unlock()
output := &ec2.DeleteSnapshotOutput{}
return output, nil
}
func (m *mockEC2Conn) WaitUntilImageAvailableWithContext(aws.Context, *ec2.DescribeImagesInput, ...request.WaiterOption) error {
m.lock.Lock()
m.waitCount++
m.lock.Unlock()
return nil
}
@ -84,6 +97,128 @@ func tState() multistep.StateBag {
return state
}
func TestStepAMIRegionCopy_duplicates(t *testing.T) {
// ------------------------------------------------------------------------
// Test that if the original region is added to both Regions and Region,
// the ami is only copied once (with encryption).
// ------------------------------------------------------------------------
stepAMIRegionCopy := StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-east-1"},
AMIKmsKeyId: "12345",
// Original region key in regionkeyids is different than in amikmskeyid
RegionKeyIds: map[string]string{"us-east-1": "12345"},
EncryptBootVolume: aws.Bool(true),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
state.Put("intermediary_image", true)
stepAMIRegionCopy.Run(context.Background(), state)
if len(stepAMIRegionCopy.Regions) != 1 {
t.Fatalf("Should have added original ami to Regions one time only")
}
// ------------------------------------------------------------------------
// Both Region and Regions set, but no encryption - shouldn't copy anything
// ------------------------------------------------------------------------
// the ami is only copied once.
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-east-1"},
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
state.Put("intermediary_image", false)
stepAMIRegionCopy.getRegionConn = getMockConn
stepAMIRegionCopy.Run(context.Background(), state)
if len(stepAMIRegionCopy.Regions) != 0 {
t.Fatalf("Should not have added original ami to Regions; not encrypting")
}
// ------------------------------------------------------------------------
// Both Region and Regions set, but no encryption - shouldn't copy anything,
// this tests false as opposed to nil value above.
// ------------------------------------------------------------------------
// the ami is only copied once.
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-east-1"},
EncryptBootVolume: aws.Bool(false),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
state.Put("intermediary_image", false)
stepAMIRegionCopy.getRegionConn = getMockConn
stepAMIRegionCopy.Run(context.Background(), state)
if len(stepAMIRegionCopy.Regions) != 0 {
t.Fatalf("Should not have added original ami to Regions once; not" +
"encrypting")
}
// ------------------------------------------------------------------------
// Multiple regions, many duplicates, and encryption (this shouldn't ever
// happen because of our template validation, but good to test it.)
// ------------------------------------------------------------------------
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
// Many duplicates for only 3 actual values
Regions: []string{"us-east-1", "us-west-2", "us-west-2", "ap-east-1", "ap-east-1", "ap-east-1"},
AMIKmsKeyId: "IlikePancakes",
// Original region key in regionkeyids is different than in amikmskeyid
RegionKeyIds: map[string]string{"us-east-1": "12345", "us-west-2": "abcde", "ap-east-1": "xyz"},
EncryptBootVolume: aws.Bool(true),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state.Put("intermediary_image", true)
stepAMIRegionCopy.Run(context.Background(), state)
if len(stepAMIRegionCopy.Regions) != 3 {
t.Fatalf("Each AMI should have been added to Regions one time only.")
}
// Also verify that we respect RegionKeyIds over AMIKmsKeyIds:
if stepAMIRegionCopy.RegionKeyIds["us-east-1"] != "12345" {
t.Fatalf("RegionKeyIds should take precedence over AmiKmsKeyIds")
}
// ------------------------------------------------------------------------
// Multiple regions, many duplicates, NO encryption
// ------------------------------------------------------------------------
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
// Many duplicates for only 3 actual values
Regions: []string{"us-east-1", "us-west-2", "us-west-2", "ap-east-1", "ap-east-1", "ap-east-1"},
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state.Put("intermediary_image", false)
stepAMIRegionCopy.Run(context.Background(), state)
if len(stepAMIRegionCopy.Regions) != 2 {
t.Fatalf("Each AMI should have been added to Regions one time only, " +
"and original region shouldn't be added at all")
}
}
func TestStepAmiRegionCopy_nil_encryption(t *testing.T) {
// create step
stepAMIRegionCopy := StepAMIRegionCopy{
@ -99,38 +234,14 @@ func TestStepAmiRegionCopy_nil_encryption(t *testing.T) {
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
state.Put("intermediary_image", false)
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete != "ami-12345" {
t.Fatalf("Should delete original intermediary ami even if not encrypted")
if stepAMIRegionCopy.toDelete != "" {
t.Fatalf("Shouldn't have an intermediary ami if encrypt is nil")
}
if len(stepAMIRegionCopy.Regions) == 0 {
t.Fatalf("Should have added original ami to original region")
}
}
func TestStepAmiRegionCopy_false_encryption(t *testing.T) {
// create step
stepAMIRegionCopy := StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: make([]string, 0),
AMIKmsKeyId: "",
RegionKeyIds: make(map[string]string),
EncryptBootVolume: aws.Bool(false),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete != "ami-12345" {
t.Fatalf("should be deleting the original intermediary ami")
}
if len(stepAMIRegionCopy.Regions) == 0 {
t.Fatalf("Should have added original ami to Regions")
if len(stepAMIRegionCopy.Regions) != 0 {
t.Fatalf("Should not have added original ami to original region")
}
}
@ -149,6 +260,7 @@ func TestStepAmiRegionCopy_true_encryption(t *testing.T) {
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
state.Put("intermediary_image", true)
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete == "" {
@ -158,3 +270,134 @@ func TestStepAmiRegionCopy_true_encryption(t *testing.T) {
t.Fatalf("Should have added original ami to Regions")
}
}
func TestStepAmiRegionCopy_nil_intermediary(t *testing.T) {
// create step
stepAMIRegionCopy := StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: make([]string, 0),
AMIKmsKeyId: "",
RegionKeyIds: make(map[string]string),
EncryptBootVolume: aws.Bool(false),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete != "" {
t.Fatalf("Should not delete original AMI if no intermediary")
}
if len(stepAMIRegionCopy.Regions) != 0 {
t.Fatalf("Should not have added original ami to Regions")
}
}
func TestStepAmiRegionCopy_AMISkipBuildRegion(t *testing.T) {
// ------------------------------------------------------------------------
// skip build region is true
// ------------------------------------------------------------------------
stepAMIRegionCopy := StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-west-1"},
AMIKmsKeyId: "",
RegionKeyIds: map[string]string{"us-west-1": "abcde"},
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
AMISkipBuildRegion: true,
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state := tState()
state.Put("intermediary_image", true)
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete == "" {
t.Fatalf("Should delete original AMI if skip_save_build_region=true")
}
if len(stepAMIRegionCopy.Regions) != 1 {
t.Fatalf("Should not have added original ami to Regions; Regions: %#v", stepAMIRegionCopy.Regions)
}
// ------------------------------------------------------------------------
// skip build region is false.
// ------------------------------------------------------------------------
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-west-1"},
AMIKmsKeyId: "",
RegionKeyIds: make(map[string]string),
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
AMISkipBuildRegion: false,
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state.Put("intermediary_image", false) // not encrypted
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete != "" {
t.Fatalf("Shouldn't have an intermediary AMI, so dont delete original ami")
}
if len(stepAMIRegionCopy.Regions) != 1 {
t.Fatalf("Should not have added original ami to Regions; Regions: %#v", stepAMIRegionCopy.Regions)
}
// ------------------------------------------------------------------------
// skip build region is false, but encrypt is true
// ------------------------------------------------------------------------
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-west-1"},
AMIKmsKeyId: "",
RegionKeyIds: map[string]string{"us-west-1": "abcde"},
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
AMISkipBuildRegion: false,
EncryptBootVolume: aws.Bool(true),
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state.Put("intermediary_image", true) //encrypted
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete == "" {
t.Fatalf("Have to delete intermediary AMI")
}
if len(stepAMIRegionCopy.Regions) != 2 {
t.Fatalf("Should have added original ami to Regions; Regions: %#v", stepAMIRegionCopy.Regions)
}
// ------------------------------------------------------------------------
// skip build region is true, and encrypt is true
// ------------------------------------------------------------------------
stepAMIRegionCopy = StepAMIRegionCopy{
AccessConfig: testAccessConfig(),
Regions: []string{"us-west-1"},
AMIKmsKeyId: "",
RegionKeyIds: map[string]string{"us-west-1": "abcde"},
Name: "fake-ami-name",
OriginalRegion: "us-east-1",
AMISkipBuildRegion: true,
EncryptBootVolume: aws.Bool(true),
}
// mock out the region connection code
stepAMIRegionCopy.getRegionConn = getMockConn
state.Put("intermediary_image", true) //encrypted
stepAMIRegionCopy.Run(context.Background(), state)
if stepAMIRegionCopy.toDelete == "" {
t.Fatalf("Have to delete intermediary AMI")
}
if len(stepAMIRegionCopy.Regions) != 1 {
t.Fatalf("Should not have added original ami to Regions; Regions: %#v", stepAMIRegionCopy.Regions)
}
}

9
builder/amazon/common/step_pre_validate.go
View file

@ -18,8 +18,9 @@ import (
// the build before actually doing any time consuming work
//
type StepPreValidate struct {
DestAmiName string
ForceDeregister bool
DestAmiName string
ForceDeregister bool
AMISkipBuildRegion bool
}
func (s *StepPreValidate) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
@ -76,6 +77,10 @@ func (s *StepPreValidate) Run(ctx context.Context, state multistep.StateBag) mul
ui.Say("Force Deregister flag found, skipping prevalidating AMI Name")
return multistep.ActionContinue
}
if s.AMISkipBuildRegion {
ui.Say("skip_build_region was set; not prevalidating AMI name")
return multistep.ActionContinue
}
ec2conn := state.Get("ec2").(*ec2.EC2)

181
builder/amazon/common/step_run_spot_instance.go
View file

@ -6,14 +6,12 @@ import (
"fmt"
"io/ioutil"
"log"
"strconv"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
"github.com/hashicorp/packer/common/random"
"github.com/hashicorp/packer/common/retry"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
@ -38,7 +36,6 @@ type StepRunSpotInstance struct {
InstanceType string
SourceAMI string
SpotPrice string
SpotPriceProduct string
SpotTags TagMap
SpotInstanceTypes []string
Tags TagMap
@ -47,57 +44,7 @@ type StepRunSpotInstance struct {
UserDataFile string
Ctx interpolate.Context
instanceId string
spotRequest *ec2.SpotInstanceRequest
}
func (s *StepRunSpotInstance) CalculateSpotPrice(az string, ec2conn ec2iface.EC2API) (string, error) {
// Calculate the spot price for a given availability zone
spotPrice := s.SpotPrice
if spotPrice == "auto" {
// Detect the spot price
startTime := time.Now().Add(-1 * time.Hour)
resp, err := ec2conn.DescribeSpotPriceHistory(&ec2.DescribeSpotPriceHistoryInput{
InstanceTypes: []*string{&s.InstanceType},
ProductDescriptions: []*string{&s.SpotPriceProduct},
AvailabilityZone: &az,
StartTime: &startTime,
})
if err != nil {
return "", fmt.Errorf("Error finding spot price: %s", err)
}
var price float64
for _, history := range resp.SpotPriceHistory {
log.Printf("[INFO] Candidate spot price: %s", *history.SpotPrice)
current, err := strconv.ParseFloat(*history.SpotPrice, 64)
if err != nil {
log.Printf("[ERR] Error parsing spot price: %s", err)
continue
}
if price == 0 || current < price {
price = current
if az == "" {
az = *history.AvailabilityZone
}
}
}
if price == 0 {
return "", fmt.Errorf("No candidate spot prices found!")
} else {
// Add 0.5 cents to minimum spot bid to ensure capacity will be available
// Avoids price-too-low error in active markets which can fluctuate
price = price + 0.005
}
spotPrice = strconv.FormatFloat(price, 'f', -1, 64)
}
s.SpotPrice = spotPrice
return spotPrice, nil
instanceId string
}
func (s *StepRunSpotInstance) CreateTemplateData(userData *string, az string,
@ -216,17 +163,6 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
az := azConfig
ui.Message(fmt.Sprintf("Finding spot price for %s %s...",
s.SpotPriceProduct, s.InstanceType))
spotPrice, err := s.CalculateSpotPrice(az, ec2conn)
if err != nil {
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Message(fmt.Sprintf("Determined spot instance price of: %s.", spotPrice))
var instanceId string
ui.Say("Interpolating tags for spot instance...")
@ -250,8 +186,10 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
// instance yet
ec2Tags.Report(ui)
spotOptions := ec2.LaunchTemplateSpotMarketOptionsRequest{
MaxPrice: &s.SpotPrice,
spotOptions := ec2.LaunchTemplateSpotMarketOptionsRequest{}
// The default is to set the maximum price to the OnDemand price.
if s.SpotPrice != "auto" {
spotOptions.SetMaxPrice(s.SpotPrice)
}
if s.BlockDurationMinutes != 0 {
spotOptions.BlockDurationMinutes = &s.BlockDurationMinutes
@ -263,6 +201,14 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
// Create a launch template for the instance
ui.Message("Loading User Data File...")
// Generate a random name to avoid conflicting with other
// instances of packer running in this AWS account
launchTemplateName := fmt.Sprintf(
"packer-fleet-launch-template-%s",
random.AlphaNum(7))
state.Put("launchTemplateName", launchTemplateName) // For the cleanup step
userData, err := s.LoadUserData()
if err != nil {
state.Put("error", err)
@ -272,7 +218,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
templateData := s.CreateTemplateData(&userData, az, state, marketOptions)
launchTemplate := &ec2.CreateLaunchTemplateInput{
LaunchTemplateData: templateData,
LaunchTemplateName: aws.String("packer-fleet-launch-template"),
LaunchTemplateName: aws.String(launchTemplateName),
VersionDescription: aws.String("template generated by packer for launching spot instances"),
}
@ -298,7 +244,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
LaunchTemplateConfigs: []*ec2.FleetLaunchTemplateConfigRequest{
{
LaunchTemplateSpecification: &ec2.FleetLaunchTemplateSpecificationRequest{
LaunchTemplateName: aws.String("packer-fleet-launch-template"),
LaunchTemplateName: aws.String(launchTemplateName),
Version: aws.String("1"),
},
Overrides: overrides,
@ -315,7 +261,31 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
// Create the request for the spot instance.
req, createOutput := ec2conn.CreateFleetRequest(createFleetInput)
ui.Message(fmt.Sprintf("Sending spot request (%s)...", req.RequestID))
// Actually send the spot connection request.
err = req.Send()
if err != nil {
if createOutput.FleetId != nil {
err = fmt.Errorf("Error waiting for fleet request (%s): %s", *createOutput.FleetId, err)
} else {
err = fmt.Errorf("Error waiting for fleet request: %s", err)
}
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(createOutput.Errors) > 0 {
errString := fmt.Sprintf("Error waiting for fleet request (%s) to become ready:", *createOutput.FleetId)
for _, outErr := range createOutput.Errors {
errString = errString + fmt.Sprintf("%s", *outErr.ErrorMessage)
}
err = fmt.Errorf(errString)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instanceId = *createOutput.Instances[0].InstanceIds[0]
// Tag the spot instance request (not the eventual spot instance)
spotTags, err := s.SpotTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
@ -324,9 +294,36 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
ui.Error(err.Error())
return multistep.ActionHalt
}
spotTags.Report(ui)
if len(spotTags) > 0 && s.SpotTags.IsSet() {
spotTags.Report(ui)
// Use the instance ID to find out the SIR, so that we can tag the spot
// request associated with this instance.
err = retry.Config{
Tries: 11,
ShouldRetry: func(error) bool { return true },
RetryDelay: (&retry.Backoff{InitialBackoff: 200 * time.Millisecond, MaxBackoff: 30 * time.Second, Multiplier: 2}).Linear,
}.Run(ctx, func(ctx context.Context) error {
_, err := ec2conn.DescribeInstances(&ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
})
return err
})
if err != nil {
err := fmt.Errorf("Error describing instance for spot request tags: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
describeOutput, err := ec2conn.DescribeInstances(&ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
})
sir := describeOutput.Reservations[0].Instances[0].SpotInstanceRequestId
// Apply tags to the spot request.
err = retry.Config{
Tries: 11,
ShouldRetry: func(error) bool { return false },
@ -334,7 +331,7 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}.Run(ctx, func(ctx context.Context) error {
_, err := ec2conn.CreateTags(&ec2.CreateTagsInput{
Tags: spotTags,
Resources: []*string{aws.String(req.RequestID)},
Resources: []*string{sir},
})
return err
})
@ -346,24 +343,6 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
}
// Actually send the spot connection request.
err = req.Send()
if err != nil {
err := fmt.Errorf("Error waiting for spot request (%s) to become ready: %s", req.RequestID, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
if len(createOutput.Errors) > 0 {
err := fmt.Errorf("error sending spot request: %s", *createOutput.Errors[0])
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instanceId = *createOutput.Instances[0].InstanceIds[0]
// Set the instance ID so that the cleanup works properly
s.instanceId = instanceId
@ -461,27 +440,9 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
}
func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
// Cancel the spot request if it exists
if s.spotRequest != nil {
ui.Say("Cancelling the spot request...")
input := &ec2.CancelSpotInstanceRequestsInput{
SpotInstanceRequestIds: []*string{s.spotRequest.SpotInstanceRequestId},
}
if _, err := ec2conn.CancelSpotInstanceRequests(input); err != nil {
ui.Error(fmt.Sprintf("Error cancelling the spot request, may still be around: %s", err))
return
}
err := WaitUntilSpotRequestFulfilled(aws.BackgroundContext(), ec2conn, *s.spotRequest.SpotInstanceRequestId)
if err != nil {
ui.Error(err.Error())
}
}
launchTemplateName := state.Get("launchTemplateName").(string)
// Terminate the source instance if it exists
if s.instanceId != "" {
@ -498,7 +459,7 @@ func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
// Delete the launch template used to create the spot fleet
deleteInput := &ec2.DeleteLaunchTemplateInput{
LaunchTemplateName: aws.String("packer-fleet-launch-template"),
LaunchTemplateName: aws.String(launchTemplateName),
}
if _, err := ec2conn.DeleteLaunchTemplate(deleteInput); err != nil {
ui.Error(err.Error())

39
builder/amazon/common/step_run_spot_instance_test.go
View file

@ -2,7 +2,6 @@ package common
import (
"bytes"
"strconv"
"testing"
"time"
@ -97,7 +96,6 @@ func getBasicStep() *StepRunSpotInstance {
InstanceType: "t2.micro",
SourceAMI: "",
SpotPrice: "auto",
SpotPriceProduct: "Linux/UNIX",
SpotTags: TagMap(nil),
Tags: TagMap{},
VolumeTags: TagMap(nil),
@ -107,20 +105,31 @@ func getBasicStep() *StepRunSpotInstance {
return &stepRunSpotInstance
}
func TestCalculateSpotPrice(t *testing.T) {
func TestCreateTemplateData(t *testing.T) {
state := tStateSpot()
stepRunSpotInstance := getBasicStep()
// Set spot price and spot price product
stepRunSpotInstance.SpotPrice = "auto"
stepRunSpotInstance.SpotPriceProduct = "Linux/UNIX"
ec2conn := getMockConnSpot()
// state := tStateSpot()
spotPrice, err := stepRunSpotInstance.CalculateSpotPrice("", ec2conn)
if err != nil {
t.Fatalf("Should not have had an error calculating spot price")
template := stepRunSpotInstance.CreateTemplateData(aws.String("userdata"), "az", state,
&ec2.LaunchTemplateInstanceMarketOptionsRequest{})
// expected := []*ec2.LaunchTemplateInstanceNetworkInterfaceSpecificationRequest{
// &ec2.LaunchTemplateInstanceNetworkInterfaceSpecificationRequest{
// DeleteOnTermination: aws.Bool(true),
// DeviceIndex: aws.Int64(0),
// Groups: aws.StringSlice([]string{"sg-0b8984db72f213dc3"}),
// SubnetId: aws.String("subnet-077fde4e"),
// },
// }
// if expected != template.NetworkInterfaces {
if template.NetworkInterfaces == nil {
t.Fatalf("Template should have contained a networkInterface object: recieved %#v", template.NetworkInterfaces)
}
sp, _ := strconv.ParseFloat(spotPrice, 64)
expected := 0.008500
if sp != expected { // 0.003500 (from spot history) + .005
t.Fatalf("Expected spot price of \"0.008500\", not %s", spotPrice)
// Rerun, this time testing that we set security group IDs
state.Put("subnet_id", "")
template = stepRunSpotInstance.CreateTemplateData(aws.String("userdata"), "az", state,
&ec2.LaunchTemplateInstanceMarketOptionsRequest{})
if template.NetworkInterfaces != nil {
t.Fatalf("Template shouldn't contain network interfaces object if subnet_id is unset.")
}
}

41
builder/amazon/ebs/builder.go
View file

@ -86,6 +86,8 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
// Accumulate any errors
var errs *packer.MultiError
var warns []string
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs,
b.config.AMIConfig.Prepare(&b.config.AccessConfig, &b.config.ctx)...)
@ -100,12 +102,20 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if b.config.RunConfig.SpotPriceAutoProduct != "" {
warns = append(warns, "spot_price_auto_product is deprecated and no "+
"longer necessary for Packer builds. In future versions of "+
"Packer, inclusion of spot_price_auto_product will error your "+
"builds. Please take a look at our current documentation to "+
"understand how Packer requests Spot instances.")
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
return warns, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
return warns, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -144,7 +154,6 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
Tags: b.config.RunTags,
SpotInstanceTypes: b.config.SpotInstanceTypes,
@ -177,8 +186,9 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
// Build the steps
steps := []multistep.Step{
&awscommon.StepPreValidate{
DestAmiName: b.config.AMIName,
ForceDeregister: b.config.AMIForceDeregister,
DestAmiName: b.config.AMIName,
ForceDeregister: b.config.AMIForceDeregister,
AMISkipBuildRegion: b.config.AMISkipBuildRegion,
},
&awscommon.StepSourceAMIInfo{
SourceAmi: b.config.SourceAmi,
@ -221,7 +231,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
@ -243,15 +253,18 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
AMIName: b.config.AMIName,
Regions: b.config.AMIRegions,
},
&stepCreateAMI{},
&stepCreateAMI{
AMISkipBuildRegion: b.config.AMISkipBuildRegion,
},
&awscommon.StepAMIRegionCopy{
AccessConfig: &b.config.AccessConfig,
Regions: b.config.AMIRegions,
AMIKmsKeyId: b.config.AMIKmsKeyId,
RegionKeyIds: b.config.AMIRegionKMSKeyIDs,
EncryptBootVolume: b.config.AMIEncryptBootVolume,
Name: b.config.AMIName,
OriginalRegion: *ec2conn.Config.Region,
AccessConfig: &b.config.AccessConfig,
Regions: b.config.AMIRegions,
AMIKmsKeyId: b.config.AMIKmsKeyId,
RegionKeyIds: b.config.AMIRegionKMSKeyIDs,
EncryptBootVolume: b.config.AMIEncryptBootVolume,
Name: b.config.AMIName,
OriginalRegion: *ec2conn.Config.Region,
AMISkipBuildRegion: b.config.AMISkipBuildRegion,
},
&awscommon.StepModifyAMIAttributes{
Description: b.config.AMIDescription,

18
builder/amazon/ebs/step_create_ami.go
View file

@ -14,7 +14,8 @@ import (
)
type stepCreateAMI struct {
image *ec2.Image
image *ec2.Image
AMISkipBuildRegion bool
}
func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
@ -25,9 +26,18 @@ func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multi
// Create the image
amiName := config.AMIName
if config.AMIEncryptBootVolume != nil {
// encrypt_boot was set, so we will create a temporary image
// and then create a copy of it with the correct encrypt_boot
state.Put("intermediary_image", false)
if config.AMIEncryptBootVolume != nil && *config.AMIEncryptBootVolume != false || s.AMISkipBuildRegion {
state.Put("intermediary_image", true)
// From AWS SDK docs: You can encrypt a copy of an unencrypted snapshot,
// but you cannot use it to create an unencrypted copy of an encrypted
// snapshot. Your default CMK for EBS is used unless you specify a
// non-default key using KmsKeyId.
// If encrypt_boot is nil or true, we need to create a temporary image
// so that in step_region_copy, we can copy it with the correct
// encryption
amiName = random.AlphaNum(7)
}

17
builder/amazon/ebssurrogate/builder.go
View file

@ -95,6 +95,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
// Accumulate any errors
var errs *packer.MultiError
var warns []string
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs,
@ -128,6 +129,14 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if b.config.RunConfig.SpotPriceAutoProduct != "" {
warns = append(warns, "spot_price_auto_product is deprecated and no "+
"longer necessary for Packer builds. In future versions of "+
"Packer, inclusion of spot_price_auto_product will error your "+
"builds. Please take a look at our current documentation to "+
"understand how Packer requests Spot instances.")
}
if b.config.Architecture == "" {
b.config.Architecture = "x86_64"
}
@ -142,11 +151,12 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, errors.New(`The only valid ami_architecture values are "x86_64" and "arm64"`))
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
return warns, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
return warns, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -183,7 +193,6 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotInstanceTypes: b.config.SpotInstanceTypes,
SpotTags: b.config.SpotTags,
Tags: b.config.RunTags,
@ -263,7 +272,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

16
builder/amazon/ebsvolume/builder.go
View file

@ -81,6 +81,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
// Accumulate any errors
var errs *packer.MultiError
var warns []string
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.launchBlockDevices.Prepare(&b.config.ctx)...)
@ -103,12 +104,20 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if b.config.RunConfig.SpotPriceAutoProduct != "" {
warns = append(warns, "spot_price_auto_product is deprecated and no "+
"longer necessary for Packer builds. In future versions of "+
"Packer, inclusion of spot_price_auto_product will error your "+
"builds. Please take a look at our current documentation to "+
"understand how Packer requests Spot instances.")
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
return warns, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
return warns, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -143,7 +152,6 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotInstanceTypes: b.config.SpotInstanceTypes,
SpotTags: b.config.SpotTags,
Tags: b.config.RunTags,
@ -214,7 +222,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

3
builder/amazon/ebsvolume/step_tag_ebs_volumes.go
View file

@ -24,6 +24,9 @@ func (s *stepTagEBSVolumes) Run(ctx context.Context, state multistep.StateBag) m
for _, instanceBlockDevices := range instance.BlockDeviceMappings {
for _, configVolumeMapping := range s.VolumeMapping {
if configVolumeMapping.DeviceName == *instanceBlockDevices.DeviceName {
if configVolumeMapping.DeleteOnTermination {
continue
}
volumes[*ec2conn.Config.Region] = append(
volumes[*ec2conn.Config.Region],
*instanceBlockDevices.Ebs.VolumeId)

16
builder/amazon/instance/builder.go
View file

@ -169,6 +169,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
// Accumulate any errors
var errs *packer.MultiError
var warns []string
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.AMIMappings.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.LaunchMappings.Prepare(&b.config.ctx)...)
@ -207,11 +208,19 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if b.config.RunConfig.SpotPriceAutoProduct != "" {
warns = append(warns, "spot_price_auto_product is deprecated and no "+
"longer necessary for Packer builds. In future versions of "+
"Packer, inclusion of spot_price_auto_product will error your "+
"builds. Please take a look at our current documentation to "+
"understand how Packer requests Spot instances.")
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
return warns, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
return warns, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -247,7 +256,6 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotInstanceTypes: b.config.SpotInstanceTypes,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
Tags: b.config.RunTags,
SpotTags: b.config.SpotTags,
UserData: b.config.UserData,
@ -316,7 +324,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.Comm.SSHInterface),
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

18
builder/azure/arm/artifact.go
View file

@ -35,6 +35,8 @@ type Artifact struct {
ManagedImageId string
ManagedImageOSDiskSnapshotName string
ManagedImageDataDiskSnapshotPrefix string
// ARM resource id for Shared Image Gallery
ManagedImageSharedImageGalleryId string
// Additional Disks
AdditionalDisks *[]AdditionalDiskArtifact
@ -52,6 +54,19 @@ func NewManagedImageArtifact(osType, resourceGroup, name, location, id, osDiskSn
}, nil
}
func NewManagedImageArtifactWithSIGAsDestination(osType, resourceGroup, name, location, id, osDiskSnapshotName, dataDiskSnapshotPrefix, destinationSharedImageGalleryId string) (*Artifact, error) {
return &Artifact{
ManagedImageResourceGroupName: resourceGroup,
ManagedImageName: name,
ManagedImageLocation: location,
ManagedImageId: id,
OSType: osType,
ManagedImageOSDiskSnapshotName: osDiskSnapshotName,
ManagedImageDataDiskSnapshotPrefix: dataDiskSnapshotPrefix,
ManagedImageSharedImageGalleryId: destinationSharedImageGalleryId,
}, nil
}
func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string, osType string) (*Artifact, error) {
if template == nil {
return nil, fmt.Errorf("nil capture template")
@ -168,6 +183,9 @@ func (a *Artifact) String() string {
if a.ManagedImageDataDiskSnapshotPrefix != "" {
buf.WriteString(fmt.Sprintf("ManagedImageDataDiskSnapshotPrefix: %s\n", a.ManagedImageDataDiskSnapshotPrefix))
}
if a.ManagedImageSharedImageGalleryId != "" {
buf.WriteString(fmt.Sprintf("ManagedImageSharedImageGalleryId: %s\n", a.ManagedImageSharedImageGalleryId))
}
} else {
buf.WriteString(fmt.Sprintf("StorageAccountLocation: %s\n", a.StorageAccountLocation))
buf.WriteString(fmt.Sprintf("OSDiskUri: %s\n", a.OSDiskUri))

24
builder/azure/arm/artifact_test.go
View file

@ -108,6 +108,30 @@ ManagedImageOSDiskSnapshotName: fakeOsDiskSnapshotName
}
}
func TestArtifactIDManagedImageWithSharedImageGalleryId(t *testing.T) {
artifact, err := NewManagedImageArtifactWithSIGAsDestination("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "fakeDataDiskSnapshotPrefix", "fakeSharedImageGallery")
if err != nil {
t.Fatalf("err=%s", err)
}
expected := `Azure.ResourceManagement.VMImage:
OSType: Linux
ManagedImageResourceGroupName: fakeResourceGroup
ManagedImageName: fakeName
ManagedImageId: fakeID
ManagedImageLocation: fakeLocation
ManagedImageOSDiskSnapshotName: fakeOsDiskSnapshotName
ManagedImageDataDiskSnapshotPrefix: fakeDataDiskSnapshotPrefix
ManagedImageSharedImageGalleryId: fakeSharedImageGallery
`
result := artifact.String()
if result != expected {
t.Fatalf("bad: %s", result)
}
}
func TestArtifactString(t *testing.T) {
template := CaptureTemplate{
Resources: []CaptureResources{

19
builder/azure/arm/azure_client.go
View file

@ -9,8 +9,10 @@ import (
"net/url"
"os"
"strconv"
"time"
"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2018-04-01/compute"
newCompute "github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2019-03-01/compute"
"github.com/Azure/azure-sdk-for-go/services/network/mgmt/2018-01-01/network"
"github.com/Azure/azure-sdk-for-go/services/resources/mgmt/2018-02-01/resources"
armStorage "github.com/Azure/azure-sdk-for-go/services/storage/mgmt/2017-10-01/storage"
@ -41,6 +43,8 @@ type AzureClient struct {
armStorage.AccountsClient
compute.DisksClient
compute.SnapshotsClient
newCompute.GalleryImageVersionsClient
newCompute.GalleryImagesClient
InspectorMaxLength int
Template *CaptureTemplate
@ -123,7 +127,7 @@ func byConcatDecorators(decorators ...autorest.RespondDecorator) autorest.Respon
}
func NewAzureClient(subscriptionID, resourceGroupName, storageAccountName string,
cloud *azure.Environment,
cloud *azure.Environment, SharedGalleryTimeout time.Duration,
servicePrincipalToken, servicePrincipalTokenVault *adal.ServicePrincipalToken) (*AzureClient, error) {
var azureClient = &AzureClient{}
@ -202,6 +206,19 @@ func NewAzureClient(subscriptionID, resourceGroupName, storageAccountName string
azureClient.AccountsClient.ResponseInspector = byConcatDecorators(byInspecting(maxlen), errorCapture(azureClient))
azureClient.AccountsClient.UserAgent = fmt.Sprintf("%s %s", useragent.String(), azureClient.AccountsClient.UserAgent)
azureClient.GalleryImageVersionsClient = newCompute.NewGalleryImageVersionsClientWithBaseURI(cloud.ResourceManagerEndpoint, subscriptionID)
azureClient.GalleryImageVersionsClient.Authorizer = autorest.NewBearerAuthorizer(servicePrincipalToken)
azureClient.GalleryImageVersionsClient.RequestInspector = withInspection(maxlen)
azureClient.GalleryImageVersionsClient.ResponseInspector = byConcatDecorators(byInspecting(maxlen), errorCapture(azureClient))
azureClient.GalleryImageVersionsClient.UserAgent = fmt.Sprintf("%s %s", useragent.String(), azureClient.GalleryImageVersionsClient.UserAgent)
azureClient.GalleryImageVersionsClient.Client.PollingDuration = SharedGalleryTimeout
azureClient.GalleryImagesClient = newCompute.NewGalleryImagesClientWithBaseURI(cloud.ResourceManagerEndpoint, subscriptionID)
azureClient.GalleryImagesClient.Authorizer = autorest.NewBearerAuthorizer(servicePrincipalToken)
azureClient.GalleryImagesClient.RequestInspector = withInspection(maxlen)
azureClient.GalleryImagesClient.ResponseInspector = byConcatDecorators(byInspecting(maxlen), errorCapture(azureClient))
azureClient.GalleryImagesClient.UserAgent = fmt.Sprintf("%s %s", useragent.String(), azureClient.GalleryImagesClient.UserAgent)
keyVaultURL, err := url.Parse(cloud.KeyVaultEndpoint)
if err != nil {
return nil, err

42
builder/azure/arm/builder.go
View file

@ -90,6 +90,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
b.config.ResourceGroupName,
b.config.StorageAccount,
b.config.cloudEnvironment,
b.config.SharedGalleryTimeout,
spnCloud,
spnKeyVault)
@ -177,6 +178,31 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
deploymentName := b.stateBag.Get(constants.ArmDeploymentName).(string)
// For Managed Images, validate that Shared Gallery Image exists before publishing to SIG
if b.config.isManagedImage() && b.config.SharedGalleryDestination.SigDestinationGalleryName != "" {
_, err = azureClient.GalleryImagesClient.Get(ctx, b.config.SharedGalleryDestination.SigDestinationResourceGroup, b.config.SharedGalleryDestination.SigDestinationGalleryName, b.config.SharedGalleryDestination.SigDestinationImageName)
if err != nil {
return nil, fmt.Errorf("the Shared Gallery Image to which to publish the managed image version to does not exist in the resource group %s", b.config.SharedGalleryDestination.SigDestinationResourceGroup)
}
// SIG requires that replication regions include the region in which the Managed Image resides
managedImageLocation := normalizeAzureRegion(b.stateBag.Get(constants.ArmLocation).(string))
foundMandatoryReplicationRegion := false
var normalizedReplicationRegions []string
for _, region := range b.config.SharedGalleryDestination.SigDestinationReplicationRegions {
// change region to lower-case and strip spaces
normalizedRegion := normalizeAzureRegion(region)
normalizedReplicationRegions = append(normalizedReplicationRegions, normalizedRegion)
if strings.EqualFold(normalizedRegion, managedImageLocation) {
foundMandatoryReplicationRegion = true
continue
}
}
if foundMandatoryReplicationRegion == false {
b.config.SharedGalleryDestination.SigDestinationReplicationRegions = append(normalizedReplicationRegions, managedImageLocation)
}
b.stateBag.Put(constants.ArmManagedImageSharedGalleryReplicationRegions, b.config.SharedGalleryDestination.SigDestinationReplicationRegions)
}
if b.config.OSType == constants.Target_Linux {
steps = []multistep.Step{
NewStepCreateResourceGroup(azureClient, ui),
@ -198,6 +224,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
NewStepSnapshotOSDisk(azureClient, ui, b.config),
NewStepSnapshotDataDisks(azureClient, ui, b.config),
NewStepCaptureImage(azureClient, ui),
NewStepPublishToSharedImageGallery(azureClient, ui, b.config),
NewStepDeleteResourceGroup(azureClient, ui),
NewStepDeleteOSDisk(azureClient, ui),
NewStepDeleteAdditionalDisks(azureClient, ui),
@ -236,6 +263,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
NewStepSnapshotOSDisk(azureClient, ui, b.config),
NewStepSnapshotDataDisks(azureClient, ui, b.config),
NewStepCaptureImage(azureClient, ui),
NewStepPublishToSharedImageGallery(azureClient, ui, b.config),
NewStepDeleteResourceGroup(azureClient, ui),
NewStepDeleteOSDisk(azureClient, ui),
NewStepDeleteAdditionalDisks(azureClient, ui),
@ -275,6 +303,9 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
if b.config.isManagedImage() {
managedImageID := fmt.Sprintf("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Compute/images/%s", b.config.SubscriptionID, b.config.ManagedImageResourceGroupName, b.config.ManagedImageName)
if b.config.SharedGalleryDestination.SigDestinationGalleryName != "" {
return NewManagedImageArtifactWithSIGAsDestination(b.config.OSType, b.config.ManagedImageResourceGroupName, b.config.ManagedImageName, b.config.manageImageLocation, managedImageID, b.config.ManagedImageOSDiskSnapshotName, b.config.ManagedImageDataDiskSnapshotPrefix, b.stateBag.Get(constants.ArmManagedImageSharedGalleryId).(string))
}
return NewManagedImageArtifact(b.config.OSType, b.config.ManagedImageResourceGroupName, b.config.ManagedImageName, b.config.manageImageLocation, managedImageID, b.config.ManagedImageOSDiskSnapshotName, b.config.ManagedImageDataDiskSnapshotPrefix)
} else if template, ok := b.stateBag.GetOk(constants.ArmCaptureTemplate); ok {
return NewArtifact(
@ -369,6 +400,13 @@ func (b *Builder) configureStateBag(stateBag multistep.StateBag) {
stateBag.Put(constants.ArmManagedImageOSDiskSnapshotName, b.config.ManagedImageOSDiskSnapshotName)
stateBag.Put(constants.ArmManagedImageDataDiskSnapshotPrefix, b.config.ManagedImageDataDiskSnapshotPrefix)
stateBag.Put(constants.ArmAsyncResourceGroupDelete, b.config.AsyncResourceGroupDelete)
if b.config.isManagedImage() && b.config.SharedGalleryDestination.SigDestinationGalleryName != "" {
stateBag.Put(constants.ArmManagedImageSigPublishResourceGroup, b.config.SharedGalleryDestination.SigDestinationResourceGroup)
stateBag.Put(constants.ArmManagedImageSharedGalleryName, b.config.SharedGalleryDestination.SigDestinationGalleryName)
stateBag.Put(constants.ArmManagedImageSharedGalleryImageName, b.config.SharedGalleryDestination.SigDestinationImageName)
stateBag.Put(constants.ArmManagedImageSharedGalleryImageVersion, b.config.SharedGalleryDestination.SigDestinationImageVersion)
stateBag.Put(constants.ArmManagedImageSubscription, b.config.SubscriptionID)
}
}
// Parameters that are only known at runtime after querying Azure.
@ -404,3 +442,7 @@ func getObjectIdFromToken(ui packer.Ui, token *adal.ServicePrincipalToken) strin
return claims["oid"].(string)
}
func normalizeAzureRegion(name string) string {
return strings.ToLower(strings.Replace(name, " ", "", -1))
}

2
builder/azure/arm/builder_acc_test.go
View file

@ -11,7 +11,7 @@ package arm
//
// The subscription in question should have a resource group
// called "packer-acceptance-test" in "South Central US" region. The
// storage account refered to in the above variable should
// storage account referred to in the above variable should
// be inside this resource group and in "South Central US" as well.
//
// In addition, the PACKER_ACC variable should also be set to

57
builder/azure/arm/config.go
View file

@ -80,6 +80,14 @@ type SharedImageGallery struct {
ImageVersion string `mapstructure:"image_version" required:"false"`
}
type SharedImageGalleryDestination struct {
SigDestinationResourceGroup string `mapstructure:"resource_group"`
SigDestinationGalleryName string `mapstructure:"gallery_name"`
SigDestinationImageName string `mapstructure:"image_name"`
SigDestinationImageVersion string `mapstructure:"image_version"`
SigDestinationReplicationRegions []string `mapstructure:"replication_regions"`
}
type Config struct {
common.PackerConfig `mapstructure:",squash"`
@ -104,6 +112,30 @@ type Config struct {
// "managed_image_name": "TargetImageName",
// "managed_image_resource_group_name": "TargetResourceGroup"
SharedGallery SharedImageGallery `mapstructure:"shared_image_gallery" required:"false"`
// The name of the Shared Image Gallery under which the managed image will be published as Shared Gallery Image version.
//
// Following is an example.
//
// <!-- -->
//
// "shared_image_gallery_destination": {
// "resource_group": "ResourceGroup",
// "gallery_name": "GalleryName",
// "image_name": "ImageName",
// "image_version": "1.0.0",
// "replication_regions": ["regionA", "regionB", "regionC"]
// }
// "managed_image_name": "TargetImageName",
// "managed_image_resource_group_name": "TargetResourceGroup"
SharedGalleryDestination SharedImageGalleryDestination `mapstructure:"shared_image_gallery_destination"`
// How long to wait for an image to be published to the shared image
// gallery before timing out. If your Packer build is failing on the
// Publishing to Shared Image Gallery step with the error `Original Error:
// context deadline exceeded`, but the image is present when you check your
// Azure dashboard, then you probably need to increase this timeout from
// its default of "60m" (valid time units include `s` for seconds, `m` for
// minutes, and `h` for hours.)
SharedGalleryTimeout time.Duration `mapstructure:"shared_image_gallery_timeout"`
// PublisherName for your base image. See
// [documentation](https://azure.microsoft.com/en-us/documentation/articles/resource-groups-vm-searching/)
// for details.
@ -721,16 +753,16 @@ func assertRequiredParametersSet(c *Config, errs *packer.MultiError) {
}
} else if c.ImageUrl == "" && c.ImagePublisher == "" {
if c.CustomManagedImageResourceGroupName == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("An custom_managed_image_resource_group_name must be specified"))
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A custom_managed_image_resource_group_name must be specified"))
}
if c.CustomManagedImageName == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A custom_managed_image_name must be specified"))
}
if c.ManagedImageResourceGroupName == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("An managed_image_resource_group_name must be specified"))
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A managed_image_resource_group_name must be specified"))
}
if c.ManagedImageName == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("An managed_image_name must be specified"))
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A managed_image_name must be specified"))
}
} else {
if c.ImagePublisher != "" || c.ImageOffer != "" || c.ImageSku != "" || c.ImageVersion != "" {
@ -785,6 +817,25 @@ func assertRequiredParametersSet(c *Config, errs *packer.MultiError) {
}
}
if c.ManagedImageName != "" && c.ManagedImageResourceGroupName != "" && c.SharedGalleryDestination.SigDestinationGalleryName != "" {
if c.SharedGalleryDestination.SigDestinationResourceGroup == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A resource_group must be specified for shared_image_gallery_destination"))
}
if c.SharedGalleryDestination.SigDestinationImageName == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("An image_name must be specified for shared_image_gallery_destination"))
}
if c.SharedGalleryDestination.SigDestinationImageVersion == "" {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("An image_version must be specified for shared_image_gallery_destination"))
}
if len(c.SharedGalleryDestination.SigDestinationReplicationRegions) == 0 {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("A list of replication_regions must be specified for shared_image_gallery_destination"))
}
}
if c.SharedGalleryTimeout == 0 {
// default to a one-hour timeout. In the sdk, the default is 15 m.
c.SharedGalleryTimeout = 60 * time.Minute
}
if c.ManagedImageOSDiskSnapshotName != "" {
if ok, err := assertManagedImageOSDiskSnapshotName(c.ManagedImageOSDiskSnapshotName, "managed_image_os_disk_snapshot_name"); !ok {
errs = packer.MultiErrorAppend(errs, err)

2
builder/azure/arm/step_delete_resource_group.go
View file

@ -56,7 +56,7 @@ func (s *StepDeleteResourceGroup) deleteResourceGroup(ctx context.Context, state
f, err := s.client.GroupsClient.Delete(ctx, resourceGroupName)
if err == nil {
if state.Get(constants.ArmAsyncResourceGroupDelete).(bool) {
// No need to wait for the complition for delete if request is Accepted
// No need to wait for the completion for delete if request is Accepted
s.say(fmt.Sprintf("\nResource Group is being deleted, not waiting for deletion due to config. Resource Group Name '%s'", resourceGroupName))
} else {
f.WaitForCompletionRef(ctx, s.client.GroupsClient.Client)

125
builder/azure/arm/step_publish_to_shared_image_gallery.go Normal file
View file

@ -0,0 +1,125 @@
package arm
import (
"context"
"fmt"
"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2019-03-01/compute"
"github.com/hashicorp/packer/builder/azure/common/constants"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type StepPublishToSharedImageGallery struct {
client *AzureClient
publish func(ctx context.Context, mdiID, miSigPubRg, miSIGalleryName, miSGImageName, miSGImageVersion string, miSigReplicationRegions []string, location string, tags map[string]*string) (string, error)
say func(message string)
error func(e error)
toSIG func() bool
}
func NewStepPublishToSharedImageGallery(client *AzureClient, ui packer.Ui, config *Config) *StepPublishToSharedImageGallery {
var step = &StepPublishToSharedImageGallery{
client: client,
say: func(message string) {
ui.Say(message)
},
error: func(e error) {
ui.Error(e.Error())
},
toSIG: func() bool {
return config.isManagedImage() && config.SharedGalleryDestination.SigDestinationGalleryName != ""
},
}
step.publish = step.publishToSig
return step
}
func (s *StepPublishToSharedImageGallery) publishToSig(ctx context.Context, mdiID string, miSigPubRg string, miSIGalleryName string, miSGImageName string, miSGImageVersion string, miSigReplicationRegions []string, location string, tags map[string]*string) (string, error) {
replicationRegions := make([]compute.TargetRegion, len(miSigReplicationRegions))
for i, v := range miSigReplicationRegions {
regionName := v
replicationRegions[i] = compute.TargetRegion{Name: &regionName}
}
galleryImageVersion := compute.GalleryImageVersion{
Location: &location,
Tags: tags,
GalleryImageVersionProperties: &compute.GalleryImageVersionProperties{
PublishingProfile: &compute.GalleryImageVersionPublishingProfile{
Source: &compute.GalleryArtifactSource{
ManagedImage: &compute.ManagedArtifact{
ID: &mdiID,
},
},
TargetRegions: &replicationRegions,
},
},
}
f, err := s.client.GalleryImageVersionsClient.CreateOrUpdate(ctx, miSigPubRg, miSIGalleryName, miSGImageName, miSGImageVersion, galleryImageVersion)
if err != nil {
s.say(s.client.LastError.Error())
return "", err
}
err = f.WaitForCompletionRef(ctx, s.client.GalleryImageVersionsClient.Client)
if err != nil {
s.say(s.client.LastError.Error())
return "", err
}
createdSGImageVersion, err := f.Result(s.client.GalleryImageVersionsClient)
if err != nil {
s.say(s.client.LastError.Error())
return "", err
}
s.say(fmt.Sprintf(" -> Shared Gallery Image Version ID : '%s'", *(createdSGImageVersion.ID)))
return *(createdSGImageVersion.ID), nil
}
func (s *StepPublishToSharedImageGallery) Run(ctx context.Context, stateBag multistep.StateBag) multistep.StepAction {
if !s.toSIG() {
return multistep.ActionContinue
}
s.say("Publishing to Shared Image Gallery ...")
var miSigPubRg = stateBag.Get(constants.ArmManagedImageSigPublishResourceGroup).(string)
var miSIGalleryName = stateBag.Get(constants.ArmManagedImageSharedGalleryName).(string)
var miSGImageName = stateBag.Get(constants.ArmManagedImageSharedGalleryImageName).(string)
var miSGImageVersion = stateBag.Get(constants.ArmManagedImageSharedGalleryImageVersion).(string)
var location = stateBag.Get(constants.ArmLocation).(string)
var tags = stateBag.Get(constants.ArmTags).(map[string]*string)
var miSigReplicationRegions = stateBag.Get(constants.ArmManagedImageSharedGalleryReplicationRegions).([]string)
var targetManagedImageResourceGroupName = stateBag.Get(constants.ArmManagedImageResourceGroupName).(string)
var targetManagedImageName = stateBag.Get(constants.ArmManagedImageName).(string)
var managedImageSubscription = stateBag.Get(constants.ArmManagedImageSubscription).(string)
var mdiID = fmt.Sprintf("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Compute/images/%s", managedImageSubscription, targetManagedImageResourceGroupName, targetManagedImageName)
s.say(fmt.Sprintf(" -> MDI ID used for SIG publish : '%s'", mdiID))
s.say(fmt.Sprintf(" -> SIG publish resource group : '%s'", miSigPubRg))
s.say(fmt.Sprintf(" -> SIG gallery name : '%s'", miSIGalleryName))
s.say(fmt.Sprintf(" -> SIG image name : '%s'", miSGImageName))
s.say(fmt.Sprintf(" -> SIG image version : '%s'", miSGImageVersion))
s.say(fmt.Sprintf(" -> SIG replication regions : '%v'", miSigReplicationRegions))
createdGalleryImageVersionID, err := s.publish(ctx, mdiID, miSigPubRg, miSIGalleryName, miSGImageName, miSGImageVersion, miSigReplicationRegions, location, tags)
if err != nil {
stateBag.Put(constants.Error, err)
s.error(err)
return multistep.ActionHalt
}
stateBag.Put(constants.ArmManagedImageSharedGalleryId, createdGalleryImageVersionID)
return multistep.ActionContinue
}
func (*StepPublishToSharedImageGallery) Cleanup(multistep.StateBag) {
}

84
builder/azure/arm/step_publish_to_shared_image_gallery_test.go Normal file
View file

@ -0,0 +1,84 @@
package arm
import (
"context"
"github.com/hashicorp/packer/builder/azure/common/constants"
"github.com/hashicorp/packer/helper/multistep"
"testing"
)
func TestStepPublishToSharedImageGalleryShouldNotPublishForVhd(t *testing.T) {
var testSubject = &StepPublishToSharedImageGallery{
publish: func(context.Context, string, string, string, string, string, []string, string, map[string]*string) (string, error) {
return "test", nil
},
say: func(message string) {},
error: func(e error) {},
toSIG: func() bool { return false },
}
stateBag := createTestStateBagStepPublishToSharedImageGalleryForVhd()
var result = testSubject.Run(context.Background(), stateBag)
if result != multistep.ActionContinue {
t.Fatalf("Expected the step to return 'ActionContinue', but got '%d'.", result)
}
if _, ok := stateBag.GetOk(constants.Error); ok == true {
t.Fatalf("Expected the step to not set stateBag['%s'], but it was.", constants.Error)
}
}
func TestStepPublishToSharedImageGalleryShouldPublishForManagedImageWithSig(t *testing.T) {
var testSubject = &StepPublishToSharedImageGallery{
publish: func(context.Context, string, string, string, string, string, []string, string, map[string]*string) (string, error) {
return "", nil
},
say: func(message string) {},
error: func(e error) {},
toSIG: func() bool { return true },
}
stateBag := createTestStateBagStepPublishToSharedImageGallery()
var result = testSubject.Run(context.Background(), stateBag)
if result != multistep.ActionContinue {
t.Fatalf("Expected the step to return 'ActionContinue', but got '%d'.", result)
}
if _, ok := stateBag.GetOk(constants.Error); ok == true {
t.Fatalf("Expected the step to not set stateBag['%s'], but it was.", constants.Error)
}
}
func createTestStateBagStepPublishToSharedImageGallery() multistep.StateBag {
stateBag := new(multistep.BasicStateBag)
stateBag.Put(constants.ArmManagedImageSigPublishResourceGroup, "Unit Test: ManagedImageSigPublishResourceGroup")
stateBag.Put(constants.ArmManagedImageSharedGalleryName, "Unit Test: ManagedImageSharedGalleryName")
stateBag.Put(constants.ArmManagedImageSharedGalleryImageName, "Unit Test: ManagedImageSharedGalleryImageName")
stateBag.Put(constants.ArmManagedImageSharedGalleryImageVersion, "Unit Test: ManagedImageSharedGalleryImageVersion")
stateBag.Put(constants.ArmLocation, "Unit Test: Location")
value := "Unit Test: Tags"
tags := map[string]*string{
"tag01": &value,
}
stateBag.Put(constants.ArmTags, tags)
stateBag.Put(constants.ArmManagedImageSharedGalleryReplicationRegions, []string{"ManagedImageSharedGalleryReplicationRegionA", "ManagedImageSharedGalleryReplicationRegionB"})
stateBag.Put(constants.ArmManagedImageResourceGroupName, "Unit Test: ManagedImageResourceGroupName")
stateBag.Put(constants.ArmManagedImageName, "Unit Test: ManagedImageName")
stateBag.Put(constants.ArmManagedImageSubscription, "Unit Test: ManagedImageSubscription")
return stateBag
}
func createTestStateBagStepPublishToSharedImageGalleryForVhd() multistep.StateBag {
stateBag := new(multistep.BasicStateBag)
stateBag.Put(constants.ArmLocation, "Unit Test: Location")
value := "Unit Test: Tags"
tags := map[string]*string{
"tag01": &value,
}
stateBag.Put(constants.ArmTags, tags)
return stateBag
}

21
builder/azure/common/constants/stateBag.go
View file

@ -30,11 +30,18 @@ const (
ArmVirtualMachineCaptureParameters string = "arm.VirtualMachineCaptureParameters"
ArmIsExistingResourceGroup string = "arm.IsExistingResourceGroup"
ArmIsManagedImage string = "arm.IsManagedImage"
ArmManagedImageResourceGroupName string = "arm.ManagedImageResourceGroupName"
ArmManagedImageLocation string = "arm.ManagedImageLocation"
ArmManagedImageName string = "arm.ManagedImageName"
ArmAsyncResourceGroupDelete string = "arm.AsyncResourceGroupDelete"
ArmManagedImageOSDiskSnapshotName string = "arm.ManagedImageOSDiskSnapshotName"
ArmManagedImageDataDiskSnapshotPrefix string = "arm.ManagedImageDataDiskSnapshotPrefix"
ArmIsManagedImage string = "arm.IsManagedImage"
ArmManagedImageResourceGroupName string = "arm.ManagedImageResourceGroupName"
ArmManagedImageLocation string = "arm.ManagedImageLocation"
ArmManagedImageName string = "arm.ManagedImageName"
ArmManagedImageSigPublishResourceGroup string = "arm.ManagedImageSigPublishResourceGroup"
ArmManagedImageSharedGalleryName string = "arm.ManagedImageSharedGalleryName"
ArmManagedImageSharedGalleryImageName string = "arm.ManagedImageSharedGalleryImageName"
ArmManagedImageSharedGalleryImageVersion string = "arm.ManagedImageSharedGalleryImageVersion"
ArmManagedImageSharedGalleryReplicationRegions string = "arm.ManagedImageSharedGalleryReplicationRegions"
ArmManagedImageSharedGalleryId string = "arm.ArmManagedImageSharedGalleryId"
ArmManagedImageSubscription string = "arm.ArmManagedImageSubscription"
ArmAsyncResourceGroupDelete string = "arm.AsyncResourceGroupDelete"
ArmManagedImageOSDiskSnapshotName string = "arm.ManagedImageOSDiskSnapshotName"
ArmManagedImageDataDiskSnapshotPrefix string = "arm.ManagedImageDataDiskSnapshotPrefix"
)

2
builder/cloudstack/builder.go
View file

@ -77,7 +77,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
&stepSetupNetworking{},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "ipaddress"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
SSHPort: commPort,
WinRMPort: commPort,

9
builder/cloudstack/ssh.go
View file

@ -6,15 +6,6 @@ import (
"github.com/hashicorp/packer/helper/multistep"
)
func commHost(state multistep.StateBag) (string, error) {
ip, hasIP := state.Get("ipaddress").(string)
if !hasIP {
return "", fmt.Errorf("Failed to retrieve IP address")
}
return ip, nil
}
func commPort(state multistep.StateBag) (int, error) {
commPort, hasPort := state.Get("commPort").(int)
if !hasPort {

6
builder/digitalocean/builder.go
View file

@ -86,7 +86,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
new(stepDropletInfo),
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "droplet_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
new(common.StepProvision),
@ -95,7 +95,9 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
new(stepShutdown),
new(stepPowerOff),
new(stepSnapshot),
&stepSnapshot{
snapshotTimeout: b.config.SnapshotTimeout,
},
}
// Run the steps

39
builder/digitalocean/builder_test.go
View file

@ -190,7 +190,46 @@ func TestBuilderPrepare_StateTimeout(t *testing.T) {
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_SnapshotTimeout(t *testing.T) {
var b Builder
config := testConfig()
// Test default
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
if b.config.SnapshotTimeout != 60*time.Minute {
t.Errorf("invalid: %s", b.config.SnapshotTimeout)
}
// Test set
config["snapshot_timeout"] = "15m"
b = Builder{}
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test bad
config["snapshot_timeout"] = "badstring"
b = Builder{}
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_PrivateNetworking(t *testing.T) {

13
builder/digitalocean/config.go
View file

@ -64,6 +64,14 @@ type Config struct {
// droplet to enter a desired state (such as "active") before timing out. The
// default state timeout is "6m".
StateTimeout time.Duration `mapstructure:"state_timeout" required:"false"`
// How long to wait for an image to be published to the shared image
// gallery before timing out. If your Packer build is failing on the
// Publishing to Shared Image Gallery step with the error `Original Error:
// context deadline exceeded`, but the image is present when you check your
// Azure dashboard, then you probably need to increase this timeout from
// its default of "60m" (valid time units include `s` for seconds, `m` for
// minutes, and `h` for hours.)
SnapshotTimeout time.Duration `mapstructure:"snapshot_timeout" required:"false"`
// The name assigned to the droplet. DigitalOcean
// sets the hostname of the machine to this value.
DropletName string `mapstructure:"droplet_name" required:"false"`
@ -127,6 +135,11 @@ func NewConfig(raws ...interface{}) (*Config, []string, error) {
c.StateTimeout = 6 * time.Minute
}
if c.SnapshotTimeout == 0 {
// Default to 60 minutes timeout, waiting for snapshot action to finish
c.SnapshotTimeout = 60 * time.Minute
}
var errs *packer.MultiError
if es := c.Comm.Prepare(&c.ctx); len(es) > 0 {
errs = packer.MultiErrorAppend(errs, es...)

10
builder/digitalocean/ssh.go
View file

@ -1,10 +0,0 @@
package digitalocean
import (
"github.com/hashicorp/packer/helper/multistep"
)
func commHost(state multistep.StateBag) (string, error) {
ipAddress := state.Get("droplet_ip").(string)
return ipAddress, nil
}

8
builder/digitalocean/step_snapshot.go
View file

@ -12,7 +12,9 @@ import (
"github.com/hashicorp/packer/packer"
)
type stepSnapshot struct{}
type stepSnapshot struct {
snapshotTimeout time.Duration
}
func (s *stepSnapshot) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*godo.Client)
@ -31,9 +33,11 @@ func (s *stepSnapshot) Run(ctx context.Context, state multistep.StateBag) multis
}
// With the pending state over, verify that we're in the active state
// because action can take a long time and may depend on the size of the final snapshot,
// the timeout is parameterized
ui.Say("Waiting for snapshot to complete...")
if err := waitForActionState(godo.ActionCompleted, dropletId, action.ID,
client, 20*time.Minute); err != nil {
client, s.snapshotTimeout); err != nil {
// If we get an error the first time, actually report it
err := fmt.Errorf("Error waiting for snapshot: %s", err)
state.Put("error", err)

2
builder/docker/builder.go
View file

@ -48,7 +48,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
&StepRun{},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: commHost(b.config.Comm.SSHHost),
SSHConfig: b.config.Comm.SSHConfigFunc(),
CustomConnect: map[string]multistep.Step{
"docker": &StepConnectDocker{},

16
builder/docker/comm.go
View file

@ -1,11 +1,19 @@
package docker
import (
"log"
"github.com/hashicorp/packer/helper/multistep"
)
func commHost(state multistep.StateBag) (string, error) {
containerId := state.Get("container_id").(string)
driver := state.Get("driver").(Driver)
return driver.IPAddress(containerId)
func commHost(host string) func(multistep.StateBag) (string, error) {
return func(state multistep.StateBag) (string, error) {
if host != "" {
log.Printf("Using ssh_host value: %s", host)
return host, nil
}
containerId := state.Get("container_id").(string)
driver := state.Get("driver").(Driver)
return driver.IPAddress(containerId)
}
}

16
builder/docker/communicator.go
View file

@ -246,18 +246,16 @@ func (c *Communicator) Download(src string, dst io.Writer) error {
// enables it to work with directories. We don't actually support
// directories in Download() but we still need to handle the tar format.
stderrOut, err := ioutil.ReadAll(stderrP)
if err != nil {
return err
}
if string(stderrOut) != "" {
return fmt.Errorf("Error downloading file: %s", string(stderrOut))
}
archive := tar.NewReader(pipe)
_, err = archive.Next()
if err != nil {
// see if we can get a useful error message from stderr, since stdout
// is messed up.
if stderrOut, err := ioutil.ReadAll(stderrP); err == nil {
if string(stderrOut) != "" {
return fmt.Errorf("Error downloading file: %s", string(stderrOut))
}
}
return fmt.Errorf("Failed to read header from tar stream: %s", err)
}

3
builder/docker/config.go
View file

@ -5,7 +5,6 @@ package docker
import (
"fmt"
"os"
"runtime"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
@ -179,7 +178,7 @@ func NewConfig(raws ...interface{}) (*Config, []string, error) {
}
if c.ContainerDir == "" {
if runtime.GOOS == "windows" {
if c.WindowsContainer {
c.ContainerDir = "c:/packer-files"
} else {
c.ContainerDir = "/packer-files"

17
builder/docker/step_temp_dir.go
View file

@ -3,6 +3,7 @@ package docker
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
@ -18,22 +19,32 @@ type StepTempDir struct {
// ConfigTmpDir returns the configuration tmp directory for Docker
func ConfigTmpDir() (string, error) {
if tmpdir := os.Getenv("PACKER_TMP_DIR"); tmpdir != "" {
return filepath.Abs(tmpdir)
}
configdir, err := packer.ConfigDir()
if err != nil {
return "", err
}
if tmpdir := os.Getenv("PACKER_TMP_DIR"); tmpdir != "" {
// override the config dir with tmp dir. Still stat it and mkdirall if
// necessary.
fp, err := filepath.Abs(tmpdir)
log.Printf("found PACKER_TMP_DIR env variable; setting tmpdir to %s", fp)
if err != nil {
return "", err
}
configdir = fp
}
td := filepath.Join(configdir, "tmp")
_, err = os.Stat(td)
if os.IsNotExist(err) {
log.Printf("Creating tempdir in %s", td)
if err = os.MkdirAll(td, 0755); err != nil {
return "", err
}
} else if err != nil {
return "", err
}
log.Printf("Set Packer temp dir to %s", td)
return td, nil
}

44
builder/googlecompute/account.go
View file

@ -1,52 +1,34 @@
package googlecompute
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"strings"
"golang.org/x/oauth2/google"
"golang.org/x/oauth2/jwt"
)
// accountFile represents the structure of the account file JSON file.
type AccountFile struct {
PrivateKeyId string `json:"private_key_id"`
PrivateKey string `json:"private_key"`
ClientEmail string `json:"client_email"`
ClientId string `json:"client_id"`
}
func parseJSON(result interface{}, text string) error {
r := strings.NewReader(text)
dec := json.NewDecoder(r)
return dec.Decode(result)
}
func ProcessAccountFile(account_file *AccountFile, text string) error {
func ProcessAccountFile(text string) (*jwt.Config, error) {
// Assume text is a JSON string
if err := parseJSON(account_file, text); err != nil {
conf, err := google.JWTConfigFromJSON([]byte(text), DriverScopes...)
if err != nil {
// If text was not JSON, assume it is a file path instead
if _, err := os.Stat(text); os.IsNotExist(err) {
return fmt.Errorf(
return nil, fmt.Errorf(
"account_file path does not exist: %s",
text)
}
b, err := ioutil.ReadFile(text)
data, err := ioutil.ReadFile(text)
if err != nil {
return fmt.Errorf(
return nil, fmt.Errorf(
"Error reading account_file from path '%s': %s",
text, err)
}
contents := string(b)
if err := parseJSON(account_file, contents); err != nil {
return fmt.Errorf(
"Error parsing account file '%s': %s",
contents, err)
conf, err = google.JWTConfigFromJSON(data, DriverScopes...)
if err != nil {
return nil, fmt.Errorf("Error parsing account_file: %s", err)
}
}
return nil
return conf, nil
}

4
builder/googlecompute/builder.go
View file

@ -36,7 +36,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
// representing a GCE machine image.
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
driver, err := NewDriverGCE(
ui, b.config.ProjectId, &b.config.Account)
ui, b.config.ProjectId, b.config.Account)
if err != nil {
return nil, err
}
@ -67,7 +67,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "instance_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
WinRMConfig: winrmConfig,
},

7
builder/googlecompute/config.go
View file

@ -15,6 +15,7 @@ import (
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"golang.org/x/oauth2/jwt"
compute "google.golang.org/api/compute/v1"
)
@ -167,7 +168,7 @@ type Config struct {
// Example: "us-central1-a"
Zone string `mapstructure:"zone" required:"true"`
Account AccountFile
Account *jwt.Config
stateTimeout time.Duration
imageAlreadyExists bool
ctx interpolate.Context
@ -311,9 +312,11 @@ func NewConfig(raws ...interface{}) (*Config, []string, error) {
}
if c.AccountFile != "" {
if err := ProcessAccountFile(&c.Account, c.AccountFile); err != nil {
cfg, err := ProcessAccountFile(c.AccountFile)
if err != nil {
errs = packer.MultiErrorAppend(errs, err)
}
c.Account = cfg
}
if c.OmitExternalIP && c.Address != "" {

16
builder/googlecompute/config_test.go
View file

@ -500,6 +500,16 @@ func testMetadataFile(t *testing.T) string {
return tf.Name()
}
// This is just some dummy data that doesn't actually work (it was revoked
// a long time ago).
const testAccountContent = `{}`
// This is just some dummy data that doesn't actually work
const testAccountContent = `{
"type": "service_account",
"project_id": "test-project-123456789",
"private_key_id": "bananaphone",
"private_key": "-----BEGIN PRIVATE KEY-----\nring_ring_ring_ring_ring_ring_ring_BANANAPHONE\n-----END PRIVATE KEY-----\n",
"client_email": "raffi-compute@developer.gserviceaccount.com",
"client_id": "1234567890",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/12345-compute%40developer.gserviceaccount.com"
}`

22
builder/googlecompute/driver_gce.go
View file

@ -35,24 +35,17 @@ type driverGCE struct {
var DriverScopes = []string{"https://www.googleapis.com/auth/compute", "https://www.googleapis.com/auth/devstorage.full_control"}
func NewClientGCE(a *AccountFile) (*http.Client, error) {
func NewClientGCE(conf *jwt.Config) (*http.Client, error) {
var err error
var client *http.Client
// Auth with AccountFile first if provided
if a.PrivateKey != "" {
log.Printf("[INFO] Requesting Google token via AccountFile...")
log.Printf("[INFO] -- Email: %s", a.ClientEmail)
if conf != nil && len(conf.PrivateKey) > 0 {
log.Printf("[INFO] Requesting Google token via account_file...")
log.Printf("[INFO] -- Email: %s", conf.Email)
log.Printf("[INFO] -- Scopes: %s", DriverScopes)
log.Printf("[INFO] -- Private Key Length: %d", len(a.PrivateKey))
conf := jwt.Config{
Email: a.ClientEmail,
PrivateKey: []byte(a.PrivateKey),
Scopes: DriverScopes,
TokenURL: "https://accounts.google.com/o/oauth2/token",
}
log.Printf("[INFO] -- Private Key Length: %d", len(conf.PrivateKey))
// Initiate an http.Client. The following GET request will be
// authorized and authenticated on the behalf of
@ -82,8 +75,8 @@ func NewClientGCE(a *AccountFile) (*http.Client, error) {
return client, nil
}
func NewDriverGCE(ui packer.Ui, p string, a *AccountFile) (Driver, error) {
client, err := NewClientGCE(a)
func NewDriverGCE(ui packer.Ui, p string, conf *jwt.Config) (Driver, error) {
client, err := NewClientGCE(conf)
if err != nil {
return nil, err
}
@ -191,6 +184,7 @@ func (d *driverGCE) GetImage(name string, fromFamily bool) (*Image, error) {
"rhel-sap-cloud",
"suse-cloud",
"suse-sap-cloud",
"suse-byos-cloud",
"ubuntu-os-cloud",
"windows-cloud",
"windows-sql-cloud",

10
builder/googlecompute/ssh.go
View file

@ -1,10 +0,0 @@
package googlecompute
import (
"github.com/hashicorp/packer/helper/multistep"
)
func commHost(state multistep.StateBag) (string, error) {
ipAddress := state.Get("instance_ip").(string)
return ipAddress, nil
}

2
builder/googlecompute/step_create_windows_password.go
View file

@ -60,7 +60,7 @@ func (s *StepCreateWindowsPassword) Run(ctx context.Context, state multistep.Sta
UserName: c.Comm.WinRMUser,
Modulus: base64.StdEncoding.EncodeToString(priv.N.Bytes()),
Exponent: base64.StdEncoding.EncodeToString(buf[1:]),
Email: c.Account.ClientEmail,
Email: c.Account.Email,
ExpireOn: time.Now().Add(time.Minute * 5),
}

27
builder/hcloud/config.go
View file

@ -25,10 +25,11 @@ type Config struct {
Endpoint string `mapstructure:"endpoint"`
PollInterval time.Duration `mapstructure:"poll_interval"`
ServerName string `mapstructure:"server_name"`
Location string `mapstructure:"location"`
ServerType string `mapstructure:"server_type"`
Image string `mapstructure:"image"`
ServerName string `mapstructure:"server_name"`
Location string `mapstructure:"location"`
ServerType string `mapstructure:"server_type"`
Image string `mapstructure:"image"`
ImageFilter *imageFilter `mapstructure:"image_filter"`
SnapshotName string `mapstructure:"snapshot_name"`
SnapshotLabels map[string]string `mapstructure:"snapshot_labels"`
@ -41,6 +42,11 @@ type Config struct {
ctx interpolate.Context
}
type imageFilter struct {
WithSelector []string `mapstructure:"with_selector"`
MostRecent bool `mapstructure:"most_recent"`
}
func NewConfig(raws ...interface{}) (*Config, []string, error) {
c := new(Config)
@ -108,9 +114,18 @@ func NewConfig(raws ...interface{}) (*Config, []string, error) {
errs, errors.New("server type is required"))
}
if c.Image == "" {
if c.Image == "" && c.ImageFilter == nil {
errs = packer.MultiErrorAppend(
errs, errors.New("image is required"))
errs, errors.New("image or image_filter is required"))
}
if c.ImageFilter != nil {
if len(c.ImageFilter.WithSelector) == 0 {
errs = packer.MultiErrorAppend(
errs, errors.New("image_filter.with_selector is required when specifying filter"))
} else if c.Image != "" {
errs = packer.MultiErrorAppend(
errs, errors.New("only one of image or image_filter can be specified"))
}
}
if c.UserData != "" && c.UserDataFile != "" {

48
builder/hcloud/step_create_server.go
View file

@ -3,8 +3,9 @@ package hcloud
import (
"context"
"fmt"
"io/ioutil"
"sort"
"strings"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
@ -50,10 +51,24 @@ func (s *stepCreateServer) Run(ctx context.Context, state multistep.StateBag) mu
sshKeys = append(sshKeys, sshKey)
}
var image *hcloud.Image
if c.Image != "" {
image = &hcloud.Image{Name: c.Image}
} else {
var err error
image, err = getImageWithSelectors(ctx, client, c)
if err != nil {
ui.Error(err.Error())
state.Put("error", err)
return multistep.ActionHalt
}
ui.Message(fmt.Sprintf("Using image %s with ID %d", image.Description, image.ID))
}
serverCreateResult, _, err := client.Server.Create(ctx, hcloud.ServerCreateOpts{
Name: c.ServerName,
ServerType: &hcloud.ServerType{Name: c.ServerType},
Image: &hcloud.Image{Name: c.Image},
Image: image,
SSHKeys: sshKeys,
Location: &hcloud.Location{Name: c.Location},
UserData: userData,
@ -185,3 +200,32 @@ func waitForAction(ctx context.Context, client *hcloud.Client, action *hcloud.Ac
}
return nil
}
func getImageWithSelectors(ctx context.Context, client *hcloud.Client, c *Config) (*hcloud.Image, error) {
var allImages []*hcloud.Image
var selector = strings.Join(c.ImageFilter.WithSelector, ",")
opts := hcloud.ImageListOpts{
ListOpts: hcloud.ListOpts{LabelSelector: selector},
Status: []hcloud.ImageStatus{hcloud.ImageStatusAvailable},
}
allImages, err := client.Image.AllWithOpts(ctx, opts)
if err != nil {
return nil, err
}
if len(allImages) == 0 {
return nil, fmt.Errorf("no image found for selector %q", selector)
}
if len(allImages) > 1 {
if !c.ImageFilter.MostRecent {
return nil, fmt.Errorf("more than one image found for selector %q", selector)
}
sort.Slice(allImages, func(i, j int) bool {
return allImages[i].Created.After(allImages[j].Created)
})
}
return allImages[0], nil
}

31
builder/hyperv/common/step_run.go
View file

@ -5,6 +5,7 @@ import (
"fmt"
"log"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
@ -12,6 +13,7 @@ import (
type StepRun struct {
GuiCancelFunc context.CancelFunc
Headless bool
SwitchName string
vmName string
}
@ -20,9 +22,29 @@ func (s *StepRun) Run(ctx context.Context, state multistep.StateBag) multistep.S
ui := state.Get("ui").(packer.Ui)
vmName := state.Get("vmName").(string)
ui.Say("Determine Host IP for HyperV machine...")
hostIp, err := driver.GetHostAdapterIpAddressForSwitch(s.SwitchName)
if err != nil {
err := fmt.Errorf("Error getting host adapter ip address: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Say(fmt.Sprintf("Host IP for the HyperV machine: %s", hostIp))
common.SetHTTPIP(hostIp)
if !s.Headless {
ui.Say("Attempting to connect with vmconnect...")
s.GuiCancelFunc, err = driver.Connect(vmName)
if err != nil {
log.Printf(fmt.Sprintf("Non-fatal error starting vmconnect: %s. continuing...", err))
}
}
ui.Say("Starting the virtual machine...")
err := driver.Start(vmName)
err = driver.Start(vmName)
if err != nil {
err := fmt.Errorf("Error starting vm: %s", err)
state.Put("error", err)
@ -32,13 +54,6 @@ func (s *StepRun) Run(ctx context.Context, state multistep.StateBag) multistep.S
s.vmName = vmName
if !s.Headless {
ui.Say("Attempting to connect with vmconnect...")
s.GuiCancelFunc, err = driver.Connect(vmName)
if err != nil {
log.Printf(fmt.Sprintf("Non-fatal error starting vmconnect: %s. continuing...", err))
}
}
return multistep.ActionContinue
}

13
builder/hyperv/common/step_type_boot_command.go
View file

@ -33,6 +33,7 @@ func (s *StepTypeBootCommand) Run(ctx context.Context, state multistep.StateBag)
ui := state.Get("ui").(packer.Ui)
driver := state.Get("driver").(Driver)
vmName := state.Get("vmName").(string)
hostIp := common.GetHTTPIP()
// Wait the for the vm to boot.
if int64(s.BootWait) > 0 {
@ -45,18 +46,6 @@ func (s *StepTypeBootCommand) Run(ctx context.Context, state multistep.StateBag)
}
}
hostIp, err := driver.GetHostAdapterIpAddressForSwitch(s.SwitchName)
if err != nil {
err := fmt.Errorf("Error getting host adapter ip address: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Say(fmt.Sprintf("Host IP for the HyperV machine: %s", hostIp))
common.SetHTTPIP(hostIp)
s.Ctx.Data = &bootCommandTemplateData{
hostIp,
httpPort,

3
builder/hyperv/iso/builder.go
View file

@ -527,7 +527,8 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
&hypervcommon.StepRun{
Headless: b.config.Headless,
Headless: b.config.Headless,
SwitchName: b.config.SwitchName,
},
&hypervcommon.StepTypeBootCommand{

26
builder/hyperv/vmcx/builder.go
View file

@ -24,10 +24,10 @@ import (
)
const (
DefaultRamSize = 1 * 1024 // 1GB
MinRamSize = 32 // 32MB
MaxRamSize = 32 * 1024 // 32GB
MinNestedVirtualizationRamSize = 4 * 1024 // 4GB
DefaultRamSize = 1 * 1024 // 1GB
MinRamSize = 32 // 32MB
MaxRamSize = 1024 * 1024 // 1TB
MinNestedVirtualizationRamSize = 4 * 1024 // 4GB
LowRam = 256 // 256MB
@ -286,6 +286,21 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs, fmt.Errorf("CloneFromVMCXPath does not exist: %s", err))
}
}
if strings.HasSuffix(strings.ToLower(b.config.CloneFromVMCXPath), ".vmcx") {
// User has provided the vmcx file itself rather than the containing
// folder.
if strings.Contains(b.config.CloneFromVMCXPath, "Virtual Machines") {
keep := strings.Split(b.config.CloneFromVMCXPath, "Virtual Machines")
b.config.CloneFromVMCXPath = keep[0]
} else {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("Unable to "+
"parse the clone_from_vmcx_path to find the vm directory. "+
"Please provide the path to the folder containing the "+
"vmcx file, not the file itself. Example: instead of "+
"C:\\path\\to\\output-hyperv-iso\\Virtual Machines\\filename.vmcx"+
", provide C:\\path\\to\\output-hyperv-iso\\."))
}
}
}
if b.config.Generation < 1 || b.config.Generation > 2 {
@ -536,7 +551,8 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
&hypervcommon.StepRun{
Headless: b.config.Headless,
Headless: b.config.Headless,
SwitchName: b.config.SwitchName,
},
&hypervcommon.StepTypeBootCommand{

43
builder/jdcloud/artifact.go Normal file
View file

@ -0,0 +1,43 @@
package jdcloud
import (
"fmt"
"sort"
"strings"
)
type Artifact struct {
ImageId string
RegionID string
}
func (*Artifact) BuilderId() string {
return BUILDER_ID
}
func (*Artifact) Files() []string {
return nil
}
// Plan
// Though this part is supposed to be an array of Image Ids associated
// with its region, but currently only a single image is supported
func (a *Artifact) Id() string {
parts := []string{fmt.Sprintf("%s:%s", a.RegionID, a.ImageId)}
sort.Strings(parts)
return strings.Join(parts, ",")
}
func (a *Artifact) String() string {
return fmt.Sprintf("A VMImage was created: %s", a.ImageId)
}
// Plan
// State and destroy function is abandoned
func (a *Artifact) State(name string) interface{} {
return nil
}
func (a *Artifact) Destroy() error {
return nil
}

91
builder/jdcloud/builder.go Normal file
View file

@ -0,0 +1,91 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
)
func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
err := config.Decode(&b.config, &config.DecodeOpts{
Interpolate: true,
InterpolateContext: &b.config.ctx,
InterpolateFilter: &interpolate.RenderFilter{
Exclude: []string{
"boot_command",
},
},
}, raws...)
if err != nil {
return nil, fmt.Errorf("[ERROR] Failed in decoding JSON->mapstructure")
}
errs := &packer.MultiError{}
errs = packer.MultiErrorAppend(errs, b.config.JDCloudCredentialConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.JDCloudInstanceSpecConfig.Prepare(&b.config.ctx)...)
if errs != nil && len(errs.Errors) != 0 {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey)
return nil, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
state := new(multistep.BasicStateBag)
state.Put("hook", hook)
state.Put("ui", ui)
state.Put("config", b.config)
steps := []multistep.Step{
&stepValidateParameters{
InstanceSpecConfig: &b.config.JDCloudInstanceSpecConfig,
},
&stepConfigCredentials{
InstanceSpecConfig: &b.config.JDCloudInstanceSpecConfig,
},
&stepCreateJDCloudInstance{
InstanceSpecConfig: &b.config.JDCloudInstanceSpecConfig,
CredentialConfig: &b.config.JDCloudCredentialConfig,
},
&communicator.StepConnect{
Config: &b.config.JDCloudInstanceSpecConfig.Comm,
SSHConfig: b.config.JDCloudInstanceSpecConfig.Comm.SSHConfigFunc(),
Host: instanceHost,
},
&common.StepProvision{},
&stepStopJDCloudInstance{
InstanceSpecConfig: &b.config.JDCloudInstanceSpecConfig,
},
&stepCreateJDCloudImage{
InstanceSpecConfig: &b.config.JDCloudInstanceSpecConfig,
},
}
b.runner = common.NewRunnerWithPauseFn(steps, b.config.PackerConfig, ui, state)
b.runner.Run(ctx, state)
if rawErr, ok := state.GetOk("error"); ok {
return nil, rawErr.(error)
}
artifact := &Artifact{
ImageId: b.config.ArtifactId,
RegionID: b.config.RegionId,
}
return artifact, nil
}

432
builder/jdcloud/common.go Normal file
View file

@ -0,0 +1,432 @@
package jdcloud
import (
"fmt"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/template/interpolate"
vm "github.com/jdcloud-api/jdcloud-sdk-go/services/vm/client"
vpc "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/client"
"log"
"strings"
"sync"
"time"
)
const (
FINE = 0
CONNECT_FAILED = "Client.Timeout exceeded"
VM_PENDING = "pending"
VM_RUNNING = "running"
VM_STARTING = "starting"
VM_STOPPING = "stopping"
VM_STOPPED = "stopped"
READY = "ready"
BUILDER_ID = "hashicorp.jdcloud"
)
var (
VmClient *vm.VmClient
VpcClient *vpc.VpcClient
Region string
)
type Config struct {
JDCloudCredentialConfig `mapstructure:",squash"`
JDCloudInstanceSpecConfig `mapstructure:",squash"`
common.PackerConfig `mapstructure:",squash"`
ctx interpolate.Context
}
type Builder struct {
config Config
runner multistep.Runner
}
func Retry(timeout time.Duration, f RetryFunc) error {
// These are used to pull the error out of the function; need a mutex to
// avoid a data race.
var resultErr error
var resultErrMu sync.Mutex
c := &StateChangeConf{
Pending: []string{"retryableerror"},
Target: []string{"success"},
Timeout: timeout,
MinTimeout: 500 * time.Millisecond,
Refresh: func() (interface{}, string, error) {
rerr := f()
resultErrMu.Lock()
defer resultErrMu.Unlock()
if rerr == nil {
resultErr = nil
return 42, "success", nil
}
resultErr = rerr.Err
if rerr.Retryable {
return 42, "retryableerror", nil
}
return nil, "quit", rerr.Err
},
}
_, waitErr := c.WaitForState()
// Need to acquire the lock here to be able to avoid race using resultErr as
// the return value
resultErrMu.Lock()
defer resultErrMu.Unlock()
// resultErr may be nil because the wait timed out and resultErr was never
// set; this is still an error
if resultErr == nil {
return waitErr
}
// resultErr takes precedence over waitErr if both are set because it is
// more likely to be useful
return resultErr
}
// RetryFunc is the function retried until it succeeds.
type RetryFunc func() *RetryError
// RetryError is the required return type of RetryFunc. It forces client code
// to choose whether or not a given error is retryable.
type RetryError struct {
Err error
Retryable bool
}
// RetryableError is a helper to create a RetryError that's retryable from a
// given error.
func RetryableError(err error) *RetryError {
if err == nil {
return nil
}
return &RetryError{Err: err, Retryable: true}
}
// NonRetryableError is a helper to create a RetryError that's _not_ retryable
// from a given error.
func NonRetryableError(err error) *RetryError {
if err == nil {
return nil
}
return &RetryError{Err: err, Retryable: false}
}
// WaitForState watches an object and waits for it to achieve the state
// specified in the configuration using the specified Refresh() func,
// waiting the number of seconds specified in the timeout configuration.
//
// If the Refresh function returns an error, exit immediately with that error.
//
// If the Refresh function returns a state other than the Target state or one
// listed in Pending, return immediately with an error.
//
// If the Timeout is exceeded before reaching the Target state, return an
// error.
//
// Otherwise, the result is the result of the first call to the Refresh function to
// reach the target state.
func (conf *StateChangeConf) WaitForState() (interface{}, error) {
log.Printf("[DEBUG] Waiting for state to become: %s", conf.Target)
notfoundTick := 0
targetOccurence := 0
// Set a default for times to check for not found
if conf.NotFoundChecks == 0 {
conf.NotFoundChecks = 20
}
if conf.ContinuousTargetOccurence == 0 {
conf.ContinuousTargetOccurence = 1
}
type Result struct {
Result interface{}
State string
Error error
Done bool
}
// Read every result from the refresh loop, waiting for a positive result.Done.
resCh := make(chan Result, 1)
// cancellation channel for the refresh loop
cancelCh := make(chan struct{})
result := Result{}
go func() {
defer close(resCh)
time.Sleep(conf.Delay)
// start with 0 delay for the first loop
var wait time.Duration
for {
// store the last result
resCh <- result
// wait and watch for cancellation
select {
case <-cancelCh:
return
case <-time.After(wait):
// first round had no wait
if wait == 0 {
wait = 100 * time.Millisecond
}
}
res, currentState, err := conf.Refresh()
result = Result{
Result: res,
State: currentState,
Error: err,
}
if err != nil {
resCh <- result
return
}
// If we're waiting for the absence of a thing, then return
if res == nil && len(conf.Target) == 0 {
targetOccurence++
if conf.ContinuousTargetOccurence == targetOccurence {
result.Done = true
resCh <- result
return
}
continue
}
if res == nil {
// If we didn't find the resource, check if we have been
// not finding it for awhile, and if so, report an error.
notfoundTick++
if notfoundTick > conf.NotFoundChecks {
result.Error = &NotFoundError{
LastError: err,
Retries: notfoundTick,
}
resCh <- result
return
}
} else {
// Reset the counter for when a resource isn't found
notfoundTick = 0
found := false
for _, allowed := range conf.Target {
if currentState == allowed {
found = true
targetOccurence++
if conf.ContinuousTargetOccurence == targetOccurence {
result.Done = true
resCh <- result
return
}
continue
}
}
for _, allowed := range conf.Pending {
if currentState == allowed {
found = true
targetOccurence = 0
break
}
}
if !found && len(conf.Pending) > 0 {
result.Error = &UnexpectedStateError{
LastError: err,
State: result.State,
ExpectedState: conf.Target,
}
resCh <- result
return
}
}
// Wait between refreshes using exponential backoff, except when
// waiting for the target state to reoccur.
if targetOccurence == 0 {
wait *= 2
}
// If a poll interval has been specified, choose that interval.
// Otherwise bound the default value.
if conf.PollInterval > 0 && conf.PollInterval < 180*time.Second {
wait = conf.PollInterval
} else {
if wait < conf.MinTimeout {
wait = conf.MinTimeout
} else if wait > 10*time.Second {
wait = 10 * time.Second
}
}
log.Printf("[TRACE] Waiting %s before next try", wait)
}
}()
// store the last value result from the refresh loop
lastResult := Result{}
timeout := time.After(conf.Timeout)
for {
select {
case r, ok := <-resCh:
// channel closed, so return the last result
if !ok {
return lastResult.Result, lastResult.Error
}
// we reached the intended state
if r.Done {
return r.Result, r.Error
}
// still waiting, store the last result
lastResult = r
case <-timeout:
log.Printf("[WARN] WaitForState timeout after %s", conf.Timeout)
log.Printf("[WARN] WaitForState starting %s refresh grace period", 30*time.Second)
// cancel the goroutine and start our grace period timer
close(cancelCh)
timeout := time.After(30 * time.Second)
// we need a for loop and a label to break on, because we may have
// an extra response value to read, but still want to wait for the
// channel to close.
forSelect:
for {
select {
case r, ok := <-resCh:
if r.Done {
// the last refresh loop reached the desired state
return r.Result, r.Error
}
if !ok {
// the goroutine returned
break forSelect
}
// target state not reached, save the result for the
// TimeoutError and wait for the channel to close
lastResult = r
case <-timeout:
log.Println("[ERROR] WaitForState exceeded refresh grace period")
break forSelect
}
}
return nil, &TimeoutError{
LastError: lastResult.Error,
LastState: lastResult.State,
Timeout: conf.Timeout,
ExpectedState: conf.Target,
}
}
}
}
type StateChangeConf struct {
Delay time.Duration // Wait this time before starting checks
Pending []string // States that are "allowed" and will continue trying
Refresh StateRefreshFunc // Refreshes the current state
Target []string // Target state
Timeout time.Duration // The amount of time to wait before timeout
MinTimeout time.Duration // Smallest time to wait before refreshes
PollInterval time.Duration // Override MinTimeout/backoff and only poll this often
NotFoundChecks int // Number of times to allow not found
// This is to work around inconsistent APIs
ContinuousTargetOccurence int // Number of times the Target state has to occur continuously
}
type NotFoundError struct {
LastError error
LastRequest interface{}
LastResponse interface{}
Message string
Retries int
}
func (e *NotFoundError) Error() string {
if e.Message != "" {
return e.Message
}
if e.Retries > 0 {
return fmt.Sprintf("couldn't find resource (%d retries)", e.Retries)
}
return "couldn't find resource"
}
// UnexpectedStateError is returned when Refresh returns a state that's neither in Target nor Pending
type UnexpectedStateError struct {
LastError error
State string
ExpectedState []string
}
func (e *UnexpectedStateError) Error() string {
return fmt.Sprintf(
"unexpected state '%s', wanted target '%s'. last error: %s",
e.State,
strings.Join(e.ExpectedState, ", "),
e.LastError,
)
}
// TimeoutError is returned when WaitForState times out
type TimeoutError struct {
LastError error
LastState string
Timeout time.Duration
ExpectedState []string
}
func (e *TimeoutError) Error() string {
expectedState := "resource to be gone"
if len(e.ExpectedState) > 0 {
expectedState = fmt.Sprintf("state to become '%s'", strings.Join(e.ExpectedState, ", "))
}
extraInfo := make([]string, 0)
if e.LastState != "" {
extraInfo = append(extraInfo, fmt.Sprintf("last state: '%s'", e.LastState))
}
if e.Timeout > 0 {
extraInfo = append(extraInfo, fmt.Sprintf("timeout: %s", e.Timeout.String()))
}
suffix := ""
if len(extraInfo) > 0 {
suffix = fmt.Sprintf(" (%s)", strings.Join(extraInfo, ", "))
}
if e.LastError != nil {
return fmt.Sprintf("timeout while waiting for %s%s: %s",
expectedState, suffix, e.LastError)
}
return fmt.Sprintf("timeout while waiting for %s%s",
expectedState, suffix)
}
type StateRefreshFunc func() (result interface{}, state string, err error)

86
builder/jdcloud/credential_config.go Normal file
View file

@ -0,0 +1,86 @@
package jdcloud
import (
"fmt"
"github.com/hashicorp/packer/template/interpolate"
"github.com/jdcloud-api/jdcloud-sdk-go/core"
vm "github.com/jdcloud-api/jdcloud-sdk-go/services/vm/client"
vpc "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/client"
"os"
)
type JDCloudCredentialConfig struct {
AccessKey string `mapstructure:"access_key"`
SecretKey string `mapstructure:"secret_key"`
RegionId string `mapstructure:"region_id"`
Az string `mapstructure:"az"`
}
func (jd *JDCloudCredentialConfig) Prepare(ctx *interpolate.Context) []error {
errorArray := []error{}
if jd == nil {
return append(errorArray, fmt.Errorf("[PRE-FLIGHT] Empty JDCloudCredentialConfig detected"))
}
if err := jd.ValidateKeyPair(); err != nil {
errorArray = append(errorArray, err)
}
if err := jd.validateRegion(); err != nil {
errorArray = append(errorArray, err)
}
if err := jd.validateAz(); err != nil {
errorArray = append(errorArray, err)
}
if len(errorArray) != 0 {
return errorArray
}
credential := core.NewCredentials(jd.AccessKey, jd.SecretKey)
VmClient = vm.NewVmClient(credential)
VpcClient = vpc.NewVpcClient(credential)
Region = jd.RegionId
return nil
}
func (jd *JDCloudCredentialConfig) ValidateKeyPair() error {
if jd.AccessKey == "" {
jd.AccessKey = os.Getenv("JDCLOUD_ACCESS_KEY")
}
if jd.SecretKey == "" {
jd.SecretKey = os.Getenv("JDCLOUD_SECRET_KEY")
}
if jd.AccessKey == "" || jd.SecretKey == "" {
return fmt.Errorf("[PRE-FLIGHT] We can't find your key pairs," +
"write them here {access_key=xxx , secret_key=xxx} " +
"or export them as env-variable, {export JDCLOUD_ACCESS_KEY=xxx, export JDCLOUD_SECRET_KEY=xxx} ")
}
return nil
}
func (config *JDCloudCredentialConfig) validateRegion() error {
regionArray := []string{"cn-north-1", "cn-south-1", "cn-east-1", "cn-east-2"}
for _, item := range regionArray {
if item == config.RegionId {
return nil
}
}
return fmt.Errorf("[PRE-FLIGHT] Invalid RegionId:%s. "+
"Legit RegionId are: {cn-north-1, cn-south-1, cn-east-1, cn-east-2}", config.RegionId)
}
func (config *JDCloudCredentialConfig) validateAz() error {
if len(config.Az) == 0 {
return fmt.Errorf("[PRE-FLIGHT] az info missing")
}
return nil
}

34
builder/jdcloud/credential_config_test.go Normal file
View file

@ -0,0 +1,34 @@
package jdcloud
import (
"testing"
)
func TestJDCloudCredentialConfig_Prepare(t *testing.T) {
creds := &JDCloudCredentialConfig{}
if err := creds.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when there's nothing set")
}
creds.AccessKey = "abc"
if err := creds.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when theres no Secret key")
}
creds.SecretKey = "123"
if err := creds.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when theres no Az and region")
}
creds.RegionId = "cn-west-1"
creds.Az = "cn-north-1c"
if err := creds.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when region_id illegal")
}
creds.RegionId = "cn-north-1"
if err := creds.Prepare(nil); err != nil {
t.Fatalf("Test shouldn't fail...")
}
}

53
builder/jdcloud/instance_config.go Normal file
View file

@ -0,0 +1,53 @@
package jdcloud
import (
"fmt"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/template/interpolate"
)
type JDCloudInstanceSpecConfig struct {
ImageId string `mapstructure:"image_id"`
InstanceName string `mapstructure:"instance_name"`
InstanceType string `mapstructure:"instance_type"`
ImageName string `mapstructure:"image_name"`
SubnetId string `mapstructure:"subnet_id"`
Comm communicator.Config `mapstructure:",squash"`
InstanceId string
ArtifactId string
PublicIpAddress string
PublicIpId string
}
func (jd *JDCloudInstanceSpecConfig) Prepare(ctx *interpolate.Context) []error {
errs := jd.Comm.Prepare(ctx)
if jd == nil {
return append(errs, fmt.Errorf("[PRE-FLIGHT] Configuration appears to be empty"))
}
if len(jd.ImageId) == 0 {
errs = append(errs, fmt.Errorf("[PRE-FLIGHT] 'image_id' empty"))
}
if len(jd.InstanceName) == 0 {
errs = append(errs, fmt.Errorf("[PRE-FLIGHT] 'instance_name' empty"))
}
if len(jd.InstanceType) == 0 {
errs = append(errs, fmt.Errorf("[PRE-FLIGHT] 'instance-type' empty"))
}
noPassword := len(jd.Comm.SSHPassword) == 0
noKeys := len(jd.Comm.SSHKeyPairName) == 0 && len(jd.Comm.SSHPrivateKeyFile) == 0
noTempKey := len(jd.Comm.SSHTemporaryKeyPairName) == 0
if noPassword && noKeys && noTempKey {
errs = append(errs, fmt.Errorf("[PRE-FLIGHT] Didn't detect any credentials, you have to specify either "+
"{password} or "+
"{key_name+local_private_key_path} or "+
"{temporary_key_pair_name} cheers :)"))
}
return errs
}

54
builder/jdcloud/instance_config_test.go Normal file
View file

@ -0,0 +1,54 @@
package jdcloud
import (
"testing"
)
func TestJDCloudInstanceSpecConfig_Prepare(t *testing.T) {
specs := &JDCloudInstanceSpecConfig{}
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when there's nothing set")
}
specs.InstanceName = "packer_test_instance_name"
specs.InstanceType = "packer_test_instance_type"
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when base-image not given")
}
specs.ImageId = "img-packer-test"
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when credentials not set")
}
specs.Comm.SSHPassword = "abc123"
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when username = nil")
}
specs.Comm.SSHUsername = "root"
if err := specs.Prepare(nil); err != nil {
t.Fatalf("Test shouldn't fail when password set ")
}
specs.Comm.SSHPassword = ""
specs.Comm.SSHTemporaryKeyPairName = "abc"
if err := specs.Prepare(nil); err != nil {
t.Fatalf("Test shouldn't fail when temp password set")
}
specs.Comm.SSHTemporaryKeyPairName = ""
specs.Comm.SSHPrivateKeyFile = "abc"
specs.Comm.SSHKeyPairName = ""
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when SSHKeypairName missing")
}
specs.Comm.SSHPrivateKeyFile = "abc"
specs.Comm.SSHKeyPairName = "123"
if err := specs.Prepare(nil); err == nil {
t.Fatalf("Test shouldn't pass when private key pair path is wrong ")
}
}

73
builder/jdcloud/step_config_credentials.go Normal file
View file

@ -0,0 +1,73 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/jdcloud-api/jdcloud-sdk-go/services/vm/apis"
"io/ioutil"
)
type stepConfigCredentials struct {
InstanceSpecConfig *JDCloudInstanceSpecConfig
ui packer.Ui
}
func (s *stepConfigCredentials) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
s.ui = state.Get("ui").(packer.Ui)
password := s.InstanceSpecConfig.Comm.SSHPassword
privateKeyPath := s.InstanceSpecConfig.Comm.SSHPrivateKeyFile
privateKeyName := s.InstanceSpecConfig.Comm.SSHKeyPairName
newKeyName := s.InstanceSpecConfig.Comm.SSHTemporaryKeyPairName
if len(privateKeyPath) > 0 && len(privateKeyName) > 0 {
s.ui.Message("\t Private key detected, we are going to login with this private key :)")
return s.ReadExistingPair()
}
if len(newKeyName) > 0 {
s.ui.Message("\t We are going to create a new key pair with its name=" + newKeyName)
return s.CreateRandomKeyPair(newKeyName)
}
if len(password) > 0 {
s.ui.Message("\t Password detected, we are going to login with this password :)")
return multistep.ActionContinue
}
s.ui.Error("[ERROR] Didn't detect any credentials, you have to specify either " +
"{password} or " +
"{key_name+local_private_key_path} or " +
"{temporary_key_pair_name} cheers :)")
return multistep.ActionHalt
}
func (s *stepConfigCredentials) ReadExistingPair() multistep.StepAction {
privateKeyBytes, err := ioutil.ReadFile(s.InstanceSpecConfig.Comm.SSHPrivateKeyFile)
if err != nil {
s.ui.Error("Cannot read local private-key, were they correctly placed? Here's the error" + err.Error())
return multistep.ActionHalt
}
s.ui.Message("\t\t Keys read successfully :)")
s.InstanceSpecConfig.Comm.SSHPrivateKey = privateKeyBytes
return multistep.ActionContinue
}
func (s *stepConfigCredentials) CreateRandomKeyPair(keyName string) multistep.StepAction {
req := apis.NewCreateKeypairRequest(Region, keyName)
resp, err := VmClient.CreateKeypair(req)
if err != nil || resp.Error.Code != FINE {
s.ui.Error(fmt.Sprintf("[ERROR] Cannot create a new key pair for you, \n error=%v \n response=%v", err, resp))
return multistep.ActionHalt
}
s.ui.Message("\t\t Keys created successfully :)")
s.InstanceSpecConfig.Comm.SSHPrivateKey = []byte(resp.Result.PrivateKey)
s.InstanceSpecConfig.Comm.SSHKeyPairName = keyName
return multistep.ActionContinue
}
func (s *stepConfigCredentials) Cleanup(state multistep.StateBag) {
}

74
builder/jdcloud/step_create_image.go Normal file
View file

@ -0,0 +1,74 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/jdcloud-api/jdcloud-sdk-go/services/vm/apis"
"time"
)
type stepCreateJDCloudImage struct {
InstanceSpecConfig *JDCloudInstanceSpecConfig
}
func (s *stepCreateJDCloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
ui.Say("Creating images")
req := apis.NewCreateImageRequest(Region, s.InstanceSpecConfig.InstanceId, s.InstanceSpecConfig.ImageName, "")
resp, err := VmClient.CreateImage(req)
if err != nil || resp.Error.Code != FINE {
ui.Error(fmt.Sprintf("[ERROR] Creating image: Error-%v ,Resp:%v", err, resp))
return multistep.ActionHalt
}
s.InstanceSpecConfig.ArtifactId = resp.Result.ImageId
if err := ImageStatusWaiter(s.InstanceSpecConfig.ArtifactId); err != nil {
ui.Error(err.Error())
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func ImageStatusWaiter(imageId string) error {
req := apis.NewDescribeImageRequest(Region, imageId)
return Retry(5*time.Minute, func() *RetryError {
resp, err := VmClient.DescribeImage(req)
if err == nil && resp.Result.Image.Status == READY {
return nil
}
if connectionError(err) {
return RetryableError(err)
} else {
return NonRetryableError(err)
}
})
}
// Delete created instance image on error
func (s *stepCreateJDCloudImage) Cleanup(state multistep.StateBag) {
if s.InstanceSpecConfig.ArtifactId != "" {
req := apis.NewDeleteImageRequest(Region, s.InstanceSpecConfig.ArtifactId)
_ = Retry(time.Minute, func() *RetryError {
_, err := VmClient.DeleteImage(req)
if err == nil {
return nil
}
if connectionError(err) {
return RetryableError(err)
} else {
return NonRetryableError(err)
}
})
}
}

223
builder/jdcloud/step_create_instance.go Normal file
View file

@ -0,0 +1,223 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/jdcloud-api/jdcloud-sdk-go/core"
"github.com/jdcloud-api/jdcloud-sdk-go/services/vm/apis"
vm "github.com/jdcloud-api/jdcloud-sdk-go/services/vm/models"
vpcApis "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/apis"
vpcClient "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/client"
vpc "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/models"
"regexp"
"time"
)
type stepCreateJDCloudInstance struct {
InstanceSpecConfig *JDCloudInstanceSpecConfig
CredentialConfig *JDCloudCredentialConfig
ui packer.Ui
}
func (s *stepCreateJDCloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
privateKey := s.InstanceSpecConfig.Comm.SSHPrivateKey
keyName := s.InstanceSpecConfig.Comm.SSHKeyPairName
password := s.InstanceSpecConfig.Comm.SSHPassword
s.ui = state.Get("ui").(packer.Ui)
s.ui.Say("Creating instances")
instanceSpec := vm.InstanceSpec{
Az: &s.CredentialConfig.Az,
InstanceType: &s.InstanceSpecConfig.InstanceType,
ImageId: &s.InstanceSpecConfig.ImageId,
Name: s.InstanceSpecConfig.InstanceName,
PrimaryNetworkInterface: &vm.InstanceNetworkInterfaceAttachmentSpec{
NetworkInterface: &vpc.NetworkInterfaceSpec{
SubnetId: s.InstanceSpecConfig.SubnetId,
Az: &s.CredentialConfig.Az,
},
},
}
if len(password) > 0 {
instanceSpec.Password = &password
}
if len(keyName) > 0 && len(privateKey) > 0 {
instanceSpec.KeyNames = []string{keyName}
}
req := apis.NewCreateInstancesRequest(Region, &instanceSpec)
resp, err := VmClient.CreateInstances(req)
if err != nil || resp.Error.Code != FINE {
err := fmt.Errorf("Error creating instance, error-%v response:%v", err, resp)
s.ui.Error(err.Error())
return multistep.ActionHalt
}
s.InstanceSpecConfig.InstanceId = resp.Result.InstanceIds[0]
instanceInterface, err := InstanceStatusRefresher(s.InstanceSpecConfig.InstanceId, []string{VM_PENDING, VM_STARTING}, []string{VM_RUNNING})
if err != nil {
s.ui.Error(err.Error())
return multistep.ActionHalt
}
instance := instanceInterface.(vm.Instance)
privateIpAddress := instance.PrivateIpAddress
networkInterfaceId := instance.PrimaryNetworkInterface.NetworkInterface.NetworkInterfaceId
s.ui.Message("Creating public-ip")
s.InstanceSpecConfig.PublicIpId, err = createElasticIp(state)
if err != nil {
s.ui.Error(err.Error())
return multistep.ActionHalt
}
s.ui.Message("Associating public-ip with instance")
err = associatePublicIp(networkInterfaceId, s.InstanceSpecConfig.PublicIpId, privateIpAddress)
if err != nil {
s.ui.Error(err.Error())
return multistep.ActionHalt
}
req_ := vpcApis.NewDescribeElasticIpRequest(Region, s.InstanceSpecConfig.PublicIpId)
eip, err := VpcClient.DescribeElasticIp(req_)
if err != nil || eip.Error.Code != FINE {
s.ui.Error(fmt.Sprintf("[ERROR] Failed in getting eip,error:%v \n response:%v", err, eip))
return multistep.ActionHalt
}
s.InstanceSpecConfig.PublicIpAddress = eip.Result.ElasticIp.ElasticIpAddress
state.Put("eip", s.InstanceSpecConfig.PublicIpAddress)
s.ui.Message(fmt.Sprintf(
"Hi, we have created the instance, its name=%v , "+
"its id=%v, "+
"and its eip=%v :) ", instance.InstanceName, s.InstanceSpecConfig.InstanceId, eip.Result.ElasticIp.ElasticIpAddress))
return multistep.ActionContinue
}
// Delete created resources {instance,ip} on error
func (s *stepCreateJDCloudInstance) Cleanup(state multistep.StateBag) {
if s.InstanceSpecConfig.PublicIpId != "" {
req := vpcApis.NewDeleteElasticIpRequest(Region, s.InstanceSpecConfig.PublicIpId)
_ = Retry(time.Minute, func() *RetryError {
_, err := VpcClient.DeleteElasticIp(req)
if err == nil {
return nil
}
if connectionError(err) {
return RetryableError(err)
} else {
return NonRetryableError(err)
}
})
}
if s.InstanceSpecConfig.InstanceId != "" {
req := apis.NewDeleteInstanceRequest(Region, s.InstanceSpecConfig.InstanceId)
_ = Retry(time.Minute, func() *RetryError {
_, err := VmClient.DeleteInstance(req)
if err == nil {
return nil
}
if connectionError(err) {
return RetryableError(err)
} else {
return NonRetryableError(err)
}
})
}
}
func createElasticIp(state multistep.StateBag) (string, error) {
generalConfig := state.Get("config").(Config)
regionId := generalConfig.RegionId
credential := core.NewCredentials(generalConfig.AccessKey, generalConfig.SecretKey)
vpcclient := vpcClient.NewVpcClient(credential)
req := vpcApis.NewCreateElasticIpsRequest(regionId, 1, &vpc.ElasticIpSpec{
BandwidthMbps: 1,
Provider: "bgp",
})
resp, err := vpcclient.CreateElasticIps(req)
if err != nil || resp.Error.Code != 0 {
return "", fmt.Errorf("[ERROR] Failed in creating new publicIp, Error-%v, Response:%v", err, resp)
}
return resp.Result.ElasticIpIds[0], nil
}
func associatePublicIp(networkInterfaceId string, eipId string, privateIpAddress string) error {
req := vpcApis.NewAssociateElasticIpRequest(Region, networkInterfaceId)
req.ElasticIpId = &eipId
req.PrivateIpAddress = &privateIpAddress
resp, err := VpcClient.AssociateElasticIp(req)
if err != nil || resp.Error.Code != FINE {
return fmt.Errorf("[ERROR] Failed in associating publicIp, Error-%v, Response:%v", err, resp)
}
return nil
}
func instanceHost(state multistep.StateBag) (string, error) {
return state.Get("eip").(string), nil
}
func InstanceStatusRefresher(id string, pending, target []string) (instance interface{}, err error) {
stateConf := &StateChangeConf{
Pending: pending,
Target: target,
Refresh: instanceStatusRefresher(id),
Delay: 3 * time.Second,
Timeout: 10 * time.Minute,
MinTimeout: 1 * time.Second,
}
if instance, err = stateConf.WaitForState(); err != nil {
return nil, fmt.Errorf("[ERROR] Failed in creating instance ,err message:%v", err)
}
return instance, nil
}
func instanceStatusRefresher(instanceId string) StateRefreshFunc {
return func() (instance interface{}, status string, err error) {
err = Retry(time.Minute, func() *RetryError {
req := apis.NewDescribeInstanceRequest(Region, instanceId)
resp, err := VmClient.DescribeInstance(req)
if err == nil && resp.Error.Code == FINE {
instance = resp.Result.Instance
status = resp.Result.Instance.Status
return nil
}
instance = nil
status = ""
if connectionError(err) {
return RetryableError(err)
} else {
return NonRetryableError(err)
}
})
return instance, status, err
}
}
func connectionError(e error) bool {
if e == nil {
return false
}
ok, _ := regexp.MatchString(CONNECT_FAILED, e.Error())
return ok
}

39
builder/jdcloud/step_stop_instance.go Normal file
View file

@ -0,0 +1,39 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/jdcloud-api/jdcloud-sdk-go/services/vm/apis"
)
type stepStopJDCloudInstance struct {
InstanceSpecConfig *JDCloudInstanceSpecConfig
}
func (s *stepStopJDCloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
ui.Say("Stopping this instance")
req := apis.NewStopInstanceRequest(Region, s.InstanceSpecConfig.InstanceId)
resp, err := VmClient.StopInstance(req)
if err != nil || resp.Error.Code != FINE {
ui.Error(fmt.Sprintf("[ERROR] Failed in trying to stop this vm: Error-%v ,Resp:%v", err, resp))
return multistep.ActionHalt
}
_, err = InstanceStatusRefresher(s.InstanceSpecConfig.InstanceId, []string{VM_RUNNING, VM_STOPPING}, []string{VM_STOPPED})
if err != nil {
ui.Error(err.Error())
return multistep.ActionHalt
}
ui.Message("Instance has been stopped :)")
return multistep.ActionContinue
}
func (s *stepStopJDCloudInstance) Cleanup(multistep.StateBag) {
return
}

109
builder/jdcloud/step_validate_parameters.go Normal file
View file

@ -0,0 +1,109 @@
package jdcloud
import (
"context"
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
vm "github.com/jdcloud-api/jdcloud-sdk-go/services/vm/apis"
vpc "github.com/jdcloud-api/jdcloud-sdk-go/services/vpc/apis"
)
type stepValidateParameters struct {
InstanceSpecConfig *JDCloudInstanceSpecConfig
ui packer.Ui
state multistep.StateBag
}
func (s *stepValidateParameters) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
s.ui = state.Get("ui").(packer.Ui)
s.state = state
s.ui.Say("Validating parameters...")
if err := s.ValidateSubnetFunc(); err != nil {
s.ui.Error(err.Error())
return multistep.ActionHalt
}
if err := s.ValidateImageFunc(); err != nil {
s.ui.Error(err.Error())
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (s *stepValidateParameters) ValidateSubnetFunc() error {
subnetId := s.InstanceSpecConfig.SubnetId
if len(subnetId) == 0 {
s.ui.Message("\t 'subnet' is not specified, we will create a new one for you :) ")
return s.CreateRandomSubnet()
}
s.ui.Message("\t validating your subnet:" + s.InstanceSpecConfig.SubnetId)
req := vpc.NewDescribeSubnetRequest(Region, subnetId)
resp, err := VpcClient.DescribeSubnet(req)
if err != nil {
return fmt.Errorf("[ERROR] Failed in validating subnet->%s, reasons:%v", subnetId, err)
}
if resp != nil && resp.Error.Code != FINE {
return fmt.Errorf("[ERROR] Something wrong with your subnet->%s, reasons:%v", subnetId, resp.Error)
}
s.ui.Message("\t subnet found:" + resp.Result.Subnet.SubnetName)
return nil
}
func (s *stepValidateParameters) ValidateImageFunc() error {
s.ui.Message("\t validating your base image:" + s.InstanceSpecConfig.ImageId)
imageId := s.InstanceSpecConfig.ImageId
req := vm.NewDescribeImageRequest(Region, imageId)
resp, err := VmClient.DescribeImage(req)
if err != nil {
return fmt.Errorf("[ERROR] Failed in validating your image->%s, reasons:%v", imageId, err)
}
if resp != nil && resp.Error.Code != FINE {
return fmt.Errorf("[ERROR] Something wrong with your image->%s, reasons:%v", imageId, resp.Error)
}
s.ui.Message("\t image found:" + resp.Result.Image.Name)
s.state.Put("source_image", &resp.Result.Image)
return nil
}
func (s *stepValidateParameters) CreateRandomSubnet() error {
newVpc, err := s.CreateRandomVpc()
if err != nil {
return err
}
req := vpc.NewCreateSubnetRequest(Region, newVpc, "created_by_packer", "192.168.0.0/20")
resp, err := VpcClient.CreateSubnet(req)
if err != nil || resp.Error.Code != FINE {
errorMessage := fmt.Sprintf("[ERROR] Failed in creating new subnet :( \n error:%v \n response:%v", err, resp)
s.ui.Error(errorMessage)
return fmt.Errorf(errorMessage)
}
s.InstanceSpecConfig.SubnetId = resp.Result.SubnetId
s.ui.Message("\t\t Hi, we have created a new subnet for you :) its name is 'created_by_packer' and its id=" + resp.Result.SubnetId)
return nil
}
func (s *stepValidateParameters) CreateRandomVpc() (string, error) {
req := vpc.NewCreateVpcRequest(Region, "created_by_packer")
resp, err := VpcClient.CreateVpc(req)
if err != nil || resp.Error.Code != FINE {
errorMessage := fmt.Sprintf("[ERROR] Failed in creating new vpc :( \n error :%v, \n response:%v", err, resp)
s.ui.Error(errorMessage)
return "", fmt.Errorf(errorMessage)
}
return resp.Result.VpcId, nil
}
func (s *stepValidateParameters) Cleanup(state multistep.StateBag) {}

2
builder/linode/builder.go
View file

@ -56,7 +56,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (ret
&stepCreateLinode{client},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: commHost(b.config.Comm.SSHHost),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

18
builder/linode/ssh.go
View file

@ -2,18 +2,26 @@ package linode
import (
"fmt"
"log"
"github.com/hashicorp/packer/helper/multistep"
"github.com/linode/linodego"
"golang.org/x/crypto/ssh"
)
func commHost(state multistep.StateBag) (string, error) {
instance := state.Get("instance").(*linodego.Instance)
if len(instance.IPv4) == 0 {
return "", fmt.Errorf("Linode instance %d has no IPv4 addresses!", instance.ID)
func commHost(host string) func(multistep.StateBag) (string, error) {
return func(state multistep.StateBag) (string, error) {
if host != "" {
log.Printf("Using ssh_host value: %s", host)
return host, nil
}
instance := state.Get("instance").(*linodego.Instance)
if len(instance.IPv4) == 0 {
return "", fmt.Errorf("Linode instance %d has no IPv4 addresses!", instance.ID)
}
return instance.IPv4[0].String(), nil
}
return instance.IPv4[0].String(), nil
}
func sshConfig(state multistep.StateBag) (*ssh.ClientConfig, error) {

2
builder/oneandone/builder.go
View file

@ -44,7 +44,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
new(stepCreateServer),
&communicator.StepConnect{
Config: &b.config.Comm,
Host: commHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "server_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

10
builder/oneandone/ssh.go
View file

@ -1,10 +0,0 @@
package oneandone
import (
"github.com/hashicorp/packer/helper/multistep"
)
func commHost(state multistep.StateBag) (string, error) {
ipAddress := state.Get("server_ip").(string)
return ipAddress, nil
}

13
builder/openstack/builder.go
View file

@ -126,16 +126,18 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
Wait: b.config.RackconnectWait,
},
&StepAllocateIp{
FloatingIPNetwork: b.config.FloatingIPNetwork,
FloatingIP: b.config.FloatingIP,
ReuseIPs: b.config.ReuseIPs,
FloatingIPNetwork: b.config.FloatingIPNetwork,
FloatingIP: b.config.FloatingIP,
ReuseIPs: b.config.ReuseIPs,
InstanceFloatingIPNet: b.config.InstanceFloatingIPNet,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: CommHost(
b.config.RunConfig.Comm.SSHHost,
computeClient,
b.config.Comm.SSHInterface,
b.config.Comm.SSHIPVersion),
b.config.SSHInterface,
b.config.SSHIPVersion),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
@ -152,6 +154,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
&stepUpdateImageTags{},
&stepUpdateImageVisibility{},
&stepAddImageMembers{},
&stepUpdateImageMinDisk{},
}
// Run!

6
builder/openstack/image_config.go
View file

@ -27,6 +27,8 @@ type ImageConfig struct {
ImageDiskFormat string `mapstructure:"image_disk_format" required:"false"`
// List of tags to add to the image after creation.
ImageTags []string `mapstructure:"image_tags" required:"false"`
// Minimum disk size needed to boot image, in gigabytes.
ImageMinDisk int `mapstructure:"image_min_disk" required:"false"`
}
func (c *ImageConfig) Prepare(ctx *interpolate.Context) []error {
@ -63,6 +65,10 @@ func (c *ImageConfig) Prepare(ctx *interpolate.Context) []error {
}
}
if c.ImageMinDisk < 0 {
errs = append(errs, fmt.Errorf("An image min disk size must be greater than or equal to 0"))
}
if len(errs) > 0 {
return errs
}

17
builder/openstack/networks.go
View file

@ -2,6 +2,7 @@ package openstack
import (
"fmt"
"log"
"github.com/google/uuid"
"github.com/gophercloud/gophercloud"
@ -66,7 +67,10 @@ func FindFreeFloatingIP(client *gophercloud.ServiceClient) (*floatingips.Floatin
// GetInstancePortID returns internal port of the instance that can be used for
// the association of a floating IP.
// It will return an ID of a first port if there are many.
func GetInstancePortID(client *gophercloud.ServiceClient, id string) (string, error) {
func GetInstancePortID(client *gophercloud.ServiceClient, id string, instance_float_net string) (string, error) {
selected_interface := 0
interfacesPage, err := attachinterfaces.List(client, id).AllPages()
if err != nil {
return "", err
@ -79,7 +83,16 @@ func GetInstancePortID(client *gophercloud.ServiceClient, id string) (string, er
return "", fmt.Errorf("instance '%s' has no interfaces", id)
}
return interfaces[0].PortID, nil
for i := 0; i < len(interfaces); i++ {
log.Printf("Instance interface: %v: %+v\n", i, interfaces[i])
if interfaces[i].NetID == instance_float_net {
log.Printf("Found preferred interface: %v\n", i)
selected_interface = i
log.Printf("Using interface value: %v", selected_interface)
}
}
return interfaces[selected_interface].PortID, nil
}
// CheckFloatingIPNetwork checks provided network reference and returns a valid

18
builder/openstack/run_config.go
View file

@ -16,6 +16,15 @@ import (
// and details on how to access that launched image.
type RunConfig struct {
Comm communicator.Config `mapstructure:",squash"`
// The type of interface to connect via SSH. Values useful for Rackspace
// are "public" or "private", and the default behavior is to connect via
// whichever is returned first from the OpenStack API.
SSHInterface string `mapstructure:"ssh_interface" required:"false"`
// The IP version to use for SSH connections, valid values are `4` and `6`.
// Useful on dual stacked instances where the default behavior is to
// connect via whichever IP address is returned first from the OpenStack
// API.
SSHIPVersion string `mapstructure:"ssh_ip_version" required:"false"`
// The ID or full URL to the base image to use. This is the image that will
// be used to launch a new server and provision it. Unless you specify
// completely custom SSH settings, the source image must have cloud-init
@ -86,6 +95,13 @@ type RunConfig struct {
// The ID or name of an external network that can be used for creation of a
// new floating IP.
FloatingIPNetwork string `mapstructure:"floating_ip_network" required:"false"`
// The ID of the network to which the instance is attached and which should
// be used to associate with the floating IP. This provides control over
// the floating ip association on multi-homed instances. The association
// otherwise depends on a first-returned-interface policy which could fail
// if the network to which it is connected is unreachable from the floating
// IP network.
InstanceFloatingIPNet string `mapstructure:"instance_floating_ip_net" required:"false"`
// A specific floating IP to assign to this instance.
FloatingIP string `mapstructure:"floating_ip" required:"false"`
// Whether or not to attempt to reuse existing unassigned floating ips in
@ -239,7 +255,7 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
errs = append(errs, errors.New("A flavor must be specified"))
}
if c.Comm.SSHIPVersion != "" && c.Comm.SSHIPVersion != "4" && c.Comm.SSHIPVersion != "6" {
if c.SSHIPVersion != "" && c.SSHIPVersion != "4" && c.SSHIPVersion != "6" {
errs = append(errs, errors.New("SSH IP version must be either 4 or 6"))
}

6
builder/openstack/ssh.go
View file

@ -14,10 +14,16 @@ import (
// CommHost looks up the host for the communicator.
func CommHost(
host string,
client *gophercloud.ServiceClient,
sshinterface string,
sshipversion string) func(multistep.StateBag) (string, error) {
return func(state multistep.StateBag) (string, error) {
if host != "" {
log.Printf("Using ssh_host value: %s", host)
return host, nil
}
s := state.Get("server").(*servers.Server)
// If we have a specific interface, try that

9
builder/openstack/step_allocate_ip.go
View file

@ -11,9 +11,10 @@ import (
)
type StepAllocateIp struct {
FloatingIPNetwork string
FloatingIP string
ReuseIPs bool
FloatingIPNetwork string
FloatingIP string
ReuseIPs bool
InstanceFloatingIPNet string
}
func (s *StepAllocateIp) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
@ -114,7 +115,7 @@ func (s *StepAllocateIp) Run(ctx context.Context, state multistep.StateBag) mult
ui.Say(fmt.Sprintf("Associating floating IP '%s' (%s) with instance port...",
instanceIP.ID, instanceIP.FloatingIP))
portID, err := GetInstancePortID(computeClient, server.ID)
portID, err := GetInstancePortID(computeClient, server.ID, s.InstanceFloatingIPNet)
if err != nil {
err := fmt.Errorf("Error getting interfaces of the instance '%s': %s", server.ID, err)
state.Put("error", err)

9
builder/openstack/step_create_image.go
View file

@ -64,6 +64,15 @@ func (s *stepCreateImage) Run(ctx context.Context, state multistep.StateBag) mul
ui.Error(err.Error())
return multistep.ActionHalt
}
err = volumeactions.SetImageMetadata(blockStorageClient, volume, volumeactions.ImageMetadataOpts{
Metadata: config.ImageMetadata,
}).ExtractErr()
if err != nil {
err := fmt.Errorf("Error setting image metadata: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
imageId = image.ImageID
} else {
imageId, err = servers.CreateImage(computeClient, server.ID, servers.CreateImageOpts{

2
builder/openstack/step_create_volume.go
View file

@ -125,7 +125,7 @@ func (s *StepCreateVolume) Cleanup(state multistep.StateBag) {
}
}
ui.Say(fmt.Sprintf("Deleting volume: %s ...", s.volumeID))
err = volumes.Delete(blockStorageClient, s.volumeID).ExtractErr()
err = volumes.Delete(blockStorageClient, s.volumeID, volumes.DeleteOpts{}).ExtractErr()
if err != nil {
ui.Error(fmt.Sprintf(
"Error cleaning up volume. Please delete the volume manually: %s", s.volumeID))

13
builder/openstack/step_get_password.go
View file

@ -8,6 +8,7 @@ import (
"time"
"github.com/gophercloud/gophercloud/openstack/compute/v2/servers"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
@ -17,8 +18,9 @@ import (
// StepGetPassword reads the password from a booted OpenStack server and sets
// it on the WinRM config.
type StepGetPassword struct {
Debug bool
Comm *communicator.Config
Debug bool
Comm *communicator.Config
BuildName string
}
func (s *StepGetPassword) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
@ -76,7 +78,12 @@ func (s *StepGetPassword) Run(ctx context.Context, state multistep.StateBag) mul
"Password (since debug is enabled) \"%s\"", s.Comm.WinRMPassword))
}
commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword, s.BuildName)
packer.LogSecretFilter.Set(s.Comm.WinRMPassword)
return multistep.ActionContinue
}
func (s *StepGetPassword) Cleanup(multistep.StateBag) {}
func (s *StepGetPassword) Cleanup(multistep.StateBag) {
commonhelper.RemoveSharedStateFile("winrm_password", s.BuildName)
}

52
builder/openstack/step_update_image_mindisk.go Normal file
View file

@ -0,0 +1,52 @@
package openstack
import (
"context"
"fmt"
"github.com/gophercloud/gophercloud/openstack/imageservice/v2/images"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
type stepUpdateImageMinDisk struct{}
func (s *stepUpdateImageMinDisk) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
imageId := state.Get("image").(string)
ui := state.Get("ui").(packer.Ui)
config := state.Get("config").(*Config)
if config.ImageMinDisk == 0 {
return multistep.ActionContinue
}
imageClient, err := config.imageV2Client()
if err != nil {
err := fmt.Errorf("Error initializing image service client: %s", err)
state.Put("error", err)
return multistep.ActionHalt
}
ui.Say(fmt.Sprintf("Updating image min disk to %d", config.ImageMinDisk))
r := images.Update(
imageClient,
imageId,
images.UpdateOpts{
images.ReplaceImageMinDisk{
NewMinDisk: config.ImageMinDisk,
},
},
)
if _, err := r.Extract(); err != nil {
err = fmt.Errorf("Error updating image min disk: %s", err)
state.Put("error", err)
return multistep.ActionHalt
}
return multistep.ActionContinue
}
func (s *stepUpdateImageMinDisk) Cleanup(multistep.StateBag) {
// No cleanup...
}

6
builder/oracle/classic/builder.go
View file

@ -99,7 +99,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: ocommon.CommHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "instance_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
@ -129,7 +129,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
KeyName: fmt.Sprintf("packer-generated-key_%s", runID),
StepConnectSSH: &communicator.StepConnectSSH{
Config: &b.config.BuilderComm,
Host: ocommon.CommHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "instance_ip"),
SSHConfig: b.config.BuilderComm.SSHConfigFunc(),
},
},
@ -162,7 +162,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
&stepCreateInstance{},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: ocommon.CommHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "instance_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

10
builder/oracle/common/ssh.go
View file

@ -1,10 +0,0 @@
package common
import (
"github.com/hashicorp/packer/helper/multistep"
)
func CommHost(state multistep.StateBag) (string, error) {
ipAddress := state.Get("instance_ip").(string)
return ipAddress, nil
}

2
builder/oracle/oci/builder.go
View file

@ -65,7 +65,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
},
&communicator.StepConnect{
Config: &b.config.Comm,
Host: ocommon.CommHost,
Host: communicator.CommHost(b.config.Comm.SSHHost, "instance_ip"),
SSHConfig: b.config.Comm.SSHConfigFunc(),
},
&common.StepProvision{},

219
builder/osc/bsu/builder.go Normal file
View file

@ -0,0 +1,219 @@
// Package bsu contains a packer.Builder implementation that
// builds OMIs for Outscale OAPI.
//
// In general, there are two types of OMIs that can be created: ebs-backed or
// instance-store. This builder _only_ builds ebs-backed images.
package bsu
import (
"context"
"crypto/tls"
"fmt"
"net/http"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/outscale/osc-go/oapi"
)
// The unique ID for this builder
const BuilderId = "oapi.outscale.bsu"
type Config struct {
common.PackerConfig `mapstructure:",squash"`
osccommon.AccessConfig `mapstructure:",squash"`
osccommon.OMIConfig `mapstructure:",squash"`
osccommon.BlockDevices `mapstructure:",squash"`
osccommon.RunConfig `mapstructure:",squash"`
VolumeRunTags osccommon.TagMap `mapstructure:"run_volume_tags"`
ctx interpolate.Context
}
type Builder struct {
config Config
runner multistep.Runner
}
func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.ctx.Funcs = osccommon.TemplateFuncs
err := config.Decode(&b.config, &config.DecodeOpts{
Interpolate: true,
InterpolateContext: &b.config.ctx,
InterpolateFilter: &interpolate.RenderFilter{
Exclude: []string{
"omi_description",
"run_tags",
"run_volume_tags",
"spot_tags",
"snapshot_tags",
"tags",
},
},
}, raws...)
if err != nil {
return nil, err
}
if b.config.PackerConfig.PackerForce {
b.config.OMIForceDeregister = true
}
// Accumulate any errors
var errs *packer.MultiError
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs,
b.config.OMIConfig.Prepare(&b.config.AccessConfig, &b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.BlockDevices.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
clientConfig, err := b.config.Config()
if err != nil {
return nil, err
}
skipClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
oapiconn := oapi.NewClient(clientConfig, skipClient)
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("oapi", oapiconn)
state.Put("clientConfig", clientConfig)
state.Put("hook", hook)
state.Put("ui", ui)
steps := []multistep.Step{
&osccommon.StepPreValidate{
DestOmiName: b.config.OMIName,
ForceDeregister: b.config.OMIForceDeregister,
},
&osccommon.StepSourceOMIInfo{
SourceOmi: b.config.SourceOmi,
OmiFilters: b.config.SourceOmiFilter,
OMIVirtType: b.config.OMIVirtType, //TODO: Remove if it is not used
},
&osccommon.StepNetworkInfo{
NetId: b.config.NetId,
NetFilter: b.config.NetFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
SubregionName: b.config.Subregion,
},
&osccommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("oapi_%s", b.config.PackerBuildName),
},
&osccommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
&osccommon.StepCleanupVolumes{
BlockDevices: b.config.BlockDevices,
},
&osccommon.StepRunSourceVm{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
BlockDevices: b.config.BlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
BsuOptimized: b.config.BsuOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: osccommon.RunSourceVmBSUExpectedRootDevice,
IamVmProfile: b.config.IamVmProfile,
VmInitiatedShutdownBehavior: b.config.VmInitiatedShutdownBehavior,
VmType: b.config.VmType,
IsRestricted: false,
SourceOMI: b.config.SourceOmi,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
},
&osccommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: osccommon.SSHHost(
oapiconn,
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&osccommon.StepStopBSUBackedVm{
Skip: false,
DisableStopVm: b.config.DisableStopVm,
},
&osccommon.StepDeregisterOMI{
AccessConfig: &b.config.AccessConfig,
ForceDeregister: b.config.OMIForceDeregister,
ForceDeleteSnapshot: b.config.OMIForceDeleteSnapshot,
OMIName: b.config.OMIName,
Regions: b.config.OMIRegions,
},
&stepCreateOMI{},
&osccommon.StepUpdateOMIAttributes{
AccountIds: b.config.OMIAccountIDs,
SnapshotAccountIds: b.config.SnapshotAccountIDs,
Ctx: b.config.ctx,
},
&osccommon.StepCreateTags{
Tags: b.config.OMITags,
SnapshotTags: b.config.SnapshotTags,
Ctx: b.config.ctx,
},
}
b.runner = common.NewRunner(steps, b.config.PackerConfig, ui)
b.runner.Run(ctx, state)
// If there was an error, return that
if rawErr, ok := state.GetOk("error"); ok {
return nil, rawErr.(error)
}
//Build the artifact
if omis, ok := state.GetOk("omis"); ok {
// Build the artifact and return it
artifact := &osccommon.Artifact{
Omis: omis.(map[string]string),
BuilderIdValue: BuilderId,
Config: clientConfig,
}
return artifact, nil
}
return nil, nil
}

53
builder/osc/bsu/builder_acc_test.go Normal file
View file

@ -0,0 +1,53 @@
//TODO: explain how to delete the image.
package bsu
import (
"crypto/tls"
"net/http"
"testing"
"github.com/hashicorp/packer/builder/osc/common"
builderT "github.com/hashicorp/packer/helper/builder/testing"
"github.com/outscale/osc-go/oapi"
)
func TestBuilderAcc_basic(t *testing.T) {
builderT.Test(t, builderT.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Builder: &Builder{},
Template: testBuilderAccBasic,
SkipArtifactTeardown: true,
})
}
func testAccPreCheck(t *testing.T) {
}
func testOAPIConn() (*oapi.Client, error) {
access := &common.AccessConfig{RawRegion: "us-east-1"}
clientConfig, err := access.Config()
if err != nil {
return nil, err
}
skipClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
return oapi.NewClient(clientConfig, skipClient), nil
}
const testBuilderAccBasic = `
{
"builders": [{
"type": "test",
"region": "eu-west-2",
"vm_type": "t2.micro",
"source_omi": "ami-65efcc11",
"ssh_username": "outscale",
"omi_name": "packer-test {{timestamp}}"
}]
}
`

131
builder/osc/bsu/builder_test.go Normal file
View file

@ -0,0 +1,131 @@
package bsu
import (
"testing"
"github.com/hashicorp/packer/packer"
)
func testConfig() map[string]interface{} {
return map[string]interface{}{
"access_key": "foo",
"secret_key": "bar",
"source_omi": "foo",
"vm_type": "foo",
"region": "us-east-1",
"ssh_username": "root",
"omi_name": "foo",
}
}
func TestBuilder_ImplementsBuilder(t *testing.T) {
var raw interface{}
raw = &Builder{}
if _, ok := raw.(packer.Builder); !ok {
t.Fatalf("Builder should be a builder")
}
}
func TestBuilder_Prepare_BadType(t *testing.T) {
b := &Builder{}
c := map[string]interface{}{
"access_key": []string{},
}
warnings, err := b.Prepare(c)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatalf("prepare should fail")
}
}
func TestBuilderPrepare_OMIName(t *testing.T) {
var b Builder
config := testConfig()
// Test good
config["omi_name"] = "foo"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test bad
config["omi_name"] = "foo {{"
b = Builder{}
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
// Test bad
delete(config, "omi_name")
b = Builder{}
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_InvalidShutdownBehavior(t *testing.T) {
var b Builder
config := testConfig()
// Test good
config["shutdown_behavior"] = "terminate"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test good
config["shutdown_behavior"] = "stop"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test bad
config["shutdown_behavior"] = "foobar"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}

116
builder/osc/bsu/step_create_omi.go Normal file
View file

@ -0,0 +1,116 @@
package bsu
import (
"context"
"fmt"
"log"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/outscale/osc-go/oapi"
)
type stepCreateOMI struct {
image *oapi.Image
}
func (s *stepCreateOMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
oapiconn := state.Get("oapi").(*oapi.Client)
vm := state.Get("vm").(oapi.Vm)
ui := state.Get("ui").(packer.Ui)
// Create the image
omiName := config.OMIName
ui.Say(fmt.Sprintf("Creating OMI %s from vm %s", omiName, vm.VmId))
createOpts := oapi.CreateImageRequest{
VmId: vm.VmId,
ImageName: omiName,
BlockDeviceMappings: config.BlockDevices.BuildOMIDevices(),
}
resp, err := oapiconn.POST_CreateImage(createOpts)
if err != nil || resp.OK == nil {
err := fmt.Errorf("Error creating OMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
image := resp.OK.Image
// Set the OMI ID in the state
ui.Message(fmt.Sprintf("OMI: %s", image.ImageId))
omis := make(map[string]string)
omis[oapiconn.GetConfig().Region] = image.ImageId
state.Put("omis", omis)
// Wait for the image to become ready
ui.Say("Waiting for OMI to become ready...")
if err := osccommon.WaitUntilImageAvailable(oapiconn, image.ImageId); err != nil {
log.Printf("Error waiting for OMI: %s", err)
imagesResp, err := oapiconn.POST_ReadImages(oapi.ReadImagesRequest{
Filters: oapi.FiltersImage{
ImageIds: []string{image.ImageId},
},
})
if err != nil {
log.Printf("Unable to determine reason waiting for OMI failed: %s", err)
err = fmt.Errorf("Unknown error waiting for OMI.")
} else {
stateReason := imagesResp.OK.Images[0].StateComment
err = fmt.Errorf("Error waiting for OMI. Reason: %s", stateReason)
}
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
imagesResp, err := oapiconn.POST_ReadImages(oapi.ReadImagesRequest{
Filters: oapi.FiltersImage{
ImageIds: []string{image.ImageId},
},
})
if err != nil {
err := fmt.Errorf("Error searching for OMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
s.image = &imagesResp.OK.Images[0]
snapshots := make(map[string][]string)
for _, blockDeviceMapping := range imagesResp.OK.Images[0].BlockDeviceMappings {
if blockDeviceMapping.Bsu.SnapshotId != "" {
snapshots[oapiconn.GetConfig().Region] = append(snapshots[oapiconn.GetConfig().Region], blockDeviceMapping.Bsu.SnapshotId)
}
}
state.Put("snapshots", snapshots)
return multistep.ActionContinue
}
func (s *stepCreateOMI) Cleanup(state multistep.StateBag) {
if s.image == nil {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if !cancelled && !halted {
return
}
oapiconn := state.Get("oapi").(*oapi.Client)
ui := state.Get("ui").(packer.Ui)
ui.Say("Deregistering the OMI because cancellation or error...")
DeleteOpts := oapi.DeleteImageRequest{ImageId: s.image.ImageId}
if _, err := oapiconn.POST_DeleteImage(DeleteOpts); err != nil {
ui.Error(fmt.Sprintf("Error Deleting OMI, may still be around: %s", err))
return
}
}

249
builder/osc/bsusurrogate/builder.go Normal file
View file

@ -0,0 +1,249 @@
// Package bsusurrogate contains a packer.Builder implementation that
// builds a new EBS-backed OMI using an ephemeral instance.
package bsusurrogate
import (
"context"
"crypto/tls"
"errors"
"fmt"
"net/http"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/outscale/osc-go/oapi"
)
const BuilderId = "oapi.outscale.bsusurrogate"
type Config struct {
common.PackerConfig `mapstructure:",squash"`
osccommon.AccessConfig `mapstructure:",squash"`
osccommon.RunConfig `mapstructure:",squash"`
osccommon.BlockDevices `mapstructure:",squash"`
osccommon.OMIConfig `mapstructure:",squash"`
RootDevice RootBlockDevice `mapstructure:"omi_root_device"`
VolumeRunTags osccommon.TagMap `mapstructure:"run_volume_tags"`
ctx interpolate.Context
}
type Builder struct {
config Config
runner multistep.Runner
}
func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.ctx.Funcs = osccommon.TemplateFuncs
err := config.Decode(&b.config, &config.DecodeOpts{
Interpolate: true,
InterpolateContext: &b.config.ctx,
InterpolateFilter: &interpolate.RenderFilter{
Exclude: []string{
"omi_description",
"run_tags",
"run_volume_tags",
"snapshot_tags",
"spot_tags",
"tags",
},
},
}, raws...)
if err != nil {
return nil, err
}
if b.config.PackerConfig.PackerForce {
b.config.OMIForceDeregister = true
}
// Accumulate any errors
var errs *packer.MultiError
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs,
b.config.OMIConfig.Prepare(&b.config.AccessConfig, &b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.BlockDevices.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RootDevice.Prepare(&b.config.ctx)...)
if b.config.OMIVirtType == "" {
errs = packer.MultiErrorAppend(errs, errors.New("omi_virtualization_type is required."))
}
foundRootVolume := false
for _, launchDevice := range b.config.BlockDevices.LaunchMappings {
if launchDevice.DeviceName == b.config.RootDevice.SourceDeviceName {
foundRootVolume = true
}
}
if !foundRootVolume {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("no volume with name '%s' is found", b.config.RootDevice.SourceDeviceName))
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
clientConfig, err := b.config.Config()
if err != nil {
return nil, err
}
skipClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
oapiconn := oapi.NewClient(clientConfig, skipClient)
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("oapi", oapiconn)
state.Put("clientConfig", clientConfig)
state.Put("hook", hook)
state.Put("ui", ui)
//VMStep
omiDevices := b.config.BuildOMIDevices()
launchDevices := b.config.BuildLaunchDevices()
steps := []multistep.Step{
&osccommon.StepPreValidate{
DestOmiName: b.config.OMIName,
ForceDeregister: b.config.OMIForceDeregister,
},
&osccommon.StepSourceOMIInfo{
SourceOmi: b.config.SourceOmi,
OmiFilters: b.config.SourceOmiFilter,
OMIVirtType: b.config.OMIVirtType, //TODO: Remove if it is not used
},
&osccommon.StepNetworkInfo{
NetId: b.config.NetId,
NetFilter: b.config.NetFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
SubregionName: b.config.Subregion,
},
&osccommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("oapi_%s", b.config.PackerBuildName),
},
&osccommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
&osccommon.StepCleanupVolumes{
BlockDevices: b.config.BlockDevices,
},
&osccommon.StepRunSourceVm{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
BlockDevices: b.config.BlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
BsuOptimized: b.config.BsuOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: osccommon.RunSourceVmBSUExpectedRootDevice,
IamVmProfile: b.config.IamVmProfile,
VmInitiatedShutdownBehavior: b.config.VmInitiatedShutdownBehavior,
VmType: b.config.VmType,
IsRestricted: false,
SourceOMI: b.config.SourceOmi,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
},
&osccommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: osccommon.SSHHost(
oapiconn,
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&osccommon.StepStopBSUBackedVm{
Skip: false,
DisableStopVm: b.config.DisableStopVm,
},
&StepSnapshotVolumes{
LaunchDevices: launchDevices,
},
&osccommon.StepDeregisterOMI{
AccessConfig: &b.config.AccessConfig,
ForceDeregister: b.config.OMIForceDeregister,
ForceDeleteSnapshot: b.config.OMIForceDeleteSnapshot,
OMIName: b.config.OMIName,
Regions: b.config.OMIRegions,
},
&StepRegisterOMI{
RootDevice: b.config.RootDevice,
OMIDevices: omiDevices,
LaunchDevices: launchDevices,
},
&osccommon.StepUpdateOMIAttributes{
AccountIds: b.config.OMIAccountIDs,
SnapshotAccountIds: b.config.SnapshotAccountIDs,
Ctx: b.config.ctx,
},
&osccommon.StepCreateTags{
Tags: b.config.OMITags,
SnapshotTags: b.config.SnapshotTags,
Ctx: b.config.ctx,
},
}
b.runner = common.NewRunner(steps, b.config.PackerConfig, ui)
b.runner.Run(ctx, state)
// If there was an error, return that
if rawErr, ok := state.GetOk("error"); ok {
return nil, rawErr.(error)
}
//Build the artifact
if omis, ok := state.GetOk("omis"); ok {
// Build the artifact and return it
artifact := &osccommon.Artifact{
Omis: omis.(map[string]string),
BuilderIdValue: BuilderId,
Config: clientConfig,
}
return artifact, nil
}
return nil, nil
}

51
builder/osc/bsusurrogate/builder_acc_test.go Normal file
View file

@ -0,0 +1,51 @@
package bsusurrogate
import (
"testing"
builderT "github.com/hashicorp/packer/helper/builder/testing"
)
func TestBuilderAcc_basic(t *testing.T) {
builderT.Test(t, builderT.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Builder: &Builder{},
Template: testBuilderAccBasic,
SkipArtifactTeardown: true,
})
}
func testAccPreCheck(t *testing.T) {
}
const testBuilderAccBasic = `
{
"builders": [{
"type": "test",
"region": "eu-west-2",
"vm_type": "t2.micro",
"source_omi": "ami-65efcc11",
"ssh_username": "outscale",
"omi_name": "packer-test {{timestamp}}",
"omi_virtualization_type": "hvm",
"subregion_name": "eu-west-2a",
"launch_block_device_mappings" : [
{
"volume_type" : "io1",
"device_name" : "/dev/xvdf",
"delete_on_vm_deletion" : false,
"volume_size" : 10,
"iops": 300
}
],
"omi_root_device":{
"source_device_name": "/dev/xvdf",
"device_name": "/dev/sda1",
"delete_on_vm_deletion": true,
"volume_size": 10,
"volume_type": "standard"
}
}]
}
`

15
builder/osc/bsusurrogate/builder_test.go Normal file
View file

@ -0,0 +1,15 @@
package bsusurrogate
import (
"testing"
"github.com/hashicorp/packer/packer"
)
func TestBuilder_ImplementsBuilder(t *testing.T) {
var raw interface{}
raw = &Builder{}
if _, ok := raw.(packer.Builder); !ok {
t.Fatal("Builder should be a builder")
}
}

46
builder/osc/bsusurrogate/root_block_device.go Normal file
View file

@ -0,0 +1,46 @@
package bsusurrogate
import (
"errors"
"github.com/hashicorp/packer/template/interpolate"
)
type RootBlockDevice struct {
SourceDeviceName string `mapstructure:"source_device_name"`
DeviceName string `mapstructure:"device_name"`
DeleteOnVmDeletion bool `mapstructure:"delete_on_vm_deletion"`
IOPS int64 `mapstructure:"iops"`
VolumeType string `mapstructure:"volume_type"`
VolumeSize int64 `mapstructure:"volume_size"`
}
func (c *RootBlockDevice) Prepare(ctx *interpolate.Context) []error {
var errs []error
if c.SourceDeviceName == "" {
errs = append(errs, errors.New("source_device_name for the root_device must be specified"))
}
if c.DeviceName == "" {
errs = append(errs, errors.New("device_name for the root_device must be specified"))
}
if c.VolumeType == "gp2" && c.IOPS != 0 {
errs = append(errs, errors.New("iops may not be specified for a gp2 volume"))
}
if c.IOPS < 0 {
errs = append(errs, errors.New("iops must be greater than 0"))
}
if c.VolumeSize < 0 {
errs = append(errs, errors.New("volume_size must be greater than 0"))
}
if len(errs) > 0 {
return errs
}
return nil
}

148
builder/osc/bsusurrogate/step_register_omi.go Normal file
View file

@ -0,0 +1,148 @@
package bsusurrogate
import (
"context"
"fmt"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/outscale/osc-go/oapi"
)
// StepRegisterOMI creates the OMI.
type StepRegisterOMI struct {
RootDevice RootBlockDevice
OMIDevices []oapi.BlockDeviceMappingImage
LaunchDevices []oapi.BlockDeviceMappingVmCreation
image *oapi.Image
}
func (s *StepRegisterOMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
oapiconn := state.Get("oapi").(*oapi.Client)
snapshotIds := state.Get("snapshot_ids").(map[string]string)
ui := state.Get("ui").(packer.Ui)
ui.Say("Registering the OMI...")
blockDevices := s.combineDevices(snapshotIds)
registerOpts := oapi.CreateImageRequest{
ImageName: config.OMIName,
Architecture: "x86_64",
RootDeviceName: s.RootDevice.DeviceName,
BlockDeviceMappings: blockDevices,
}
registerResp, err := oapiconn.POST_CreateImage(registerOpts)
if err != nil {
state.Put("error", fmt.Errorf("Error registering OMI: %s", err))
ui.Error(state.Get("error").(error).Error())
return multistep.ActionHalt
}
// Set the OMI ID in the state
ui.Say(fmt.Sprintf("OMI: %s", registerResp.OK.Image.ImageId))
omis := make(map[string]string)
omis[oapiconn.GetConfig().Region] = registerResp.OK.Image.ImageId
state.Put("omis", omis)
// Wait for the image to become ready
ui.Say("Waiting for OMI to become ready...")
if err := osccommon.WaitUntilImageAvailable(oapiconn, registerResp.OK.Image.ImageId); err != nil {
err := fmt.Errorf("Error waiting for OMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
imagesResp, err := oapiconn.POST_ReadImages(oapi.ReadImagesRequest{
Filters: oapi.FiltersImage{
ImageIds: []string{registerResp.OK.Image.ImageId},
},
})
if err != nil {
err := fmt.Errorf("Error searching for OMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
s.image = &imagesResp.OK.Images[0]
snapshots := make(map[string][]string)
for _, blockDeviceMapping := range imagesResp.OK.Images[0].BlockDeviceMappings {
if blockDeviceMapping.Bsu.SnapshotId != "" {
snapshots[oapiconn.GetConfig().Region] = append(snapshots[oapiconn.GetConfig().Region], blockDeviceMapping.Bsu.SnapshotId)
}
}
state.Put("snapshots", snapshots)
return multistep.ActionContinue
}
func (s *StepRegisterOMI) Cleanup(state multistep.StateBag) {
if s.image == nil {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if !cancelled && !halted {
return
}
oapiconn := state.Get("oapi").(*oapi.Client)
ui := state.Get("ui").(packer.Ui)
ui.Say("Deregistering the OMI because cancellation or error...")
deregisterOpts := oapi.DeleteImageRequest{ImageId: s.image.ImageId}
if _, err := oapiconn.POST_DeleteImage(deregisterOpts); err != nil {
ui.Error(fmt.Sprintf("Error deregistering OMI, may still be around: %s", err))
return
}
}
func (s *StepRegisterOMI) combineDevices(snapshotIds map[string]string) []oapi.BlockDeviceMappingImage {
devices := map[string]oapi.BlockDeviceMappingImage{}
for _, device := range s.OMIDevices {
devices[device.DeviceName] = device
}
// Devices in launch_block_device_mappings override any with
// the same name in ami_block_device_mappings, except for the
// one designated as the root device in ami_root_device
for _, device := range s.LaunchDevices {
snapshotId, ok := snapshotIds[device.DeviceName]
if ok {
device.Bsu.SnapshotId = snapshotId
}
if device.DeviceName == s.RootDevice.SourceDeviceName {
device.DeviceName = s.RootDevice.DeviceName
}
devices[device.DeviceName] = copyToDeviceMappingImage(device)
}
blockDevices := []oapi.BlockDeviceMappingImage{}
for _, device := range devices {
blockDevices = append(blockDevices, device)
}
return blockDevices
}
func copyToDeviceMappingImage(device oapi.BlockDeviceMappingVmCreation) oapi.BlockDeviceMappingImage {
deviceImage := oapi.BlockDeviceMappingImage{
DeviceName: device.DeviceName,
VirtualDeviceName: device.VirtualDeviceName,
Bsu: oapi.BsuToCreate{
DeleteOnVmDeletion: device.Bsu.DeleteOnVmDeletion,
Iops: device.Bsu.Iops,
SnapshotId: device.Bsu.SnapshotId,
VolumeSize: device.Bsu.VolumeSize,
VolumeType: device.Bsu.VolumeType,
},
}
return deviceImage
}

107
builder/osc/bsusurrogate/step_snapshop_volumes.go Normal file
View file

@ -0,0 +1,107 @@
package bsusurrogate
import (
"context"
"fmt"
"sync"
"time"
multierror "github.com/hashicorp/go-multierror"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/outscale/osc-go/oapi"
)
// StepSnapshotVolumes creates snapshots of the created volumes.
//
// Produces:
// snapshot_ids map[string]string - IDs of the created snapshots
type StepSnapshotVolumes struct {
LaunchDevices []oapi.BlockDeviceMappingVmCreation
snapshotIds map[string]string
}
func (s *StepSnapshotVolumes) snapshotVolume(ctx context.Context, deviceName string, state multistep.StateBag) error {
oapiconn := state.Get("oapi").(*oapi.Client)
ui := state.Get("ui").(packer.Ui)
vm := state.Get("vm").(oapi.Vm)
var volumeId string
for _, volume := range vm.BlockDeviceMappings {
if volume.DeviceName == deviceName {
volumeId = volume.Bsu.VolumeId
}
}
if volumeId == "" {
return fmt.Errorf("Volume ID for device %s not found", deviceName)
}
ui.Say(fmt.Sprintf("Creating snapshot of EBS Volume %s...", volumeId))
description := fmt.Sprintf("Packer: %s", time.Now().String())
createSnapResp, err := oapiconn.POST_CreateSnapshot(oapi.CreateSnapshotRequest{
VolumeId: volumeId,
Description: description,
})
if err != nil {
return err
}
// Set the snapshot ID so we can delete it later
s.snapshotIds[deviceName] = createSnapResp.OK.Snapshot.SnapshotId
// Wait for snapshot to be created
err = osccommon.WaitUntilSnapshotCompleted(oapiconn, createSnapResp.OK.Snapshot.SnapshotId)
return err
}
func (s *StepSnapshotVolumes) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
s.snapshotIds = map[string]string{}
var wg sync.WaitGroup
var errs *multierror.Error
for _, device := range s.LaunchDevices {
wg.Add(1)
go func(device oapi.BlockDeviceMappingVmCreation) {
defer wg.Done()
if err := s.snapshotVolume(ctx, device.DeviceName, state); err != nil {
errs = multierror.Append(errs, err)
}
}(device)
}
wg.Wait()
if errs != nil {
state.Put("error", errs)
ui.Error(errs.Error())
return multistep.ActionHalt
}
state.Put("snapshot_ids", s.snapshotIds)
return multistep.ActionContinue
}
func (s *StepSnapshotVolumes) Cleanup(state multistep.StateBag) {
if len(s.snapshotIds) == 0 {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {
oapiconn := state.Get("oapi").(*oapi.Client)
ui := state.Get("ui").(packer.Ui)
ui.Say("Removing snapshots since we cancelled or halted...")
for _, snapshotId := range s.snapshotIds {
_, err := oapiconn.POST_DeleteSnapshot(oapi.DeleteSnapshotRequest{SnapshotId: snapshotId})
if err != nil {
ui.Error(fmt.Sprintf("Error: %s", err))
}
}
}
}

87
builder/osc/bsuvolume/artifact.go Normal file
View file

@ -0,0 +1,87 @@
package bsuvolume
import (
"fmt"
"log"
"sort"
"strings"
"github.com/hashicorp/packer/packer"
"github.com/outscale/osc-go/oapi"
)
// map of region to list of volume IDs
type BsuVolumes map[string][]string
// Artifact is an artifact implementation that contains built AMIs.
type Artifact struct {
// A map of regions to EBS Volume IDs.
Volumes BsuVolumes
// BuilderId is the unique ID for the builder that created this AMI
BuilderIdValue string
// Client connection for performing API stuff.
Conn *oapi.Client
}
func (a *Artifact) BuilderId() string {
return a.BuilderIdValue
}
func (*Artifact) Files() []string {
// We have no files
return nil
}
// returns a sorted list of region:ID pairs
func (a *Artifact) idList() []string {
parts := make([]string, 0, len(a.Volumes))
for region, volumeIDs := range a.Volumes {
for _, volumeID := range volumeIDs {
parts = append(parts, fmt.Sprintf("%s:%s", region, volumeID))
}
}
sort.Strings(parts)
return parts
}
func (a *Artifact) Id() string {
return strings.Join(a.idList(), ",")
}
func (a *Artifact) String() string {
return fmt.Sprintf("EBS Volumes were created:\n\n%s", strings.Join(a.idList(), "\n"))
}
func (a *Artifact) State(name string) interface{} {
return nil
}
func (a *Artifact) Destroy() error {
errors := make([]error, 0)
for region, volumeIDs := range a.Volumes {
for _, volumeID := range volumeIDs {
log.Printf("Deregistering Volume ID (%s) from region (%s)", volumeID, region)
input := oapi.DeleteVolumeRequest{
VolumeId: volumeID,
}
if _, err := a.Conn.POST_DeleteVolume(input); err != nil {
errors = append(errors, err)
}
}
}
if len(errors) > 0 {
if len(errors) == 1 {
return errors[0]
} else {
return &packer.MultiError{Errors: errors}
}
}
return nil
}

29
builder/osc/bsuvolume/block_device.go Normal file
View file

@ -0,0 +1,29 @@
package bsuvolume
import (
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/template/interpolate"
)
type BlockDevice struct {
osccommon.BlockDevice `mapstructure:"-,squash"`
Tags osccommon.TagMap `mapstructure:"tags"`
}
func commonBlockDevices(mappings []BlockDevice, ctx *interpolate.Context) (osccommon.BlockDevices, error) {
result := make([]osccommon.BlockDevice, len(mappings))
for i, mapping := range mappings {
interpolateBlockDev, err := interpolate.RenderInterface(&mapping.BlockDevice, ctx)
if err != nil {
return osccommon.BlockDevices{}, err
}
result[i] = *interpolateBlockDev.(*osccommon.BlockDevice)
}
return osccommon.BlockDevices{
LaunchBlockDevices: osccommon.LaunchBlockDevices{
LaunchMappings: result,
},
}, nil
}

198
builder/osc/bsuvolume/builder.go Normal file
View file

@ -0,0 +1,198 @@
// The ebsvolume package contains a packer.Builder implementation that
// builds EBS volumes for Outscale using an ephemeral instance,
package bsuvolume
import (
"context"
"crypto/tls"
"fmt"
"log"
"net/http"
osccommon "github.com/hashicorp/packer/builder/osc/common"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/outscale/osc-go/oapi"
)
const BuilderId = "oapi.outscale.bsuvolume"
type Config struct {
common.PackerConfig `mapstructure:",squash"`
osccommon.AccessConfig `mapstructure:",squash"`
osccommon.RunConfig `mapstructure:",squash"`
VolumeMappings []BlockDevice `mapstructure:"bsu_volumes"`
launchBlockDevices osccommon.BlockDevices
ctx interpolate.Context
}
type Builder struct {
config Config
runner multistep.Runner
}
type EngineVarsTemplate struct {
BuildRegion string
SourceOMI string
}
func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.ctx.Funcs = osccommon.TemplateFuncs
// Create passthrough for {{ .BuildRegion }} and {{ .SourceOMI }} variables
// so we can fill them in later
b.config.ctx.Data = &EngineVarsTemplate{
BuildRegion: `{{ .BuildRegion }}`,
SourceOMI: `{{ .SourceOMI }} `,
}
err := config.Decode(&b.config, &config.DecodeOpts{
Interpolate: true,
InterpolateContext: &b.config.ctx,
}, raws...)
if err != nil {
return nil, err
}
// Accumulate any errors
var errs *packer.MultiError
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.launchBlockDevices.Prepare(&b.config.ctx)...)
for _, d := range b.config.VolumeMappings {
if err := d.Prepare(&b.config.ctx); err != nil {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("OMIMapping: %s", err.Error()))
}
}
b.config.launchBlockDevices, err = commonBlockDevices(b.config.VolumeMappings, &b.config.ctx)
if err != nil {
errs = packer.MultiErrorAppend(errs, err)
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
packer.LogSecretFilter.Set(b.config.AccessKey, b.config.SecretKey, b.config.Token)
return nil, nil
}
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
clientConfig, err := b.config.Config()
if err != nil {
return nil, err
}
skipClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
oapiconn := oapi.NewClient(clientConfig, skipClient)
// Setup the state bag and initial state for the steps
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("oapi", oapiconn)
state.Put("hook", hook)
state.Put("ui", ui)
log.Printf("[DEBUG] launch block devices %#v", b.config.launchBlockDevices)
instanceStep := &osccommon.StepRunSourceVm{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
BlockDevices: b.config.launchBlockDevices,
Comm: &b.config.RunConfig.Comm,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
BsuOptimized: b.config.BsuOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamVmProfile: b.config.IamVmProfile,
VmInitiatedShutdownBehavior: b.config.VmInitiatedShutdownBehavior,
VmType: b.config.VmType,
SourceOMI: b.config.SourceOmi,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
}
// Build the steps
steps := []multistep.Step{
&osccommon.StepSourceOMIInfo{
SourceOmi: b.config.SourceOmi,
OmiFilters: b.config.SourceOmiFilter,
},
&osccommon.StepNetworkInfo{
NetId: b.config.NetId,
NetFilter: b.config.NetFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
SecurityGroupFilter: b.config.SecurityGroupFilter,
SubnetId: b.config.SubnetId,
SubnetFilter: b.config.SubnetFilter,
SubregionName: b.config.Subregion,
},
&osccommon.StepKeyPair{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
DebugKeyPath: fmt.Sprintf("oapi_%s.pem", b.config.PackerBuildName),
},
&osccommon.StepSecurityGroup{
SecurityGroupFilter: b.config.SecurityGroupFilter,
SecurityGroupIds: b.config.SecurityGroupIds,
CommConfig: &b.config.RunConfig.Comm,
TemporarySGSourceCidr: b.config.TemporarySGSourceCidr,
},
instanceStep,
&stepTagBSUVolumes{
VolumeMapping: b.config.VolumeMappings,
Ctx: b.config.ctx,
},
&osccommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: osccommon.SSHHost(
oapiconn,
b.config.SSHInterface),
SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
},
&common.StepProvision{},
&common.StepCleanupTempKeys{
Comm: &b.config.RunConfig.Comm,
},
&osccommon.StepStopBSUBackedVm{
Skip: b.config.IsSpotVm(),
DisableStopVm: b.config.DisableStopVm,
},
}
// Run!
b.runner = common.NewRunner(steps, b.config.PackerConfig, ui)
b.runner.Run(ctx, state)
// If there was an error, return that
if rawErr, ok := state.GetOk("error"); ok {
return nil, rawErr.(error)
}
// Build the artifact and return it
artifact := &Artifact{
Volumes: state.Get("bsuvolumes").(BsuVolumes),
BuilderIdValue: BuilderId,
Conn: oapiconn,
}
ui.Say(fmt.Sprintf("Created Volumes: %s", artifact))
return artifact, nil
}

86
builder/osc/bsuvolume/builder_acc_test.go Normal file
View file

@ -0,0 +1,86 @@
//TODO: explain how to delete the image.
package bsuvolume
import (
"crypto/tls"
"net/http"
"testing"
"github.com/hashicorp/packer/builder/osc/common"
builderT "github.com/hashicorp/packer/helper/builder/testing"
"github.com/outscale/osc-go/oapi"
)
func TestBuilderAcc_basic(t *testing.T) {
builderT.Test(t, builderT.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Builder: &Builder{},
Template: testBuilderAccBasic,
SkipArtifactTeardown: true,
})
}
func testAccPreCheck(t *testing.T) {
}
func testOAPIConn() (*oapi.Client, error) {
access := &common.AccessConfig{RawRegion: "us-east-1"}
clientConfig, err := access.Config()
if err != nil {
return nil, err
}
skipClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
return oapi.NewClient(clientConfig, skipClient), nil
}
const testBuilderAccBasic = `
{
"builders": [
{
"type": "test",
"region": "eu-west-2",
"vm_type": "t2.micro",
"source_omi": "ami-65efcc11",
"ssh_username": "outscale",
"bsu_volumes": [
{
"volume_type": "gp2",
"device_name": "/dev/xvdf",
"delete_on_vm_deletion": false,
"tags": {
"zpool": "data",
"Name": "Data1"
},
"volume_size": 10
},
{
"volume_type": "gp2",
"device_name": "/dev/xvdg",
"tags": {
"zpool": "data",
"Name": "Data2"
},
"delete_on_vm_deletion": false,
"volume_size": 10
},
{
"volume_size": 10,
"tags": {
"Name": "Data3",
"zpool": "data"
},
"delete_on_vm_deletion": false,
"device_name": "/dev/xvdh",
"volume_type": "gp2"
}
]
}
]
}
`

92
builder/osc/bsuvolume/builder_test.go Normal file
View file

@ -0,0 +1,92 @@
package bsuvolume
import (
"testing"
"github.com/hashicorp/packer/packer"
)
func testConfig() map[string]interface{} {
return map[string]interface{}{
"access_key": "foo",
"secret_key": "bar",
"source_omi": "foo",
"vm_type": "foo",
"region": "us-east-1",
"ssh_username": "root",
}
}
func TestBuilder_ImplementsBuilder(t *testing.T) {
var raw interface{}
raw = &Builder{}
if _, ok := raw.(packer.Builder); !ok {
t.Fatalf("Builder should be a builder")
}
}
func TestBuilder_Prepare_BadType(t *testing.T) {
b := &Builder{}
c := map[string]interface{}{
"access_key": []string{},
}
warnings, err := b.Prepare(c)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatalf("prepare should fail")
}
}
func TestBuilderPrepare_InvalidKey(t *testing.T) {
var b Builder
config := testConfig()
// Add a random key
config["i_should_not_be_valid"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}
func TestBuilderPrepare_InvalidShutdownBehavior(t *testing.T) {
var b Builder
config := testConfig()
// Test good
config["shutdown_behavior"] = "terminate"
config["skip_region_validation"] = true
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test good
config["shutdown_behavior"] = "stop"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err != nil {
t.Fatalf("should not have error: %s", err)
}
// Test bad
config["shutdown_behavior"] = "foobar"
warnings, err = b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
if err == nil {
t.Fatal("should have error")
}
}

Some files were not shown because too many files have changed in this diff Show more