* Fix resource leaks in provisioners and commands
Close file handles, HTTP response bodies, and pipe ends that were
previously leaked on certain code paths:
- workspace_new.go: Close state file opened via os.Open
- file/resource_provisioner.go: Close temp file handle before returning
the file path to caller
- login.go: Move defer resp.Body.Close() before ioutil.ReadAll so the
body is closed even when ReadAll fails
- local-exec/resource_provisioner.go: Close the read end of os.Pipe
after the copy goroutine completes
---------
Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
The `-upgrade` and `-lockfile=readonly` flags are mutually exclusive, and this is now enforced by the arguments package. This refactor has also fixed a historical bug where that validation error was output twice.
The getProvidersFromConfig code now depends on the flag validation happening in the calling code. However there is another check in downstream code, in `saveDependencyLockFile` when we overwrite the dependency lock file's contents. There, an error occurs if a diff in the locks is detected during `readonly` mode. This protects against incorrect use of `getProvidersFromConfig` with unvalidated inputs, if that happens in future.
Added details about the default behaviour for the lock-file CLI flags, and made it clearer that the upgrade flag only enables upgrading the provider used for state storage.
* refactor: Replace use of prepareInstallerEvents method. This will allow finer control of callbacks when implementing security related features
* feat: Users are prompted to approve a provider used for PSS on first use, and only if downloaded via HTTP.
Prompts include signer details and key ID data.
* test: Users see "Authentication: unauthenticated" in prompt if network mirror doesn't include hashes
They'll see authentication data in all other prompt scenarios. There's no auth when using an fs mirror, but when those are in use we trust the providers already and no prompts are raised.
* refactor: Simplify how we prepare installation event callbacks by defining reused callbacks
* refactor: Remove unused parameters from `getProvidersFromState`
* refactor: Move method `ValidWorkspaceName` and related const into arguments package, update references
This function is primarily used to validate arguments, but it's used once (in (m *Meta) Workspace) in a different context, so I've left the original function signature in the command package.
* feat: Add `workspace new`-related code to arguments package
* refactor: Update `workspace new` to use the arguments package when parsing arguments
Note that this changes the format of errors returned when arg parsing fails - this is now wrapped in a diagnostic so now starts with `\nError: ` and has an extra trailing newline.
* feat: Add `workspace select`-related code to arguments package
* refactor: Update `workspace select` to use the arguments package when parsing arguments
Note that this changes the format of errors returned when arg parsing fails - this is now wrapped in a diagnostic so now starts with `\nError: ` and has an extra trailing newline.
* test: Add pre-release to available providers. Existing tests don't use it because the configuration doesn't specify the prerelease as the version constraint.
This was the gap in my understanding - prerelease providers are handled differently in the installer logic. I believe it boils down to this: 29863fd2ba/versions/set_released.go (L5-L7)
* test: Add failing tests that show expected behaviour when a pre-release of a provider is used
* fix: Ensure that any relevant version constraints from the config are available to use when installing providers present in the state
* test: Add test defining pre-release download behaviour that was present before v1.15
* test: Update test to reflect that 2 step init is no longer experimental or specific to the PSS project
We can't detect why a provider function is missing is all cases, so
get rid of the extra logic trying to do that.
We also admit here that we need to allow a provider function within a
variable validation condition, because that is not the only way these
could fail during init, and it's otherwise valid during the next phase.
We can't fully detect why a module source or version might be unknown,
so instead use a simpler error that covers all possible reasons. If we
do find a way to get more specific in the future, we can extent it then,
but the current errors were trying to be specific when they couldn't.
* fix: Limit backend validation to just the backend type.
Also, remove similar validation for state stores, which would cause a similar issue in future.
* test: Add a new test showing behavior when removed backend types are in a config.
* feat: Add `workspace show`-related code to arguments package
* refactor: Update `workspace show` to use the arguments package when parsing arguments
* refactor: Split code for workspace subcommands into separate files in arguments package
* refactor: Move code common to all workspace commands into separate file in arguments package
* feat: Add `workspace delete`-related code to arguments package
* refactor: Update `workspace delete` to use the arguments package when parsing arguments
* refactor: Add `NewMockHTTPMirrorSource` test helper. This required refactoring `NewMockHTTPMirrorSource` to pull out reused logic.
* refactor: Move structs defining expected HTTP responses from a network mirror into `getproviders` package
* feat: Add `newHTTPMirrorProviderSourceUsingTestHttpServer` test helper, for tests that need to include use of a network mirror when downloading providers.
* refactor: Split temp file creation logic out of FakeInstallablePackageMeta & FakePackageMetaViaHTTP. Handle cleanup internally.
* refactor: Use 'Must' helpers in `newMockProviderSourceUsingTestHttpServer`
* refactor: Update tests using `newMockProviderSource` to not handle 'close' callback
* test: Move existing provider upgrade test into subtest, preparing for more subtests later
* feat: `init` command blocks upgrading the PSS provider
* test: Add test showing that upgrading non-state store providers isn't blocked as long as the state store provider is pinned in required_providers.
* feat: Upgrading the state store provider is possible during init, but it requires reconfiguring the state store.
* test: Revert change to shared test fixture, update -upgrade tests to create their own main.tf files.
Errors during initializing the config loader and collecting variable
values were printed as human readable message even if `-json` was
specified. This PR fixes this behavior and turns both errors into JSON.
* test: Add a test that asserts human output from workspace commands with colour enabled or disabled
* refactor: Update `workspace list` to use cli.Ui in a Views-like way for human output. Update how output is returned by the command to use a single List method.
* refactor: Make `workspace list` command parse its arguments using the `arguments` package
* refactor: Replace use of `ModulePath` with ` c.WorkingDir.RootModuleDir()` to separate concerns
* test: Update tests that feature the `workspace list` command to include a WorkingDir value.
This is necessary after the changes in b32b60f6a7
* feat: Detect unexpected arguments and flags using the arguments package.
* refactor: Remove duplicate call to c.View.Configure, add code comments explaining code.
* fix: Make sure argument parsing errors are handled as soon as the view is usable.
* test: Update TestWorkspace_extraArgError to account for new validation via the arguments package
* fix: Update outdated copyright headers
* test: Update test name
* refactor: Reintroduce the old behaviour when unexpected positional arguments were present by using a specific ParseWorkspaceList method
* feat: Any state migration situations involving PSS are now resolved through init -reconfigure or the new state migrate command.
Also, remove methods and structs that aren't used anymore: stateStore_to_backend, backend_to_stateStore, stateStore_changed, removeLocalState, errStateStoreClearSaved
* test: Fix test log contents to reference correct test name
* fix: Make sure that errStateStoreInitDiag suggests `init` when the store is uninitialised, and suggest `state migrate` + `init -reconfigure` when a migration is needed
* fix: Make errBackendToStateStoreInitDiag suggest the correct commands for a state migration.
* fix: Standardise on "run <command>" over "use <command>" in error messages
* test: Update tests about handling changes in a state store's configuration.
We need to re-implement these tests in the context of testing a new `state migrate` command
* test: Remove test about handling provider updates in the context of PSS.
We need to re-implement this test in the context of testing a new `state migrate` command with an `-upgrade` flag. We also need to implement behaviour in `init` that blocks upgrading the PSS provider.
* test: Update test where `init` detects a `state_store` block is removed.
* test: Update test where `init` detects a migration from state_store to backend
* test: Update test where `init` detects a migration from backend to state_store
* test: Update test where `init` detects a migration from the cloud backend to state_store
* test: Update E2E test error message assertions to match recent changes
* feat: Enable use of developer override providers when using PSS
* feat: Store how a provider was supplied to Terraform in the backend state file. This allows triggering a prompt for state migration if there's a change between using a managed provider versus an unmanaged/override provider.
* test: Update all PSS-related backend state files in test fixtures to include new, necessary provider_supply_mode data.
* fix: Avoid duplicate dev_override warnings
* refactor: Change how we determine how a provider is supplied to Terraform. By using supply mode data we prevent multiple parts of the code checking if a given provider is reattached/builtin/etc...
* test: Add separate tests showing a built-in or dev override provider being used for the main workflow using PSS
* test: Add E2E tests that characterise current behaviour when how a provider is supplied is changed partway through different workflows
* fix: Tolerate missing default workspace when using PSS and init handles minor changes in state store configuration.
* feat: `init`-related code will prompt a user to migrate state if Terraform detects a swap between managed and unmanaged providers being used for PSS.
Internally, what happens is the different version data (null versus a value being present) impacts the hash of the state store. This then triggers logic where TF checks if a migration is needed or if the new hash value should be silently updated in the backend state file. We always prompt for migration.
* refactor: Add `NotManagedByTerraform` method to `ProviderSupplyMode`
* Apply suggestions from code review
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
adds json output for state show. The output is modeled after Validate's json output, which includes any diagnostics as part of the json response (instead of returning non-json errors) once the view is configured.
* test: Add test defining how workspace subcommands react to unexpected arguments.
* refactor: Update `workspace new` and `select` commands to access the config path using `c.WorkingDir.RootModuleDir()`
* refactor: Update `workspace delete` command to access the config path using `c.WorkingDir.RootModuleDir()`
This results in test failures because the delete command's implementation doesn't indirectly invoke the (Meta).fixupMissingWorkingDir method. Other workspace subcommands do invoke the fixup method via calling (Meta).Workspace, which causes (Meta).DataDir to be invoked, which invokes the fixup.
* tests: Update all workspace tests to include a `WorkingDir` value in the `Meta`.
Prior to this change some commands would fixup this missing data, and depending on whether the meta was reused or not that would impact testing other commands. Commands that tested only the delete command would fail due to a missing workspace due to this.
