* fix: Fail apply command if the plan file was generated for a workspace that isn't the selected workspace.
* Add change file
* test: Update test helper to include Workspace name in plan representation
* fix: Make error message more generic, so is applicable to backend and cloud blocks.
* fix: Make error message specific to backend or cloud block
* test: Add separate tests for backend/cloud usage
* test: Update remaining tests to include a value for Workspace in mocked plans
* Apply suggestions from code review
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* fix: Panic when a plan file has missing workspace data
* test: Update test to match changes in error text
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* refactor: Update Plan struct's Backend field to be a pointer
* fix: Add some refactorings or protections against nil pointers when accessing the Backend field on a Plan struct
* refactor: Update Plan struct's StateStore field to be a pointer, update a nil check in calling code.
* Minor fixes in diagnostics
This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places.
* Rename test to make it specific to use of backend block in config
* Update initBackend to accept whole initArgs collection
* Only process --backend-config data, when setting up a `backend`, if that data isn't empty
* Simplify how mock provider factories are made in tests
* Update mock provider's default logic to track and manage existing workspaces
* Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data.
* Add function for converting a providerreqs.Version to a hashicorp/go-version Version.
This is needed for using locks when creating the backend state file.
* Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist.
* Update test fixtures to match the hashicorp/test mock provider used in PSS tests
* Allow tests to obtain locks that include `testingOverrides` providers.
The `testingOverrides` field will only be set in tests, so this should not impact end users.
* Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests.
* Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests.
* Allow reattached providers to be used during init for PSS
* Rename variable to `backendHash` so relation to `backend` is clearer
* Allow `(m *Meta) Backend` to return warning diagnostics
* Protect against nil testingOverrides in providerFactoriesFromLocks
* Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist.
The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`.
* Address code consistency check failure on PR
* Refactor use of mock in test that's experiencing EOF error...
* Remove test that requires test to supply input for user prompt
This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input.
* Allow -create-default-workspace to be used regardless of whether input is enabled or disabled
* Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable
* Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin.
Note: We cannot call t.Fatal from a non-test goroutine.
* Use Errorf instead
* Allow backend state files to not include version data when a builtin or reattached provider is in use.
* Add clarifying comment about re-attached providers when finding the matching entry in required_providers
* Report that the default workspace was created to the view
* Refactor: use error comparison via `errors.Is` to identify when no workspaces exist.
* Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func.
* Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use.
* Slight rewording of output message about default workspace
* Update test to assert new output about default workspace
* Replace use of `testChdir` with `t.Chdir`
* Update tests to use temporary directories with copied content, instead of using directories in the repo directly.
* Remove stacks copy of testChdir function
This has been replaced with t.Chdir from the standard library
* Replace remaining simple usage of testChdir
* Update guidance for using `tempWorkingDir`
* Replace use of testChdir in a function reused in a single test
* Update comments to no longer recommend using testChdir
* Remove testChdir function!
* Split code for backend state file vs backend state
* Rename BackendState to BackendConfigState
* Spelling error
* Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface.
* Split tests for backend state file vs backend config state structs
* Rename StateStorageConfigState to StateStoreConfigState
* Clarify test name, add comments
* Add tests for StateStoreConfigState methods
* Add test showing state_store in JSON is parsed correctly
* Add detection of malformed backend state files that contain both backend and state_store fields
* Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks
* Rename `state_storage` to `state_store`
* Rename `state_storage` to `state_store` in filenames
* Move`ConfigState` to its own file
* Fix test name, remove whitespace
* Update `StateStoreConfigState` comment using review suggestion
* Update error message to no longer allude to the environment TF is being run in
* Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data
* Update test helper so it doesn't accidentally supply validation in tests
* Add protection against saving an empty backend state file
* Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods
* Add Validate method to StateStoreConfigState, use in backend state encoding logic
* Refactor to use new features in registry dependency
Historically (before there were "backends") Terraform had a single state
file format used both for real state snapshots and for tracking where
remote state was stored.
Terraform v0.12 caused these two to be split because we adopted state
snapshot version 4 for real snapshots but retained a subset of version 3
for tracking the remote backend configuration in the local working
directory.
Unfortunately we previously kept that working by retaining a snapshot of
the entire Terraform v0.11 "terraform" package as legacy/terraform, which
happened to still be around because we also needed to retain a copy of
the entire legacy SDK to keep the remote state backends working.
This now hoists just the tiny slice of legacy terraform package
functionality needed to implement the "backend state" file format into
package workdir. This package is a good home for it because it's part of
the working directory state. Ideally it would be accessed through methods
of the workdir.Dir type, but that's too disruptive a refactor to combine
into this and so that'll need to wait for another day; for now we'll keep
the existing callers doing their access through our "clistate" package
that is itself a forked snapshot of what statemgr.Filesystem used to be
in Terraform v0.11.
This removes all but one of the uses of "legacy/terraform" aside from calls
in the other packages under "legacy". We'll clean up the last one in a
later commit, because it's not related to the backend state file format.
These ideas are both already implied by some logic elsewhere in the system,
but until now we didn't have the decision logic centralized in a single
place that could therefore evolve over time without necessarily always
updating every caller together.
We'll now have the modules runtime produce its own boolean ruling about
each characteristic, which callers can rely on for the mechanical
decision-making of whether to offer the user an "approve" prompt, and
whether to remind the user after apply that it was an incomplete plan
that will probably therefore need at least one more plan/apply round to
converge.
The "Applyable" flag directly replaces the previous method Plan.CanApply,
with equivalent logic. Making this a field instead of a method means that
we can freeze it as part of a saved plan, rather than recalculating it
when we reload the plan, and we can export the field value in our export
formats like JSON while ensuring it'll always be consistent with what
Terraform is using internally.
Callers can (and should) still use other context in the plan to return
more tailored messages for specific situations they already know about
that might be useful to users, but with these flags as a baseline callers
can now just fall back to a generic presentation when encountering a
situation they don't yet understand, rather than making the wrong decision
and causing something strange to happen. That is: a lack of awareness of
a new rule will now cause just a generic message in the UI, rather than
incorrect behavior.
This commit mostly just deals with populating the flags, and then all of
the direct consequences of that on our various tests. Further changes to
actually make use of these flags elsewhere in the system will follow in
later commits, both in this repository and in other repositories.
For a very long time we've had an annoying discrepancy between the
in-memory state model and our state snapshot format where the in-memory
format stores output values for all modules whereas the snapshot format
only tracks the root module output values because those are all we
actually need to preserve between runs.
That design wart was a result of us using the state both as an internal
and an external artifact, due to having nowhere else to store the
transient values of non-root module output values while Terraform Core
does its work.
We now have namedvals.State to internally track all of the throwaway
results from named values that don't need to persist between runs, so now
we'll use that for our internal work instead and reserve the states.State
model only for the data that we will preserve between runs in state
snapshots.
The namedvals internal model isn't really designed to support enumerating
all of the output values for a particular module call, but our expression
evaluator currently depends on being able to do that and so we have a
temporary inefficient implementation of that which just scans the entire
table of values as a stopgap just to avoid this commit growing even larger
than it already is. In a future commit we'll rework the evaluator to
support the PartialEval mode and at the same time move the responsiblity
for enumerating all of the output values into the evaluator itself, since
it should be able to determine what it's expecting by analyzing the
configuration rather than just by trusting that earlier evaluation has
completed correctly.
Because our legacy state string serialization previously included output
values for all modules, some of our context tests were accidentally
depending on the implementation detail of how those got stored internally.
Those tests are updated here to test only the data that is a real part
of Terraform Core's result, by ensuring that the relevant data appears
somewhere either in a root output value or in a resource attribute.
This is a mostly mechanical refactor with a handful of changes which
are necessary due to the semantic difference between earlyconfig and
configs.
When parsing root and descendant modules in the module installer, we now
check the core version requirements inline. If the Terraform version is
incompatible, we drop any other module loader diagnostics. This ensures
that future language additions don't clutter the output and confuse the
user.
We also add two new checks during the module load process:
* Don't try to load a module with a `nil` source address. This is a
necessary change due to the move away from earlyconfig.
* Don't try to load a module with a blank name (i.e. `module ""`).
Because our module loading manifest uses the stringified module path
as its map key, this causes a collision with the root module, and a
later panic. This is the bug which triggered this refactor in the
first place.
Go 1.19's "fmt" has some awareness of the new doc comment formatting
conventions and adjusts the presentation of the source comments to make
it clearer how godoc would interpret them. Therefore this commit includes
various updates made by "go fmt" to acheve that.
In line with our usual convention that we make stylistic/grammar/spelling
tweaks typically only when we're "in the area" changing something else
anyway, I also took this opportunity to review most of the comments that
this updated to see if there were any other opportunities to improve them.
* Add golden JSON test for Terraform plan
* Add data source to golden JSON plan
* Move output comparison code into shared helper function
* Add note for maintainer to contact TFC when UI changes
UI changes may potentially impact the behavior of structured run output
on TFC.
* Add test_data_source to other mock providers
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Earlier work to make "terraform init" interruptible made the getproviders
package context-aware in order to allow provider installation to be cancelled.
Here we make a similar change for module installation, which is now also
cancellable with SIGINT. This involves plumbing context through initwd and
getmodules. Functions which can make network requests now include a context
parameter whose cancellation cancels those requests.
Since the module installation code is shared, "terraform get" is now
also interruptible during module installation.
We introduced this experiment to gather feedback, and the feedback we saw
led to us deciding to do another round of design work before we move
forward with something to meet this use-case.
In addition to being experimental, this has only been included in alpha
releases so far, and so on both counts it is not protected by the
Terraform v1.0 Compatibility Promises.
In historical versions of Terraform the responsibility to check this was
inside the terraform.NewContext function, along with various other
assorted concerns that made that function particularly complicated.
More recently, we reduced the responsibility of the "terraform" package
only to instantiating particular named plugins, assuming that its caller
is responsible for selecting appropriate versions of any providers that
_are_ external. However, until this commit we were just assuming that
"terraform init" had correctly selected appropriate plugins and recorded
them in the lock file, and so nothing was dealing with the problem of
ensuring that there haven't been any changes to the lock file or config
since the most recent "terraform init" which would cause us to need to
re-evaluate those decisions.
Part of the game here is to slightly extend the role of the dependency
locks object to also carry information about a subset of provider
addresses whose lock entries we're intentionally disregarding as part of
the various little edge-case features we have for overridding providers:
dev_overrides, "unmanaged providers", and the testing overrides in our
own unit tests. This is an in-memory-only annotation, never included in
the serialized plan files on disk.
I had originally intended to create a new package to encapsulate all of
this plugin-selection logic, including both the version constraint
checking here and also the handling of the provider factory functions, but
as an interim step I've just made version constraint consistency checks
the responsibility of the backend/local package, which means that we'll
always catch problems as part of preparing for local operations, while
not imposing these additional checks on commands that _don't_ run local
operations, such as "terraform apply" when in remote operations mode.
Previously the planfile.Create function had accumulated probably already
too many positional arguments, and I'm intending to add another one in
a subsequent commit and so this is preparation to make the callsites more
readable (subjectively) and make it clearer how we can extend this
function's arguments to include further components in a plan file.
There's no difference in observable functionality here. This is just
passing the same set of arguments in a slightly different way.
Remove answers from testInputResponse as they are given, and raise an
error during cleanup if any answers remain unused.
This enables tests to ensure that the expected mock answers are actually
used in a test; previously, an entire branch of code including an input
sequence could be omitted and the test(s) would not fail.
The only test that had unused answers in this map is one leftover from
legacy state migrations, a prompt that was removed in
7c93b2e5e6
Thus far our various interactions with the bits of state we keep
associated with a working directory have all been implemented directly
inside the "command" package -- often in the huge command.Meta type -- and
not managed collectively via a single component.
There's too many little codepaths reading and writing from the working
directory and data directory to refactor it all in one step, but this is
an attempt at a first step towards a future where everything that reads
and writes from the current working directory would do so via an object
that encapsulates the implementation details and offers a high-level API
to read and write all of these session-persistent settings.
The design here continues our gradual path towards using a dependency
injection style where "package main" is solely responsible for directly
interacting with the OS command line, the OS environment, the OS working
directory, the stdio streams, and the CLI configuration, and then
communicating the resulting information to the rest of Terraform by wiring
together objects. It seems likely that eventually we'll have enough wiring
code in package main to justify a more explicit organization of that code,
but for this commit the new "workdir.Dir" object is just wired directly in
place of its predecessors, without any significant change of code
organization at that top layer.
This first commit focuses on the main files and directories we use to
find provider plugins, because a subsequent commit will lightly reorganize
the separation of concerns for plugin launching with a similar goal of
collecting all of the relevant logic together into one spot.
* command: new command, terraform add, generates resource templates
terraform add ADDRESS generates a resource configuration template with all required (and optionally optional) attributes set to null. This can optionally also pre-populate nonsesitive attributes with values from an existing resource of the same type in state (sensitive vals will be populated with null and a comment indicating sensitivity)
* website: terraform add documentation
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
