* add tests that highlight known issues in the destroy mechanism
* separate refresh during destroy plans
* use the refresh outputs during destroy plans
* copywrite headers
* stacks: support sensitive input values in planned changes
* stacks: Fix panic when applying sensitive inputs
The stacks-specific check that a given plan's input variable values
remain unchanged at apply time would previously panic when given a
marked value. Marks are not important for this check, so we simply
remove them to ensure that it passes. This seems in line with how
AssertObjectCompatible works, which I'm taking as precedent.
The new test added in this commit panics without the corresponding code
change.
---------
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
We should not need to encode and decode change values within core, since
the encoded version is only technically needed for serialization. This
pattern stems from the conversion to current changes system, but back
then we did not have easy access to the correct schemas at the time to
encode and decode the entire set of changes.
Moving the core handling of changes to only use the decoded values will
drastically improve evaluation efficiency, removing a round trip
through encoded values for every resource reference.
* stacks: fix bug preventing cross-provider move refactorings
* also make provider functions work
* stacks: add support for provider functions in .tfstack.hcl files
Previously we had the entire raw prior state serialized as part of the
"plan header" message in the raw plan, which meant that the maximum state
size was constrained by the maximum allowed gRPC message size.
Instead now we'll use a separate raw plan element for each raw prior state
element, so that we're limited only in the size of individual items rather
than size of the state as a whole.
This deals with the last remaining (non-deprecated) case where our RPC
protocol tries to pack an entire raw state or plan into a single protobuf
message, and so we're now standardized on using a streaming approach in
all cases.
Previously we expected clients to provide an inline raw prior state to
PlanStackChanges and an inline raw plan to ApplyStackChanges, which was
a simpler design but meant that we might end up generating a state or plan
that's too large to be submitted in a single gRPC request, which would then
be difficult to resolve.
Instead we'll offer separate RPC functions for loading raw state and plan
using a gRPC streaming approach, which better mirrors the streaming
approach we use to _emit_ these artifacts. Although we don't actually need
this benefit right now, this makes it possible in principle for a client
that's running PlanStackChanges to feed back the raw planned actions
concurrently into OpenPlan and thus avoid buffering the whole plan on the
client side at all.
This required resolving the pre-existing FIXME about the inconsistency
where stackeval wants a raw plan for apply but expects the caller to
have dealt with loading the prior state for planning. Here it's resolved
in the direction of the caller (rpcapi) always being responsible for
loading both artifacts, because that means we can continue supporting the
old inline approach for a while without that complexity having to infect
the lower layers.
Ideally we should remove the legacy approach before this API becomes
constrained by compatibility promises, but I've preserved the old API
for now to give us some flexibility in when we update the existing
clients of this API to use the new approach.
Co-authored-by: Martin Atkins <mart@degeneration.co.uk>
When the topmost stack configuration declares an ephemeral input variable,
its values must be provided separately for each of the plan and apply
phases.
Therefore here we extend the API to allow specifying input variable values
during the apply phase, and add rules to check whether all of the
apply-time-required input variables have been specified and whether any
non-ephemeral variables are either unspecified or re-specified with equal
values during the apply phase.
This also extends the FindStackConfigurationComponents response to include
more metadata about the input variables and output values so that a caller
can know which ones are ephemeral. The name of that RPC function had
already become a little too specific with the inclusion of embedded stack
information and is even moreso now; we might choose to rename it to
something more generic like "AnalyzeStackConfiguration" in future, but
that'd be a breaking change and therefore requires more coordination.
* stacks: emit events for deferred actions
* deferral allowed is always on
* Update internal/rpcapi/stacks.go
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
---------
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
* stacks: add deferred resource instance planned change to protobuf
* stacks: add deferred resource instance to stack plan sequence
* stacks: add planned change for deferred actions
* stacks: refactor planned change resource instance planned
moving the components out of the main function definition so that we can reuse the implementation for deferred resource instances which wraps the message used for PlannedChangeResourceInstancePlanned
* stacks: track deferred changes in stackplan
* add simple tests
* fix tests
* address comments
---------
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
In the very first implementation of "sensitive values" we were
unfortunately not disciplined about separating the idea of "marked value"
from the idea of "sensitive value" (where the latter is a subset of the
former). The first implementation just assumed that any marking whatsoever
meant "sensitive".
We later improved that by adding the marks package and the marks.Sensitive
value to standardize on the representation of "sensitive value" as being
a value marked with _that specific mark_.
However, we did not perform a thorough review of all of the mark-handling
codepaths to make sure they all agreed on that definition. In particular,
the state and plan models were both designed as if they supported arbitrary
marks but then in practice marks other than marks.Sensitive would be
handled in various inconsistent ways: dropped entirely, or interpreted as
if marks.Sensitive, and possibly do so inconsistently when a value is
used only in memory vs. round-tripped through a wire/file format.
The goal of this commit is to resolve those oddities so that there are now
two possible situations:
- General mark handling: some codepaths genuinely handle marks
generically, by transporting them from input value to output value in
a way consistent with how cty itself deals with marks. This is the
ideal case because it means we can add new marks in future and assume
these codepaths will handle them correctly without any further
modifications.
- Sensitive-only mark preservation: the codepaths that interact with our
wire protocols and file formats typically have only specialized support
for sensitive values in particular, and lack support for any other
marks. Those codepaths are now subject to a new rule where they must
return an error if asked to deal with any other mark, so that if we
introduce new marks in future we'll be forced either to define how we'll
avoid those markings reaching the file/wire formats or extend the
file/wire formats to support the new marks.
Some new helper functions in package marks are intended to standardize how
we deal with the "sensitive values only" situations, in the hope that
this will make it easier to keep things consistent as the codebase evolves
in future.
In practice the modules runtime only ever uses marks.Sensitive as a mark
today, so all of these checks are effectively covering "should never
happen" cases. The only other mark Terraform uses is an implementation
detail of "terraform console" and does not interact with any of the
codepaths that only support sensitive values in particular.
Components expect that check results will be round-tripped
through the plan in order to update unknown check results
during the apply. We hadn't wired this up for stack plans,
resulting in panics when the apply tries to update a check
status that doesn't exist.
Due to an oversight in our handling of resource instance objects that are
neither in configuration nor plan -- which is true for data resources that
have since been removed from the configuration -- we were generating plan
change objects that were lacking a provider configuration address, which
made them syntactically invalid and thus not reloadable using the
raw plan parser.
This is a bit of a strange situation since we don't technically _need_ a
provider configuration address for these; all we're going to do is just
unceremoniously delete them from the state during apply anyway. However,
we always have the provider configuration address available anyway, so
adding this in here is overall simpler than changing the parser, the
models it populates, and all of the downstream users of those models to
treat this field as optional.
This commit is more test case than it is fix, since the fix was relatively
straightforward once I had a test case to reproduce the problem it's
fixing.
These ideas are both already implied by some logic elsewhere in the system,
but until now we didn't have the decision logic centralized in a single
place that could therefore evolve over time without necessarily always
updating every caller together.
We'll now have the modules runtime produce its own boolean ruling about
each characteristic, which callers can rely on for the mechanical
decision-making of whether to offer the user an "approve" prompt, and
whether to remind the user after apply that it was an incomplete plan
that will probably therefore need at least one more plan/apply round to
converge.
The "Applyable" flag directly replaces the previous method Plan.CanApply,
with equivalent logic. Making this a field instead of a method means that
we can freeze it as part of a saved plan, rather than recalculating it
when we reload the plan, and we can export the field value in our export
formats like JSON while ensuring it'll always be consistent with what
Terraform is using internally.
Callers can (and should) still use other context in the plan to return
more tailored messages for specific situations they already know about
that might be useful to users, but with these flags as a baseline callers
can now just fall back to a generic presentation when encountering a
situation they don't yet understand, rather than making the wrong decision
and causing something strange to happen. That is: a lack of awareness of
a new rule will now cause just a generic message in the UI, rather than
incorrect behavior.
This commit mostly just deals with populating the flags, and then all of
the direct consequences of that on our various tests. Further changes to
actually make use of these flags elsewhere in the system will follow in
later commits, both in this repository and in other repositories.
Components can emit sensitive values as outputs, which can be consumed
as inputs to other components. This commit ensures that such values are
correctly processed in order to pass their sensitivity to the modules
runtime.
