* Update protoc version in downloader script
* go get google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1
This matched terraform-plugin-go
* make protobuf
* Run `make protobuf`
* Update generator to handle generic types from google.golang.org/grpc
Looks like this was added in v1.69.3 in https://github.com/grpc/grpc-go/pull/7057 ?
* Run `make generate`
* Fix "cannot infer Res" compile error - more usage of generics
* More fixing compile errors due to switching to use of a generic
* Make putting `google.golang.org/grpc` import into generated files conditional
* Run `make generate`
* Update more places where generics now need to be used
* Update generator to handle any types from google.golang.org/grpc in same switch case.
* go get github.com/hashicorp/go-azure-helpers@v0.72.0
* .go-version to 1.24.1
* Run `./scripts/syncdeps.sh`
This propagates the Go version upgrade
* gcs : go get google.golang.org/api@v0.155.0
* gcs: go mod tidy
* Run `./scripts/syncdeps.sh`
* make generate && make protobuf
* stacks: support sensitive input values in planned changes
* stacks: Fix panic when applying sensitive inputs
The stacks-specific check that a given plan's input variable values
remain unchanged at apply time would previously panic when given a
marked value. Marks are not important for this check, so we simply
remove them to ensure that it passes. This seems in line with how
AssertObjectCompatible works, which I'm taking as precedent.
The new test added in this commit panics without the corresponding code
change.
---------
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
* stacks: fix bug preventing cross-provider move refactorings
* also make provider functions work
* stacks: add support for provider functions in .tfstack.hcl files
Previously we had the entire raw prior state serialized as part of the
"plan header" message in the raw plan, which meant that the maximum state
size was constrained by the maximum allowed gRPC message size.
Instead now we'll use a separate raw plan element for each raw prior state
element, so that we're limited only in the size of individual items rather
than size of the state as a whole.
This deals with the last remaining (non-deprecated) case where our RPC
protocol tries to pack an entire raw state or plan into a single protobuf
message, and so we're now standardized on using a streaming approach in
all cases.
When the topmost stack configuration declares an ephemeral input variable,
its values must be provided separately for each of the plan and apply
phases.
Therefore here we extend the API to allow specifying input variable values
during the apply phase, and add rules to check whether all of the
apply-time-required input variables have been specified and whether any
non-ephemeral variables are either unspecified or re-specified with equal
values during the apply phase.
This also extends the FindStackConfigurationComponents response to include
more metadata about the input variables and output values so that a caller
can know which ones are ephemeral. The name of that RPC function had
already become a little too specific with the inclusion of embedded stack
information and is even moreso now; we might choose to rename it to
something more generic like "AnalyzeStackConfiguration" in future, but
that'd be a breaking change and therefore requires more coordination.
* stacks: emit events for deferred actions
* deferral allowed is always on
* Update internal/rpcapi/stacks.go
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
---------
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
* stacks: add deferred resource instance planned change to protobuf
* stacks: add deferred resource instance to stack plan sequence
* stacks: add planned change for deferred actions
* stacks: refactor planned change resource instance planned
moving the components out of the main function definition so that we can reuse the implementation for deferred resource instances which wraps the message used for PlannedChangeResourceInstancePlanned
* stacks: track deferred changes in stackplan
* add simple tests
* fix tests
* address comments
---------
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
Many of the changes between these two versions seem to be in parts of the
library that Terraform doesn't use, but there are a few sensitive parts:
- Slight changes to details of how the library implements the JSON
encoding of protobuf. Terraform CLI/Core don't use this, but some
of the backends might use it indirectly when talking to their
underlying APIs.
- Some details of the "protoreflect" package for working with
protobuf descriptors dynamically (rather than using generated stubs)
have changed, but it seems pretty marginal and unlikely to cause
problems.
Components expect that check results will be round-tripped
through the plan in order to update unknown check results
during the apply. We hadn't wired this up for stack plans,
resulting in panics when the apply tries to update a check
status that doesn't exist.
These ideas are both already implied by some logic elsewhere in the system,
but until now we didn't have the decision logic centralized in a single
place that could therefore evolve over time without necessarily always
updating every caller together.
We'll now have the modules runtime produce its own boolean ruling about
each characteristic, which callers can rely on for the mechanical
decision-making of whether to offer the user an "approve" prompt, and
whether to remind the user after apply that it was an incomplete plan
that will probably therefore need at least one more plan/apply round to
converge.
The "Applyable" flag directly replaces the previous method Plan.CanApply,
with equivalent logic. Making this a field instead of a method means that
we can freeze it as part of a saved plan, rather than recalculating it
when we reload the plan, and we can export the field value in our export
formats like JSON while ensuring it'll always be consistent with what
Terraform is using internally.
Callers can (and should) still use other context in the plan to return
more tailored messages for specific situations they already know about
that might be useful to users, but with these flags as a baseline callers
can now just fall back to a generic presentation when encountering a
situation they don't yet understand, rather than making the wrong decision
and causing something strange to happen. That is: a lack of awareness of
a new rule will now cause just a generic message in the UI, rather than
incorrect behavior.
This commit mostly just deals with populating the flags, and then all of
the direct consequences of that on our various tests. Further changes to
actually make use of these flags elsewhere in the system will follow in
later commits, both in this repository and in other repositories.
Previously we were directly using the planproto.DynamicValue message type,
but that's symmetrical with plans.DynamicValue in that it encodes only
the value itself and not associated metadata such as the paths that have
sensitive value marks.
This new tfstackdata1.DynamicValue therefore echoes
terraform1.DynamicValue by carrying both the value and the metadata
together.
This implies a slight change to one of the existing message types that
was encoding a map of planned input values for each component instance,
but in practice we're not currently making use of that data anyway -- it's
there to enable a hypothetical extra correctness check in the stacks
runtime so that we can detect inconsistency problems closer to their
source -- and so this change doesn't affect anything in practice. Before
we make use of it we'll probably want to change the internal API a little
so that we can preserve the sensitive value metadata on those values, but
that's beyond the scope of this PR.
The main point of adding this is to track the output values for a component
instance as part of the raw state, and so the new field for that is also
added here although nothing will read or write it yet. Actual use of that
new field will follow in future commits.
Previously we were still partially relying on the record of this in the
embedded planproto object, but since we're now allowing "planned changes"
that don't actually include a change in that traditional sense we need to
track the provider configuration address separately as a top-level field.
As with some of the addresses before, this means we're now storing the
provider config address redundantly in two places when there _is_ a
planproto change. Hopefully we'll improve that someday, but we don't need
to sweat it too much right now because this only affects the internal
raw plan format that is not subject to any compatibility promises between
releases, so we can freely change it in any future version.
Previously our raw plan data structure only included information about
resource instances that have planned actions, which meant we were missing
those which only get their state updated during refresh and don't actually
need any real work done.
Now we'll track everything, accepting that some "planned changes" for
resource instance objects will not actually include a planned change in
the typical sense, and instead only represent a state update.
We need to retain the prior state (in a form that Terraform Core's modules
runtime can accept) between the plan and apply phases because Terraform
uses it to verify that the provider's "final plan" is consistent with
the initial plan.
We previously tried to do this by reusing the "old" value from the planned
change, but that's not really possible in practice because of the
historical implementation detail that Terraform wants state information
in a JSON format and plan information in MessagePack.
This also contains the beginnings of handling the "discarding" of
unrecognized keys from the state data structures, although we'll probably
need to do more in that area later since this is just the bare minimum.
We previously took a few shortcuts just to more quickly produce a proof
of concept. This gets us considerably closer to a properly-working handoff
from plan to apply, at least for create, update, and replace actions.
Destroying stuff that's been removed from the configuration will take some
further work since currently everything we do is driven by objects in the
configuration. That will follow in later commits.
This gets us ever closer to being able to preserve resource instance
objects from one run to the next. In subsequent commits we'll make use of
the "LoadFromProto" option to load the prior state during the planning
phase and take it into account when we're deciding what actions to
propose.
Previously we just had a stub that only generated basic external
description objects. Now we'll also emit "raw state" objects for these,
and have some initial support for deposed objects too.
We need to preserve the input variable values provided during the plan
phase so we can reuse them during the apply phase to recalculate derived
results based on new information learned during the apply process.
We also need to recalculate the input values for individual components
during the apply phase, so we can react to previously-unknown values that
come from other upstream components.
This represents our _internal_ plan and state formats, which are private
to Terraform Core so that we can evolve them over time without needing to
coordinate with other systems.
The plan parts of this are the stack-specific container messages around
the existing "tfplan" proto schema that traditional Terraform uses for
saved plan files.
The state parts of this are really just stubs since we're not really
dealing with state just yet, but they're here to illustrate where we
intend to expand in future commits when we start implementing the apply
phase and the ability to plan based on a prior state.
