The new Setup.Stop RPC allows clients to abort long-running operations.
Each operation implementation must add itself to the server's stopper,
and listen on the given channel for a signal to abort.
In this commit, plan, apply, and validate operations translate the stop
signal into a context cancellation. This alone is not yet sufficient for
a truly graceful shutdown, so in later commits we need to adjust the
stacks runtime to instruct the modules runtime to stop on cancellation.
Adds support for including fields typically found in the Terraform CLI configuration in the RPC API handshake. This allows us to include global configuration arguments that impact the RPC API session without requiring instrumentation for each RPC service. The new Config struct currently supports only service discovery credentials, but it can be expanded in the future.
We allow experiments only in alpha builds, and so we propagate the flag
for whether that's allowed in from "package main". We previously had that
plumbed in only as far as the rpcapi startup.
This plumbs the flag all the way into package stackeval so that we can
in turn propagate it to Terraform's module config loader, which is
ultimately the one responsible for ensuring that language experiments can
be enabled only when the flag is set.
Therefore it will now be possible to opt in to language experiments in
modules that are used in stack components.
Right now we are only using rpcapi from external callers inside Terraform
Cloud, but in the long run we will also want to expose equivalent
functionality through Terraform CLI.
rpcapi.NewInternalClient provides a relatively-lightweight way to achieve
that without requiring a child process and sockets. Instead, it just
allocates a buffer and passes gRPC messages between client and server
using that buffer.
This is of course still not as efficient as a direct function call, but
the kinds of operations that RPC API exposes tend to have far more
expensive elements than the RPC call itself, and using the RPC protocol
as the internal API should begin to establish a more explicit architecture
boundary between Terraform CLI and Terraform Core, so that clients of
the RPC API have the same level of access to Terraform Core as the CLI
workflow does.
So far we don't have any internal callers for this because nothing in the
RPC API has a corresponding CLI entry point. The first callers of this
internal RPC interface will probably be the commands providing the
CLI-driven development workflow for stacks.
This implements the rpcapi operations for building and manipulating
provider plugin cache directories, which in later commits we'll be able
to pass to some of the Stacks service operations so that the stacks
runtime can work with non-builtin providers.
Previously we were trying to add new services to the gRPC server after it
was already running, which worked okay in the contrived fake setup I was
using to test it in isolation but is rejected by the real gRPC server
implementation, which requires all service registration to happen prior
to starting the server.
Instead then, we'll use some wrapper server implementations that initially
just return "Unavailable" errors for every request but can then be tardily
initialized with a real server implementation that takes over serving
requests once registered.
This involves some gnarly code generation for the wrapper server
implementations because this situation is beyond the current capabilities
of Go generics. The code generator implementation is pretty gross but this
approach was the least horrible of various other things I tried in order
to avoid writing a code generator.
The code generator may need some additional work later once we introduce
our first streaming RPC function, but after that it should hopefully
remain unchanged for a long time as long as we stick to the current API
design idiom for future services.
As with many other systems that involve a client indirectly manipulating
objects on the other side of a trust boundary, we're going to use opaque
"handles" to represent objects that remain active on the server side
between different RPC API calls.
A handle is just a 64-bit integer corresponding to an entry in a lookup
table maintained by the server, in the "handleTable" type.
In the public API we don't make any promises about uniqueness of handles
between different types of objects, but to minimize the impact of client
bugs we'll internally ensure that we never issue the same handle number
for two objects of different types. This means that a client getting their
handles mixed up will typically cause an explicit error rather than just
doing something weird with an unrelated object that happened to have a
matching handle.
Our internal API for this will use explicit separate methods for each type
of handle and use type parameters so that the Go compiler can call us out
if we're inconsistent in how we're using the handles. From a client
perspective though these all just layer to protobuf int64; clients might
choose to add their own similar abstraction to track handle types, but
that's their own business since we can't represent such things efficiently
within the protocol buffers schema model.
This commit includes initial implementations of
Dependencies.OpenSourceBundle and Dependencies.CloseSourceBundle just as
some initial evidence that this design works. We'll continue implementing
the other real service functions in later commits once the basic
infrastructure (like this) is in place.
This doesn't actually do anything useful yet, but this does at least make
the RPC server accessible to external callers where they can handshake and
then get an error saying that nothing else is implemented.
