vault/buf.yaml

140 lines
5.7 KiB
YAML
Raw Permalink Normal View History

# Copyright IBM Corp. 2016, 2025
# SPDX-License-Identifier: BUSL-1.1
version: v1
Add vault community changes for the Seal HA project. (#22515) * Seal HA: Use new SealWrappedValue type to abstract seal wrapped values Introduce SealWrappedValue to abstract seal wrapped values. Make SealWrappedValue capable of marshalling into a BlobInfo, when there is plaintext or a single encryption, or to a custom serialization consisting of a header, length and a marshalled MultiWrapValue protobuf. * Vault-13769: Support configuring and using multiple seals for unsealing * Make sealWrapBackend start using multiple seals * Make seal.Access no longer implement wrapping.Wrapper. Instead, add the Encrypt and Decrypt methods to the Access interface. * Make raft snapshot system use funcs SealWrapValue + UnsealWrapValue. Move the snapshot.Sealer implementation to the vault package to avoid circular imports. * Update sealWrapBackend to use multiple seals for encryption. Use all the encryption wrappers when storing seal wrapped values. Try do decrypt using the highest priority wrapper, but try all combinations of encrypted values and wrappers if necessary. * Allow the use of multiple seals for entropy augmentation Add seal_name variable in entropy stanza Add new MultiSourcer to accommodate the new entropy augmentation behavior. * Individually health check each wrapper, and add a sys/seal-backend-status endpoint. * Address a race, and also a failed test mock that I didn't catch * Track partial wrapping failures... ... where one or more but not all access.Encrypts fail for a given write. Note these failures by adding a time ordered UUID storage entry containing the path in a special subdirectory of root storage. Adds a callback pattern to accomplish this, with certain high value writes like initial barrier key storage not allowing a partial failure. The followup work would be to detect return to health and iterate through these storage entries, rewrapping. * Add new data structure to track seal config generation (#4492) * Add new data structure to track seal config generation * Remove import cycle * Fix undefined variable errors * update comment * Update setSeal response * Fix setSealResponse in operator_diagnose * Scope the wrapper health check locks individually (#4491) * Refactor setSeal function in server.go. (#4505) Refactor setSeal function in server.go. * Decouple CreateSecureRandomReaderFunc from seal package. Instead of using a list of seal.SealInfo structs, make CreateSecureRandomReaderFunc use a list of new EntropySourcerInfo structs. This brakes the denpency of package configutil on the seal package. * Move SealGenerationInfo tracking to the seal Access. * Move SealGenerationInfo tracking to the seal Access. The SealGenerationInfo is now kept track by a Seal's Access instead of by the Config object. The access implementation now records the correct generation number on seal wrapped values. * Only store and read SealGenerationInfo if VAULT_ENABLE_SEAL_HA_BETA is true. * Add MultiWrapValue protobuf message MultiWrapValue can be used to keep track of different encryptions of a value. --------- Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com> * Use generation to determine if a seal wrapped value is up-to-date. (#4542) * Add logging to seal Access implementation. * Seal HA buf format run (#4561) * Run buf format. * Add buf.lock to ensure go-kms-wrapping module is imported. * Vault-18958: Add unit tests for config checks * Add safety logic for seal configuration changes * Revert "Add safety logic for seal configuration changes" This reverts commit 7fec48035a5cf274e5a4d98901716d08d766ce90. * changes and tests for checking seal config * add ent tests * remove check for empty name and add type into test cases * add error message for empty name * fix no seals test --------- Co-authored-by: divyapola5 <divya@hashicorp.com> * Handle migrations between single-wrapper and multi-wrapper autoSeals * Extract method SetPhysicalSealConfig. * Extract function physicalSealConfig. The extracted function is the only code now reading SealConfig entries from storage. * Extract function setPhysicalSealConfig. The extracted function is the only code now writing SealConfig entries from storage (except for migration from the old recovery config path). * Move SealConfig to new file vault/seal_config.go. * Add SealConfigType quasy-enumeration. SealConfigType is to serve as the typed values for field SealConfig.Type. * Rename Seal.RecoveryType to RecoverySealConfigType. Make RecoverySealConfigType return a SealConfigType instead of a string. * Rename Seal.BarrierType to BarrierSealConfigType. Make BarrierSealConfigType return a SealConfigType. Remove seal.SealType (really a two-step rename to SealConfigType). * Add Seal methods ClearBarrierConfig and ClearRecoveryConfig. * Handle autoseal <-> multiseal migrations. While going between single-wrapper and multiple-wrapper autoseals are not migrations that require an unwrap seal (such as going from shamir to autoseal), the stored "barrier" SealConfig needs to be updated in these cases. Specifically, the value of SealConfg.Type is "multiseal" for autoSeals that have more than one wrapper; on the other hand, for autoseals with a single wrapper, SealConfig.Type is the type of the wrapper. * Remove error return value from NewAutoSeal constructor. * Automatically rewrap partially seal wrapped values on an interval * Add in rewrapping of partially wrapped values on an interval, regardless of seal health/status. * Don't set SealGenerationInfo Rewrapped flag in the partial rewrap call. * Unexport the SealGenerationInfo's Rewrapped field, add a mutex to it for thread safe access, and add accessor methods for it. * Add a success callback to the manual seal rewrap process that updates the SealGenerationInfo's rewrapped field. This is done via a callback to avoid an import cycle in the SealRewrap code. * Fix a failing seal wrap backend test which was broken by the unexporting of SealGenerationInfo's Rewrapped field. * Nil check the seal rewrap success callback before calling it. * Change SealGenerationInfo rewrapped parameter to an atomic.Bool rather than a sync.RWMutex for simplicity and performance. * Add nil check for SealAccess before updating SealGenerationInfo rewrapped status during seal rewrap call. * Update partial rewrap check interval from 10 seconds to 1 minute. * Update a reference to SealGenerationInfo Rewrapped field to use new getter method. * Fix up some data raciness in partial rewrapping. * Account for possibly nil storage entry when retrieving partially wrapped value. * Allow multi-wrapper autoSeals to include disabled seal wrappers. * Restore propagation of wrapper configuration errors by setSeal. Function setSeal is meant to propagate non KeyNotFound errors returned by calls to configutil.ConfigureWrapper. * Remove unused Access methods SetConfig and Type. * Allow multi-wrapper autoSeals to include disabled seal wrappers. Make it possible for an autoSeal that uses multiple wrappers to include disabled wrappers that can be used to decrypt entries, but are skipped for encryption. e an unwrapSeal when there are disabled seals. * Fix bug with not providing name (#4580) * add suffix to name defaults * add comment * only change name for disabled seal * Only attempt to rewrap partial values when all seals are healthy. * Only attempt to rewrap partial values when all seals are healthy. * Change logging level from info to debug for notice about rewrap skipping based on seal health. * Remove stale TODOs and commented out code. --------- Co-authored-by: rculpepper <rculpepper@hashicorp.com> Co-authored-by: Larroyo <95649169+DeLuci@users.noreply.github.com> Co-authored-by: Scott G. Miller <smiller@hashicorp.com> Co-authored-by: Divya Pola <87338962+divyapola5@users.noreply.github.com> Co-authored-by: Matt Schultz <matt.schultz@hashicorp.com> Co-authored-by: divyapola5 <divya@hashicorp.com> Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
2023-08-24 11:40:01 -04:00
breaking:
use:
- FILE
deps:
- buf.build/hashicorp/go-kms-wrapping
lint:
ignore_only:
ENUM_VALUE_PREFIX:
- sdk/helper/clientcountutil/generation/generate_data.proto
- vault/hcp_link/proto/node_status/status.proto
- vault/replication_services_ent.proto
ENUM_ZERO_VALUE_SUFFIX:
- sdk/helper/clientcountutil/generation/generate_data.proto
- vault/hcp_link/proto/node_status/status.proto
- vault/replication_services_ent.proto
FIELD_LOWER_SNAKE_CASE:
- enthelpers/wal/types_ent.proto
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/logical/identity.proto
- sdk/plugin/pb/backend.proto
- vault/hcp_link/proto/meta/meta.proto
- vault/hcp_link/proto/node_status/status.proto
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
IMPORT_USED:
- vault/hcp_link/proto/node_status/status.proto
- vault/replication_services_ent.proto
PACKAGE_DIRECTORY_MATCH:
- builtin/logical/pki/metadata.proto
- enthelpers/merkle/types_ent.proto
- enthelpers/wal/types_ent.proto
- helper/forwarding/types.proto
- helper/identity/mfa/types.proto
- helper/identity/types.proto
- helper/storagepacker/types.proto
- physical/raft/types.proto
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/database/dbplugin/v5/proto/database_ent.proto
- sdk/helper/pluginutil/multiplexing.proto
- sdk/logical/event.proto
- sdk/logical/identity.proto
- sdk/logical/observation.proto
- sdk/logical/plugin.proto
- sdk/logical/version.proto
- sdk/plugin/pb/backend.proto
- sdk/plugin/pb/system_view_service_ent.proto
- vault/activity/activity_log.proto
VAULT-42187: add a new RPC to send data protection call count billing data from standby node to active node (#12213) (#12306) * define the proto, define the RPC * create new standby and active node workers * fix bugs: set up consumption billing worker on standby nodes * add external unit tests * change type of request from int to any * remove active node worker - the existing logic already updates and flushes data to storage for active node inside updateBillingMetrics * fix things around after conflict merge * make fmt * protobuf lint * protobuf lint issue fix * implement cadence based testing of the standby worker * protobuf formatting * expand the RPC to include transform call counts, update data returned from endpoint to include transform, remove all todo comments, add tests, fix test helper methods * create oss no-op stub for the standby worker * remove unnecessary locks from util testing methods, remove unused method * Update vault/consumption_billing.go * remove unused method * feedback: remove a check on whether system view is initialized * re-add the check on initialized state of system view * access the standby state field directly instead of using a method that uses a lock causing deadlock * feedback: use require.Eventually instead of manual looping to wait for primary to update the count * feedback: simplify logic to identify initialized consumption billing worker, remove unused methods * feedback: use log.Error instead of log.Warn * Update vault/consumption_billing_util_ent.go * feedback: increase the timeout for the rpc call to 30 secs * feedback: fix comments in tests to clarify active node and standby node definitions --------- Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com> Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
2026-02-11 16:05:01 -05:00
- vault/billing/billing.proto
- sdk/helper/clientcountutil/generation/generate_data.proto
- vault/hcp_link/proto/link_control/link_control.proto
- vault/hcp_link/proto/meta/meta.proto
- vault/hcp_link/proto/node_status/status.proto
- vault/replication/replication_resolver_ent.proto
- vault/seal/multi_wrap_value.proto
- vault/tokens/token.proto
PACKAGE_SAME_DIRECTORY:
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
PACKAGE_SAME_GO_PACKAGE:
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
PACKAGE_VERSION_SUFFIX:
- builtin/logical/pki/metadata.proto
- enthelpers/merkle/types_ent.proto
- enthelpers/wal/types_ent.proto
- helper/forwarding/types.proto
- helper/identity/mfa/types.proto
- helper/identity/types.proto
- helper/storagepacker/types.proto
- physical/raft/types.proto
- sdk/database/dbplugin/database.proto
- sdk/helper/pluginutil/multiplexing.proto
- sdk/logical/event.proto
- sdk/logical/identity.proto
- sdk/logical/observation.proto
- sdk/logical/plugin.proto
- sdk/logical/version.proto
- sdk/plugin/pb/backend.proto
- sdk/plugin/pb/system_view_service_ent.proto
- vault/activity/activity_log.proto
VAULT-42187: add a new RPC to send data protection call count billing data from standby node to active node (#12213) (#12306) * define the proto, define the RPC * create new standby and active node workers * fix bugs: set up consumption billing worker on standby nodes * add external unit tests * change type of request from int to any * remove active node worker - the existing logic already updates and flushes data to storage for active node inside updateBillingMetrics * fix things around after conflict merge * make fmt * protobuf lint * protobuf lint issue fix * implement cadence based testing of the standby worker * protobuf formatting * expand the RPC to include transform call counts, update data returned from endpoint to include transform, remove all todo comments, add tests, fix test helper methods * create oss no-op stub for the standby worker * remove unnecessary locks from util testing methods, remove unused method * Update vault/consumption_billing.go * remove unused method * feedback: remove a check on whether system view is initialized * re-add the check on initialized state of system view * access the standby state field directly instead of using a method that uses a lock causing deadlock * feedback: use require.Eventually instead of manual looping to wait for primary to update the count * feedback: simplify logic to identify initialized consumption billing worker, remove unused methods * feedback: use log.Error instead of log.Warn * Update vault/consumption_billing_util_ent.go * feedback: increase the timeout for the rpc call to 30 secs * feedback: fix comments in tests to clarify active node and standby node definitions --------- Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com> Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
2026-02-11 16:05:01 -05:00
- vault/billing/billing.proto
- sdk/helper/clientcountutil/generation/generate_data.proto
- vault/hcp_link/proto/link_control/link_control.proto
- vault/hcp_link/proto/meta/meta.proto
- vault/hcp_link/proto/node_status/status.proto
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
- vault/seal/multi_wrap_value.proto
- vault/tokens/token.proto
RPC_REQUEST_RESPONSE_UNIQUE:
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/database/dbplugin/v5/proto/database_ent.proto
- sdk/plugin/pb/backend.proto
- sdk/plugin/pb/system_view_service_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
RPC_REQUEST_STANDARD_NAME:
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/database/dbplugin/v5/proto/database_ent.proto
- sdk/logical/version.proto
- sdk/plugin/pb/backend.proto
- sdk/plugin/pb/system_view_service_ent.proto
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
RPC_RESPONSE_STANDARD_NAME:
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/database/dbplugin/v5/proto/database_ent.proto
- sdk/logical/version.proto
- sdk/plugin/pb/backend.proto
- vault/hcp_link/proto/meta/meta.proto
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto
SERVICE_SUFFIX:
- sdk/database/dbplugin/database.proto
- sdk/database/dbplugin/v5/proto/database.proto
- sdk/database/dbplugin/v5/proto/database_ent.proto
- sdk/helper/pluginutil/multiplexing.proto
- sdk/logical/version.proto
- sdk/plugin/pb/backend.proto
- sdk/plugin/pb/system_view_service_ent.proto
- vault/hcp_link/proto/link_control/link_control.proto
- vault/hcp_link/proto/meta/meta.proto
- vault/replication/replication_resolver_ent.proto
- vault/replication_services_ent.proto
- vault/request_forwarding_service.proto