vault/enos/modules/vault_verify_read_data/scripts/verify-data.sh

40 lines
798 B
Bash
Raw Normal View History

#!/usr/bin/env bash
# Copyright (c) HashiCorp, Inc.
# SPDX-License-Identifier: BUSL-1.1
set -e
function retry {
local retries=$1
shift
local count=0
until "$@"; do
exit=$?
wait=$((2 ** count))
count=$((count + 1))
if [ "$count" -lt "$retries" ]; then
sleep "$wait"
else
return "$exit"
fi
done
return 0
}
[QT-19] Enable Enos replication scenario (#17748) * Add initial replication scenario config Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add support for replication with different backend and seal types Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update Consul versions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Additional config for replicaiton scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Refactor replication modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more steps for replication Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Work in progress with unsealing followers on secondary cluster Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more replication scenario steps Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * More updates Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Working shamir scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update to unify get Vault IP module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove duplicate module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix race condition for secondary followers unseal Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Use consistent naming for module directories Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario with latest test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Verify replication with awskms Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add write and retrive data support for all scenarios * Update all scenarios to verify write and read kv data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix write and read data modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add comments explaining the module run Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address review feedback and update consul version Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address more review feedback Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove vault debug logging Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Exclude ent.fips1402 and ent.hsm.fips1402 packages from Enos test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add verification for replication connection status * Currently this verification fails on Consul due to VAULT-12332 Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add replication scenario to Enos README Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update README as per review suggesstions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * [QT-452] Add recovery keys to scenario outputs Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix replication output var Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix autopilot scenario deps and add retry for read data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>
2023-01-13 11:43:26 -05:00
fail() {
echo "$1" 1>&2
return 1
}
[QT-602] Run `proxy` and `agent` test scenarios (#23176) Update our `proxy` and `agent` scenarios to support new variants and perform baseline verification and their scenario specific verification. We integrate these updated scenarios into the pipeline by adding them to artifact samples. We've also improved the reliability of the `autopilot` and `replication` scenarios by refactoring our IP address gathering. Previously, we'd ask vault for the primary IP address and use some Terraform logic to determine followers. The leader IP address gathering script was also implicitly responsible for ensuring that a found leader was within a given group of hosts, and thus waiting for a given cluster to have a leader, and also for doing some arithmetic and outputting `replication` specific output data. We've broken these responsibilities into individual modules, improved their error messages, and fixed various races and bugs, including: * Fix a race between creating the file audit device and installing and starting vault in the `replication` scenario. * Fix how we determine our leader and follower IP addresses. We now query vault instead of a prior implementation that inferred the followers and sometimes did not allow all nodes to be an expected leader. * Fix a bug where we'd always always fail on the first wrong condition in the `vault_verify_performance_replication` module. We also performed some maintenance tasks on Enos scenarios byupdating our references from `oss` to `ce` to handle the naming and license changes. We also enabled `shellcheck` linting for enos module scripts. * Rename `oss` to `ce` for license and naming changes. * Convert template enos scripts to scripts that take environment variables. * Add `shellcheck` linting for enos module scripts. * Add additional `backend` and `seal` support to `proxy` and `agent` scenarios. * Update scenarios to include all baseline verification. * Add `proxy` and `agent` scenarios to artifact samples. * Remove IP address verification from the `vault_get_cluster_ips` modules and implement a new `vault_wait_for_leader` module. * Determine follower IP addresses by querying vault in the `vault_get_cluster_ips` module. * Move replication specific behavior out of the `vault_get_cluster_ips` module and into it's own `replication_data` module. * Extend initial version support for the `upgrade` and `autopilot` scenarios. We also discovered an issue with undo_logs that has been described in the VAULT-20259. As such, we've disabled the undo_logs check until it has been fixed. Signed-off-by: Ryan Cragun <me@ryan.ec>
2023-09-26 17:37:28 -04:00
binpath="${VAULT_INSTALL_DIR}/vault"
[QT-19] Enable Enos replication scenario (#17748) * Add initial replication scenario config Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add support for replication with different backend and seal types Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update Consul versions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Additional config for replicaiton scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Refactor replication modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more steps for replication Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Work in progress with unsealing followers on secondary cluster Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more replication scenario steps Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * More updates Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Working shamir scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update to unify get Vault IP module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove duplicate module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix race condition for secondary followers unseal Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Use consistent naming for module directories Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario with latest test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Verify replication with awskms Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add write and retrive data support for all scenarios * Update all scenarios to verify write and read kv data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix write and read data modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add comments explaining the module run Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address review feedback and update consul version Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address more review feedback Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove vault debug logging Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Exclude ent.fips1402 and ent.hsm.fips1402 packages from Enos test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add verification for replication connection status * Currently this verification fails on Consul due to VAULT-12332 Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add replication scenario to Enos README Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update README as per review suggesstions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * [QT-452] Add recovery keys to scenario outputs Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix replication output var Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix autopilot scenario deps and add retry for read data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>
2023-01-13 11:43:26 -05:00
test -x "$binpath" || fail "unable to locate vault binary at $binpath"
[QT-19] Enable Enos replication scenario (#17748) * Add initial replication scenario config Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add support for replication with different backend and seal types Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update Consul versions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Additional config for replicaiton scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Refactor replication modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more steps for replication Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Work in progress with unsealing followers on secondary cluster Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add more replication scenario steps Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * More updates Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Working shamir scenario Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update to unify get Vault IP module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove duplicate module Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix race condition for secondary followers unseal Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Use consistent naming for module directories Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update replication scenario with latest test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Verify replication with awskms Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add write and retrive data support for all scenarios * Update all scenarios to verify write and read kv data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix write and read data modules Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add comments explaining the module run Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address review feedback and update consul version Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Address more review feedback Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Remove vault debug logging Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Exclude ent.fips1402 and ent.hsm.fips1402 packages from Enos test matrix Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add verification for replication connection status * Currently this verification fails on Consul due to VAULT-12332 Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Add replication scenario to Enos README Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Update README as per review suggesstions Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * [QT-452] Add recovery keys to scenario outputs Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix replication output var Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> * Fix autopilot scenario deps and add retry for read data Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com> Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>
2023-01-13 11:43:26 -05:00
# To keep the authentication method and module verification consistent between all
# Enos scenarios we authenticate using testuser created by vault_verify_write_data module
[QT-602] Run `proxy` and `agent` test scenarios (#23176) Update our `proxy` and `agent` scenarios to support new variants and perform baseline verification and their scenario specific verification. We integrate these updated scenarios into the pipeline by adding them to artifact samples. We've also improved the reliability of the `autopilot` and `replication` scenarios by refactoring our IP address gathering. Previously, we'd ask vault for the primary IP address and use some Terraform logic to determine followers. The leader IP address gathering script was also implicitly responsible for ensuring that a found leader was within a given group of hosts, and thus waiting for a given cluster to have a leader, and also for doing some arithmetic and outputting `replication` specific output data. We've broken these responsibilities into individual modules, improved their error messages, and fixed various races and bugs, including: * Fix a race between creating the file audit device and installing and starting vault in the `replication` scenario. * Fix how we determine our leader and follower IP addresses. We now query vault instead of a prior implementation that inferred the followers and sometimes did not allow all nodes to be an expected leader. * Fix a bug where we'd always always fail on the first wrong condition in the `vault_verify_performance_replication` module. We also performed some maintenance tasks on Enos scenarios byupdating our references from `oss` to `ce` to handle the naming and license changes. We also enabled `shellcheck` linting for enos module scripts. * Rename `oss` to `ce` for license and naming changes. * Convert template enos scripts to scripts that take environment variables. * Add `shellcheck` linting for enos module scripts. * Add additional `backend` and `seal` support to `proxy` and `agent` scenarios. * Update scenarios to include all baseline verification. * Add `proxy` and `agent` scenarios to artifact samples. * Remove IP address verification from the `vault_get_cluster_ips` modules and implement a new `vault_wait_for_leader` module. * Determine follower IP addresses by querying vault in the `vault_get_cluster_ips` module. * Move replication specific behavior out of the `vault_get_cluster_ips` module and into it's own `replication_data` module. * Extend initial version support for the `upgrade` and `autopilot` scenarios. We also discovered an issue with undo_logs that has been described in the VAULT-20259. As such, we've disabled the undo_logs check until it has been fixed. Signed-off-by: Ryan Cragun <me@ryan.ec>
2023-09-26 17:37:28 -04:00
retry 5 "$binpath" login -method=userpass username=testuser password=passuser1
retry 5 "$binpath" kv get secret/test