diff --git a/website/source/assets/images/vault-mount-filter-0.png b/website/source/assets/images/vault-mount-filter-0.png new file mode 100644 index 0000000000..bb3e753340 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-0.png differ diff --git a/website/source/assets/images/vault-mount-filter-10.png b/website/source/assets/images/vault-mount-filter-10.png new file mode 100644 index 0000000000..fec646c82b Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-10.png differ diff --git a/website/source/assets/images/vault-mount-filter-11.png b/website/source/assets/images/vault-mount-filter-11.png new file mode 100644 index 0000000000..5725405158 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-11.png differ diff --git a/website/source/assets/images/vault-mount-filter-12.png b/website/source/assets/images/vault-mount-filter-12.png new file mode 100644 index 0000000000..43f32cc2b6 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-12.png differ diff --git a/website/source/assets/images/vault-mount-filter-13.png b/website/source/assets/images/vault-mount-filter-13.png new file mode 100644 index 0000000000..a68e2c0e54 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-13.png differ diff --git a/website/source/assets/images/vault-mount-filter-2.png b/website/source/assets/images/vault-mount-filter-2.png new file mode 100644 index 0000000000..df218fffc2 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-2.png differ diff --git a/website/source/assets/images/vault-mount-filter-3.png b/website/source/assets/images/vault-mount-filter-3.png new file mode 100644 index 0000000000..063d043da5 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-3.png differ diff --git a/website/source/assets/images/vault-mount-filter-4.png b/website/source/assets/images/vault-mount-filter-4.png new file mode 100644 index 0000000000..b85878b1d7 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-4.png differ diff --git a/website/source/assets/images/vault-mount-filter-5.png b/website/source/assets/images/vault-mount-filter-5.png new file mode 100644 index 0000000000..fa4a0b7113 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-5.png differ diff --git a/website/source/assets/images/vault-mount-filter-6.png b/website/source/assets/images/vault-mount-filter-6.png new file mode 100644 index 0000000000..4f8fcbafd9 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-6.png differ diff --git a/website/source/assets/images/vault-mount-filter-7.png b/website/source/assets/images/vault-mount-filter-7.png new file mode 100644 index 0000000000..af154af462 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-7.png differ diff --git a/website/source/assets/images/vault-mount-filter-8.png b/website/source/assets/images/vault-mount-filter-8.png new file mode 100644 index 0000000000..7417b3b176 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-8.png differ diff --git a/website/source/assets/images/vault-mount-filter-9.png b/website/source/assets/images/vault-mount-filter-9.png new file mode 100644 index 0000000000..c1824d682c Binary files /dev/null and b/website/source/assets/images/vault-mount-filter-9.png differ diff --git a/website/source/assets/images/vault-mount-filter.png b/website/source/assets/images/vault-mount-filter.png new file mode 100644 index 0000000000..662a063243 Binary files /dev/null and b/website/source/assets/images/vault-mount-filter.png differ diff --git a/website/source/guides/operations/index.html.md b/website/source/guides/operations/index.html.md index a7e2d41e62..bcef88fbcf 100644 --- a/website/source/guides/operations/index.html.md +++ b/website/source/guides/operations/index.html.md @@ -33,6 +33,11 @@ walks you through the commands to activate the Vault servers in replication mode Please note that [Vault Replication](/docs/vault-enterprise/replication/index.html) is a Vault Enterprise feature. +- **[Enterprise Only]** [Mount Filter](/guides/operations/mount-filter.html) +guide demonstrates how to selectively filter out secret engines from being +replicated across clusters. This feature can help organizations to comply with +***General Data Protection Regulation (GDPR)***. + - **[Enterprise Only]** [Vault Auto-unseal using AWS Key Management Service (KMS)](/guides/operations/autounseal-aws-kms.html) guide demonstrates an example of how to use Terraform to provision an instance that utilizes an encryption key from AWS Key Management Service (KMS). diff --git a/website/source/guides/operations/mount-filter.html.md b/website/source/guides/operations/mount-filter.html.md new file mode 100644 index 0000000000..72efe50b44 --- /dev/null +++ b/website/source/guides/operations/mount-filter.html.md @@ -0,0 +1,521 @@ +--- +layout: "guides" +page_title: "Vault Mount Filter - Guides" +sidebar_current: "guides-operations-mount-filter" +description: |- + This guide demonstrates how to selectively filter out secret mounts for + Performance Replication. +--- + +# Vault Mount Filter + +~> **Enterprise Only:** Mount filter feature is a part of _Vault Enterprise Premium_. + +Mount filters are a new way of controlling which secrets are moved across +clusters and physical regions as a result of replication. With mount filters, +users can select which secret engines will be replicated as part of a +performance replication relationship. + +By default, all non-local secret engines and associated data are replicated as +part of replication. The mount filter feature allows users to whitelist or +blacklist which secret engines are replicated, thereby allowing users to further +control the movement of secrets across their infrastructure. + +![Performance Replication](/assets/images/vault-mount-filter-2.png) + + +## Reference Materials + +- Preparing for GDPR Compliance with HashiCorp Vault [webinar](https://www.hashicorp.com/resources/preparing-for-gdpr-compliance-with-hashicorp-vault) +- Preparing for GDPR Compliance with HashiCorp Vault [blog post](https://www.hashicorp.com/blog/preparing-for-gdpr-compliance-with-hashicorp-vault) +- [Create Mounts Filter (API)](/api/system/replication-performance.html#create-mounts-filter) +- [Performance Replication and Disaster Recovery (DR) Replication](/docs/enterprise/replication/index.html#performance-replication-and-disaster-recovery-dr-replication) + +## Estimated Time to Complete + +10 minutes + +## Challenge + +[**General Data Protection Regulation (GDPR)**](https://www.eugdpr.org/) is designed +to strengthen data protection and privacy for all individuals within the +European Union. It requires that personally identifiable data not be physically +transferred to locations outside the European Union unless the region or country +has an equal rigor of data protection regulation as the EU. + +Failure to abide by GDPR will result in fines as high as 20 million EUR or 4% of +the global annual revenue (whichever greater). + + +## Solution + +Leverage Vault's **mount filter** feature to abide by data movements and +sovereignty regulations while ensuring performance access across geographically +distributed regions. + +The [***Preparing for GDPR Compliance with HashiCorp +Vault***](https://www.hashicorp.com/resources/preparing-for-gdpr-compliance-with-hashicorp-vault) +webinar discusses the GDPR compliance further in details. + +[![YouTube](/assets/images/vault-mount-filter.png)](https://youtu.be/hmf6sN4W8pE) + +## Prerequisites + +This intermediate Vault operations guide assumes that you have some working +knowledge of Vault. + +You need two Vault Enterprise clusters: one representing the EU cluster, and +another representing the US cluster both backed by Consul for storage. + + +## Steps + +**Scenario:** You have a Vault cluster in EU and wish to span across the United +States by setting up a secondary cluster and enable the performance +replication. However, some data must remain in EU and should ***not*** be +replicated to the US cluster. + +![Guide Scenario](/assets/images/vault-mount-filter-0.png) + +Leverage the mount filter feature to blacklist the secrets, that are subject to +GDPR, from being replicated across the regions. + +1. [Segment GDPR and non-GDPR secret engines](#step1) +1. [Enable performance replication with mount filter](#step2) +1. [Verify the replication mount filter](#step3) +1. [Enable a local secret engine](#step4) + +~> **NOTE:** Ensure that GDPR data is segmented by secret mount and blacklist +the movement of those secret mounts to non-GDPR territories. + + +### Step 1: Segment GDPR and non-GDPR secret engines + +In the EU cluster (primary cluster), enable key/value secret engines: + +- At **`EU_GDPR_data`** for GDPR data +- At **`US_NON_GDPR_data`** for non-GDPR data localized for US + +#### CLI command + +```shell +# For GDPR data +$ vault secrets enable -path=EU_GDPR_data kv-v2 + +# For non-GDPR data accessible in US +$ vault secrets enable -path=US_NON_GDPR_data kv-v2 +``` + +#### API call using cURL + +```shell +# For GDPR data +$ curl --header "X-Vault-Token: ..." \ + --request POST \ + --data '{"type":"kv-v2"}' \ + https://eu-west-1.compute.com:8200/v1/sys/mounts/EU_GDPR_data + +# For non-GDPR data accessible in US +$ curl --header "X-Vault-Token: ..." \ + --request POST \ + --data '{"type":"kv-v2"}' \ + https://eu-west-1.compute.com:8200/v1/sys/mounts/US_NON_GDPR_data +``` + +#### Web UI + +Open a web browser and launch the Vault UI (e.g. +https://eu-west-1.compute.com:8200/ui) and then login. + +Select **Enable new engine** and enter corresponding parameter values: + +![GDPR KV](/assets/images/vault-mount-filter-3.png) + +![Non-GDPR KV](/assets/images/vault-mount-filter-4.png) + + +Click **Enable Engine** to complete. + + +### Step 2: Enable performance replication with mount filter + +#### CLI command + +1. Enable performance replication on the **primary** cluster. + + ```plaintext + $ vault write -f sys/replication/performance/primary/enable + WARNING! The following warnings were returned from Vault: + + * This cluster is being enabled as a primary for replication. Vault will be + unavailable for a brief period and will resume service shortly. + ``` + +1. Generate a secondary token. + + ```plaintext + $ vault write sys/replication/performance/primary/secondary-token id="secondary" + Key Value + --- ----- + wrapping_token: eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJhZGRyIjoiaHR0cDovLzE3Mi4zMS4yMC4xODA6ODIwMyIsImV4cCI6MTUyNTg0ODAxMywiaWF0IjoxNTI1ODQ2MjEzLCJqdGkiOiJlNTFiMjUxZi01ZTg2LTg4OWEtNGZmMy03NTQzMjRkNTdlMGQiLCJ0eXBlIjoid3JhcHBpbmcifQ.MIGGAkE2dDj3nmaoLHg7oldQ1iZPD0U8doyj3x3mQUVfTl8W99QYG8GM6VGVzhRPGvKctGriuo2oXN_8euWQb01M1y6n7gJBSu-qdXw-v2RieOyopAHls1bWhw4sO9Nlds8IDFA15vqkLXnq2g4_5lvlhxpP7B8dEOHvWXkHG4kJ_mKvrgR0dU0 + wrapping_accessor: 6ded4fb0-5e8c-2a37-1b3e-823673220348 + wrapping_token_ttl: 30m + wrapping_token_creation_time: 2018-05-09 06:10:13.437421436 +0000 UTC + wrapping_token_creation_path: sys/replication/performance/primary/secondary-token + ``` + +1. Create a **mount filter** to blacklist `EU_GDPR_data`. + + ```plaintext + $ vault write sys/replication/performance/primary/mount-filter/secondary \ + mode="blacklist" paths="EU_GDPR_data/" + ``` + +1. Enable performance replication on the **secondary** cluster. + + ```plaintext + $ vault write sys/replication/performance/secondary/enable token="..." + ``` + Where the `token` is the `wrapping_token` obtained from the primary cluster. + + !> **NOTE:** This will immediately clear all data in the secondary cluster. + +#### API call using cURL + +1. Enable performance replication on the **primary** cluster. + + ```plaintext + $ curl --header "X-Vault-Token: ..." \ + --request POST \ + --data '{"primary_cluster_addr":"https://eu-west-1.compute.com:8200"}' \ + https://eu-west-1.compute.com:8200/v1/sys/replication/performance/primary/enable + ``` + +1. Generate a secondary token. + + ```plaintext + $ curl --header "X-Vault-Token: ..." \ + --request POST \ + --data '{ "id": "secondary"}' \ + https://eu-west-1.compute.com:8200/v1/sys/replication/performance/primary/secondary-token | jq + { + "request_id": "", + "lease_id": "", + "renewable": false, + "lease_duration": 0, + "data": null, + "wrap_info": { + "token": "eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJhZGRyIjoiaHR0cDovLzEyNy4wLjAuMTo4MjAzIiwiZXhwIjoxNTI1ODI5Njc2LCJpYXQiOjE1MjU4Mjc4NzYsImp0aSI6IjAwNmVkMDdjLWQ0MzYtZWViYy01OWYwLTdiMTU0ZGFmMDNiMCIsInR5cGUiOiJ3cmFwcGluZyJ9.MIGHAkF6saWWL-oRQMJIoUnaUOHNkcoHZCBwQs6mSMjPBopMi8DkGCJGBrh4jgV2mSzwFY1r5Ne7O66HmuMpm40MsYqjAQJCANSco_Sx5q6FmQSfoY-HtsVO1_YKWF4O6B7gYCvPKYkODMIwe5orCSgmIDyXHZt-REPm0sfdk4ZNyRCIRK5hDWyQ", + "accessor": "6ea2a4e2-2926-120f-f288-c2348c78fb3e", + "ttl": 1800, + "creation_time": "2018-05-09T01:04:36.514715311Z", + "creation_path": "sys/replication/performance/primary/secondary-token" + }, + "warnings": null, + "auth": null + } + ``` + +1. Create a **mount filter** to blacklist `EU_GDPR_data`. + + ```plaintext + $ tee payload.json < **NOTE:** This will immediately clear all data in the secondary cluster. + + +#### Web UI + +1. Select **Replication** and check the **Performance** radio button. + ![Performance Replication - primary](/assets/images/vault-mount-filter-5.png) + +1. Click **Enable replication**. + +1. Select the **Secondaries** tab, and then click **Add**. + ![Performance Replication - primary](/assets/images/vault-mount-filter-6.png) + +1. Populate the **Secondary ID** field, and then select **Configure performance +mount filtering** to set your mount filter options. You can filter by +whitelisting or blacklisting. For this example, select **Blacklist**. + +1. Check **EU_GDPR_data** to prevent it from being replicated to the secondary +cluster. + ![Performance Replication - primary](/assets/images/vault-mount-filter-7.png) + +1. Click **Generate token**. + ![Performance Replication - primary](/assets/images/vault-mount-filter-8.png) + +1. Click **Copy** to copy the token. + +1. Now, launch the Vault UI for the secondary cluster (e.g. https://us-central.compute.com:8201/ui), and then click **Replication**. + +1. Check the **Performance** radio button, and then select **secondary** under the **Cluster mode**. Paste the token you copied from the primary. + ![Performance Replication - secondary](/assets/images/vault-mount-filter-9.png) + +1. Click **Enable replication**. + +
+ +~> **NOTE:** At this point, the secondary cluster must be unsealed using the +**primary cluster's unseal key**. If the secondary is in an HA cluster, ensure +that each standby is sealed and unsealed with the primary’s unseal keys. The +secondary cluster mirrors the configuration of its primary cluster's backends +such as auth methods, secret engines, audit devices, etc. It uses the primary as +the _source of truth_ and ass token requests to the primary. + + +Restart the secondary vault server (e.g. `https://us-central.compute.com:8201`) +and unseal it with the primary cluster's unseal key. + +```plaintext +$ vault operator unseal +Unseal Key (will be hidden): +``` + +The initial root token on the secondary no longer works. Use the auth methods +configured on the primary cluster to log into the secondary. + +**Example:** + +Enable and configure the userpass auth method on the **primary** cluster and +create a new username and password. + +```shell +# Enable the userpass auth method on the primary +$ vault auth enable userpass + +# Create a user with admin policy +$ vault write auth/userpass/users/james password="passw0rd" policy="admin" +``` + +-> Alternatively, you can [generate a new root token](/guides/operations/generate-root.html) +using the primary cluster's unseal key. However, it is recommended that root +tokens are only used for just enough initial setup or in emergencies. + + +Log into the **secondary** cluster using the enabled auth method. + +```plaintext +$ vault login -method=userpass username=james password="passw0rd" +``` + + +### Step 3: Verify the replication mount filter + +Once the replication completes, verify that the secrets stored in the +`EU_GDPR_data` never get replicated to the US cluster. + +#### CLI command + +On the **EU** cluster, write some secrets: + +```shell +# Write some secret at EU_GDPR_data/secret +$ vault kv put EU_GDPR_data/secret pswd="password" +Key Value +--- ----- +created_time 2018-05-10T18:00:38.912587665Z +deletion_time n/a +destroyed false +version 1 + +# Write some secret at US_NON_GDPR_data/secret +$ vault kv put US_NON_GDPR_data/secret apikey="my-api-key" +Key Value +--- ----- +created_time 2018-05-10T18:04:37.554665851Z +deletion_time n/a +destroyed false +version 1 +``` + +From the **US** cluster, read the secrets: + +```shell +# Read the secrets at EU_GDPR_data/secret +$ vault kv get EU_GDPR_data/secret +No value found at EU_GDPR_data/secret + +# Read the secrets at US_NON_GDPR_data/secret +$ vault kv get US_NON_GDPR_data/secret +====== Metadata ====== +Key Value +--- ----- +created_time 2018-05-10T18:09:07.717250408Z +deletion_time n/a +destroyed false +version 1 + +===== Data ===== +Key Value +--- ----- +apikey my-api-key +``` + + +#### API call using cURL + +On the **EU** cluster, write some secret: + +```shell +# Create the request payload +$ tee payload.json < **Create secret**: + +![Secrets](/assets/images/vault-mount-filter-12.png) + +Enter the values and click **Save**. Repeat the step to write some secrets at +the **US_NON_GDPR_data** path as well. + + +On the **US** cluster, select **US_NON_GDPR_data**. You should be able to see +the `apikey` under `US_NON_GDPR_data/secret`. + +![Secrets](/assets/images/vault-mount-filter-13.png) + +The **EU_GDPR_data** data is not replicated, so you won't be able to see the +secrets. + + +### Step 4: Enable a local secret engine + +When replication is enabled, you can mark the secrets engine local only. Local +secret engines are not replicated or removed by replication. + +Login to the **secondary** cluster and enable key/value secret engine at +`US_ONLY_data` to store secrets only valid for the US region. + +#### CLI command + +Pass the `-local` flag: + +```plaintext +$ vault secrets enable -local -path=US_ONLY_data kv-v2 +``` + +#### API call using cURL + +Pass the `local` parameter in the API request: + +```plaintext +$ tee payload.json < + +-> **NOTE:** `US_ONLY_data` only exists locally in the secondary cluster that +you won't be able to see it from the primary cluster. + + + + +## Next steps + +Read [Vault Deployment Reference +Architecture](/guides/operations/reference-architecture.html) to learn more +about the recommended deployment practices. diff --git a/website/source/guides/operations/reference-architecture.html.md b/website/source/guides/operations/reference-architecture.html.md index 09edd59cc6..02caa07527 100644 --- a/website/source/guides/operations/reference-architecture.html.md +++ b/website/source/guides/operations/reference-architecture.html.md @@ -232,6 +232,9 @@ Vault performance replication allows for secrets management across many sites. Secrets, authentication methods, authorization policies and other details are replicated to be active and available in multiple locations. +Refer to the [Vault Mount Filter](/guides/operations/mount-filter.html) guide +about filtering out secret engines from being replicated across regions. + #### Disaster Recovery Replication Vault disaster recovery replication ensures that a standby Vault cluster is kept diff --git a/website/source/layouts/guides.erb b/website/source/layouts/guides.erb index 4f5490d78a..c82727912f 100644 --- a/website/source/layouts/guides.erb +++ b/website/source/layouts/guides.erb @@ -22,6 +22,9 @@ > Replication Setup & Guidance + > + Mount Filter + > Vault Auto-unseal with AWS KMS