- actions/cache => v5.0.4
Dep updates
- actions/download-artifact => v8.0.1
Support for CJK characters
- dorny/paths-filter => v4.0.1
Node 24, support for merge queues
- hashicorp/action-setup-enos => v1.52
Security release for downstream vuln
- pnpm/action-setup => v5.0.0
Node 24, support for native caching
- slackapi/slack-github-action => v3.0.1
Node 24, lots of internal dep updates, ability to run Slack commands
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
- docker/setup-buildx-action v3.12.0 => v4.0.0
Node 24 upgrade, switch to ESM, some deprecated inputs have been
removed.
- docker/build-push-action v6.19.2 => v7.0.0
Node 24 upgrade, switch to ESM, some deprecated envs have been
removed.
- actions/setup-node v6.2.0 => v6.3.0
Bug fixes, internal dep updates, support for parsing `devEngines`.
- action-setup-enos v1.50 => v1.51
Use enos 0.0.36
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Update to the latest actions. The primary motivation here is to get the
latest action-setup-enos.
- actions/cache => v5.0.3: security patches
- actions/checkout => v6.0.2: small fixes to git user-agent and tag
fetching
- hashicorp/action-setup-enos => v1.50: security patches
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
- actions/checkout -> v6.0.2: some minor changes around setting the
ACTIONS_ORCHESTRATION_ID and some fixes to `fetch-tags`.
- actions/setup-python -> v6.2.0: Node 24 compat
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* actions(runners): add backup self-hosted runner types
We've previously added backup runner types for various self-hosted
runners but were not exhaustive. This change adds at least one backup
instance type to each specified on-demand runner type.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
- actions/cache -> v5.0.2: A bugfix around not retrying cache entries on
429s.
- actions/setup-go -> v6.2.0: NodeJS bump and internal actions/cache
bump. We don't use the caching in setup-go so this ought to have no
impact for us.
- actions/setup-node -> v6.2.0: internal bump of actions/cache.
- pnpm/action-setup -> v4.2.0: Adds support for .npmrc file.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* move from yarn to pnpm for package management
* remove lodash.template patch override
* remove .yarn folder
* update GHA to use pnpm
* add @babel/plugin-proposal-decorators
* remove .yarnrc.yml
* add lock file to copywrite ignore
* add @codemirror/view as a dep for its types
* use more strict setting about peerDeps
* address some peerDep issues with ember-power-select and ember-basic-dropdown
* enable TS compilation for the kubernetes engine
* enable TS compilation in kv engine
* ignore workspace file
* use new headless mode in CI
* update enos CI scenarios
* add qs and express resolutions
* run 'pnpm up glob' and 'pnpm up js-yaml' to upgrade those packages
* run 'pnpm up preact' because posthog-js had a vulnerable install. see https://github.com/advisories/GHSA-36hm-qxxp-pg3
* add work around for browser timeout errors in test
* update other references of yarn to pnpm
Co-authored-by: Matthew Irish <39469+meirish@users.noreply.github.com>
This change does a few things that might not be obvious:
- We stop requesting the previous runner image. This will result in us
using Docker 29 instead of 28. With this comes changes in our
container build system, most notably that container images are now
exported as OCI images. Every container runtime that we support also
supports OCI images so this ought to have no meaningful impact to
downstream users. One noticeable change is that the image layers are
now compressed so the final image size on disk will be considerably
smaller than before.
- Upgrade `hashicorp/action-setup-enos` to the latest version. This is not
strictly required for this change but as we just released a new version of
the CLI it makes sense to update it here. We should also note that recently
we released a new version of `terraform-provider-enos` which contains
necessary for this change as our docker and kind resources needed to be
updated handle OCI and Docker exported images. Previously they relied on
files that existed only in Docker images.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* actions(setup-enos): update action-setup-enos to pull in enos 0.0.34 (#10561)
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Our service users now have compatible use-case's that allow us to use
the service user credentials everywhere. Drop `action-doormat` so that
our workflows execute correctly in the `hashicorp/vault` context.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* SECVULN-22299: Use Doormat GitHub Action in CI
* remove step id
* remove step id
* grab aws account id in separate step
* add oidc perms
* add perms for other workflows
* remove usages of aws login creds
* add conditions for CE vs ent
* fix lint
* test perms
* add perms
* fix metadata
* update role arn
* use ci role arn
* print secret
* try again
* try workaround
* update all arns
* remove echo step
* cleanup
* cleanup
* address feedback
* re-add perms
* use service account
* fix conflict
* address feedback
* add read permission
* use write-all
* expose role arn
Co-authored-by: Charles Nwokotubo <charles.nwokotubo@hashicorp.com>
* actions: use self-hosted runners in hashicorp/vault
While it is recommended that we use self-hosted runners for every
workflow in private and internal accounts, this change was primarily
motivated by different runner types using different cache paths. By
using the same runner type everywhere we can avoid double caches of the
internal Vault tools.
* disable the terraform wrapper in ci-bootstrap to handle updated action
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Update our pins to the latest version. Essentially all of these are
related actions needing to run on Node 24. Both our self-hosted and the
Github hosted runners that we use are all on a new enough version of
actions/runner that it shouldn't be a problem.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Various small changes and tweaks to our CI/CD workflows to allow for running CE branches in the context of `hashicorp/vault-enterprise`.
Signed-off-by: Ryan Cragun <me@ryan.ec>
In order to take advantage of enos' ability to outline scenarios and to
inventory what verification they perform we needed to retrofit all of
that information to our existing scenarios and steps.
This change introduces an initial set of descriptions and verification
declarations that we can continue to refine over time.
As doing this required that I re-read every scenanario in its entirety I
also updated and fixed a few things along the way that I noticed,
including adding a few small features to enos that we utilize to make
handling initial versions programtic between versions instead of having a
delta between our globals in each branch.
* Update autopilot and in-place upgrade initial versions
* Programatically determine which initial versions to use based on Vault
version
* Partially normalize steps between scenarios to make comparisons easier
* Update the MOTD to explain that VAULT_ADDR and VAULT_TOKEN have been
set
* Add scenario and step descriptions to scenarios
* Add initial scenario quality verification declarations to scenarios
* Unpin Terraform in scenarios as >= 1.8.4 should work fine
Update the Github Actions pins to use the next generation of actions
that are supported by CRT.
In some cases these are simply to resolve Node 16 deprecations. In
others, we can now use `action/upload-artifact@v4` and
`actions/download-artifact@v4` since the next generation of actions like
`hashicorp/actions-docker-build@v2` and
`hashicorp/actions-persist-metadata@v2` use the `v4` versions of these.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Pin to the latest actions in preparation for the migration to
`actions/upload-artifact@v4`, `actions/download-artifact@v4`, and
`hashicorp/actions-docker-build@v2` on May 6 or 7.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Add support for testing Vault Enterprise with HA seal support by adding
a new `seal_ha` scenario that configures more than one seal type for a
Vault cluster. We also extend existing scenarios to support testing
with or without the Seal HA code path enabled.
* Extract starting vault into a separate enos module to allow for better
handling of complex clusters that need to be started more than once.
* Extract seal key creation into a separate module and provide it to
target modules. This allows us to create more than one seal key and
associate it with instances. This also allows us to forego creating
keys when using shamir seals.
* [QT-615] Add support for configuring more that one seal type to
`vault_cluster` module.
* [QT-616] Add `seal_ha` scenario
* [QT-625] Add `seal_ha_beta` variant to existing scenarios to test with
both code paths.
* Unpin action-setup-terraform
* Add `kms:TagResource` to service user IAM profile
Signed-off-by: Ryan Cragun <me@ryan.ec>
Terraform 1.6.x seems to have some incompatiblity with the current
version fo enos and its usage of tfjson. Pin to 1.5.x until it has been
resolved.
```
│ Error: json: cannot unmarshal array into Go struct field rawState.checks of type tfjson.CheckResultStatic
│
```
Signed-off-by: Ryan Cragun <me@ryan.ec>
We further optimize the CI workflow for better costs and speed.
We tested the Go CI workflows across several instance classes
and update our compute choices. We achieve an average execution
speed improvement of 2-2.5 minutes per test workflow while
reducing the infrastructure cost by about 20%. We also also save
another ~2 minutes by installing `gotestsum` from the Github release
instead of downloading the Go modules and compiling it every time.
In addition to the speed improvements, we also further reduced our cache
usage by updating the `security-scan` workflow to not cache Go modules.
We also use the `cache/save` and `cache/restore` actions for timing
caches. This results is saving half as many cache results for timing
data.
*UI test results*
results for 2x runs:
* c6a.2xlarge (12m54s, 11m55s)
* c6a.4xlarge (10m47s, 11m6s)
* c6a.8xlarge (11m32s, 10m51s)
* m5.2xlarge (15m23s, 14m16s)
* m5.4xlarge (14m48s, 12m54s)
* m5.8xlarge (12m27s, 12m24s)
* m6a.2xlarge (11m55s, 12m20s)
* m6a.4xlarge (10m54s, 10m43s)
* m6a.8xlarge (10m33s, 10m51s)
Current runner:
m5.2xlarge (15m23s, 14m16s, avg 14m50s) @ 0.448/hr = $0.11
Faster candidates
* c6a.2xlarge (12m54s, 11m55s, avg 12m24s) @ 0.3816/hr = $0.078
* m6a.2xlarge (11m55s, 12m20s, avg 12m8s) @ 0.4032/hr = $0.081
* c6a.4xlarge (10m47s, 11m6s, avg 10m56s) @ 0.7632/hr = $0.139
* m6a.4xlarge (10m54s, 10m43s, avg 10m48s) @ 0.8064/hr = $0.140
Best bang for the buck for test-ui:
m6a.2xlarge, > 25% cost savings from current and we save ~2.5 minutes.
*Go test results*
During testing the external replication tests, when not broken up, will
always take the longest. Our original analysis focuses on this job.
Most other tests groups will finish ~3m faster so we'll use subtract
that time when estimating the cost for the whole job.
external replication job results:
* c6a.2xlarge (20m49s, 19m20s, avg 20m5s)
* c6a.4xlarge (19m1s, 19m38s, avg 19m20s)
* c6a.8xlarge (19m51s, 18m54s, avg 19m23s)
* m5.2xlarge (22m12s, 20m29s, avg 21m20s)
* m5.4xlarge (20m7s, 19m3s, avg 20m35s)
* m5.8xlarge (20m24s, 19m42s, avg 20m3s)
* m6a.2xlarge (21m10s, 19m37s, avg 20m23s)
* m6a.4xlarge (18m58s, 19m51s, avg 19m24s)
* m6a.8xlarge (19m27s, 18m47s, avg 19m7s)
There is little separation in time when we increase class size. In the
best case a class size increase yields about a ~5% performance increase
and doubles the cost. For test-go our best bang for the buck is
certainly going to be in the 2xlarge class.
Current runner:
m5.2xlarge (22m12s, 20m29s, avg 21m20s) @ 0.448/hr (16@avg-3m + 1@avg) = $2.35
Candidates in the same class
* c6a.2xlarge (20m49s, 19m20s, avg 20m5s) @ 0.3816/hr (16@avg-3m + 1@avg) = $1.86
* m6a.2xlarge (21m10s, 19m37s, avg 20m23s) @ 0.4032/hr (16@avg-3m + 1@avg) = $2.00
Best bang for the buck for test-go:
c6a.2xlarge: 20% cost savings and save about ~2.25 minutes.
We ran the tests with similar instances and saw similar execution times as
with test-go. Therefore we can use the same recommended instance sizes.
After breaking up test-go's external replication tests, the longest group
was shorter on average. I choose to look at group 3 as it was usually the
longest grouping:
* c6a.2xlarge: (14m51s, 14m48s)
* c6a.4xlarge: (14m14s, 14m15)
* c6a.8xlarge: (14m0s, 13m54s)
* m5.2xlarge: (15m36s, 15m35s)
* m5.4xlarge: (14m46s, 14m49s)
* m5.8xlarge: (14m25s, 14m25s)
* m6a.2xlarge: 14m51s, 14m53s)
* m6a.4xlarge: 14m16s, 14m16s)
* m6a.8xlarge: (14m2s, 13m57s)
Again, we see ~5% performance gains between the 2x and 8x instance classes
at quadruple the cost. The c6a and m6a families are almost identical, with
the c6a class being cheaper.
*Notes*
* UI and Go Test timing results: https://github.com/hashicorp/vault-enterprise/actions/runs/5556957460/jobs/10150759959
* Go Test with data race detection timing results: https://github.com/hashicorp/vault-enterprise/actions/runs/5558013192
* Go Test with replication broken up: https://github.com/hashicorp/vault-enterprise/actions/runs/5558490899
Signed-off-by: Ryan Cragun <me@ryan.ec>
In order to reliably store Go test times in the Github Actions cache we
need to reduce our cache thrashing by not using more than 10gb over all
of our caches. This change reduces our cache usage significantly by
sharing Go module cache between our Go CI workflows and our build
workflows. We lose our per-builder cache which will result in a bit of
performance hit, but we'll enable better automatic rebalancing of our CI
workflows. Overall we should see a per branch reduction in cache sizes
from ~17gb to ~850mb.
Some preliminary investigation into this new strategy:
Prior build workflow strategy on a cache miss:
Download modules: ~20s
Build Vault: ~40s
Upload cache: ~30s
Total: ~1m30s
Prior build workflow strategy on a cache hit:
Download and decompress modules and build cache: ~12s
Build Vault: ~15s
Total: ~28s
New build workflow strategy on a cache miss:
Download modules: ~20
Build Vault: ~40s
Upload cache: ~6s
Total: ~1m6s
New build workflow strategy on a cache hit:
Download and decompress modules: ~3s
Build Vault: ~40s
Total: ~43s
Expected time if we used no Go caching:
Download modules: ~20
Build Vault: ~40s
Total: ~1m
Signed-off-by: Ryan Cragun <me@ryan.ec>
Improve our build workflow execution time by using custom runners,
improved caching and conditional Web UI builds.
Runners
-------
We improve our build times[0] by using larger custom runners[1] when
building the UI and Vault.
Caching
-------
We improve Vault caching by keeping a cache for each build job. This
strategy has the following properties which should result in faster
build times when `go.sum` hasn't been changed from prior builds, or
when a pull request is retried or updated after a prior successful
build:
* Builds will restore cached Go modules and Go build cache according to
the Go version, platform, architecture, go tags, and hash of `go.sum`
that relates to each individual build workflow. This reduces the
amount of time it will take to download the cache on hits and upload
the cache on misses.
* Parallel build workflows won't clobber each others build cache. This
results in much faster compile times after cache hits because the Go
compiler can reuse the platform, architecture, and tag specific build
cache that it created on prior runs.
* Older modules and build cache will not be uploaded when creating a new
cache. This should result in lean cache sizes on an ongoing basis.
* On cache misses we will have to upload our compressed module and build
cache. This will slightly extend the build time for pull requests that
modify `go.sum`.
Web UI
------
We no longer build the web UI in every build workflow. Instead we separate
the UI building into its own workflow and cache the resulting assets.
The same UI assets are restored from cache during build worklows. This
strategy has the following properties:
* If the `ui` directory has not changed from prior builds we'll restore
`http/web_ui` from cache and skip building the UI for no reason.
* We continue to use the built-in `yarn` caching functionality in
`action/setup-node`. The default mode saves the `yarn` global cache.
to improve UI build times if the cache has not been modified.
Changes
-------
* Add per platform/archicture Go module and build caching
* Move UI building into a separate job and cache the result
* Restore UI cache during build
* Pin workflows
Notes
-----
[0] https://hashicorp.atlassian.net/browse/QT-578
[1] https://github.com/hashicorp/vault/actions/runs/5415830307/jobs/9844829929
Signed-off-by: Ryan Cragun <me@ryan.ec>
* updates github workflows to read node version from .nvmrc file
* updates to double quotes for shell expression
* removes set-output workflow command
* updates to use node-version-file option for gh workflows
* pins node version to 16
