USGv6[0] requires implementing §4.1.1 of the NISTv6-r1 profile[1] for
IPv6-Only capabilities. This section requires that whenever Vault
displays IPv6 addresses (including CLI output, Web UI, logs, etc.) that
_all_ IPv6 addresses must conform to RFC-5952 §4 text representation
recommendations[2].
These recommendations do not prevent us from accepting RFC-4241[3] IPv6
addresses, however, whenever these same addresses are displayed they
must conform to the strict RFC-5952 §4 guidelines.
This PR implements handling of IPv6 address conformance in our
`vault server` routine. We handle conformance normalization for all
server, http_proxy, listener, seal, storage and telemetry
configuration where an input could contain an IPv6 address, whether
configured via an HCL file or via corresponding environment variables.
The approach I've taken is to handle conformance normalization at
parse time to ensure that all log output and subsequent usage
inside of Vaults various subsystems always reference a conformant
address, that way we don't need concern ourselves with conformance
later. This approach ought to be backwards compatible to prior loose
address configuration requirements, with the understanding that
going forward all IPv6 representation will be strict regardless of
what has been configured.
In many cases I've updated our various parser functions to call the
new `configutil.NormalizeAddr()` to apply conformance normalization.
Others required no changes because they rely on standard library URL
string output, which always displays IPv6 URLs in a conformant way.
Not included in this changes is any other vault exec mode other than
server. Client, operator commands, agent mode, proxy mode, etc. will
be included in subsequent changes if necessary.
[0]: https://www.nist.gov/publications/usgv6-profile
[1]: https://www.nist.gov/publications/nist-ipv6-profile
[2]: https://www.rfc-editor.org/rfc/rfc5952.html#section-4
[3]: https://www.rfc-editor.org/rfc/rfc4291
Signed-off-by: Ryan Cragun <me@ryan.ec>
* Fix "t.Fatal from a non-test goroutine" errors in cache_test.go
- t.Fatal(f) should not be called within a Go routine based on it's documentation and only from the main test's thread.
- In 1.24 this seems to cause build failures
* Address all "non-constant format string errors" from go vet
- Within 1.24 these now cause test builds to fail
…" from go vet
* Do not use static certificates for diagnose tests
* Fix operator command tests, move PKI CA creation code into testhelper lib
* Fix compilation error from refactoring
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Add an ENV var to disable the DRBG in a pinch
* update go.mod
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Add an ENV var to disable the DRBG in a pinch
* Use DRBG based RSA key generation everywhere
* update go.mod
* fix import
* Remove rsa2 alias, remove test code
* move cryptoutil/rsa.go to sdk
* move imports too
* remove makefile change
* rsa2->rsa
* more rsa2->rsa, remove test code
* fix some overzelous search/replace
* Update to a real tag
* changelog
* copyright
* work around copyright check
* work around copyright check pt2
* bunch of dupe imports
* missing import
* wrong license
* fix go.mod conflict
* missed a spot
* dupe import
* initial implementation of unseal trace
* close file if we fail to start the trace
didn't bother to check the error from traceFile.Close()
* use reloadable config instead of env var
* license
* remove leftover
* allow setting custom dir and remove new package
* bring back StartDebugTrace
after talking to Kuba it sounds like it's a good idea to try to move stuff out of core, so even if there's no immediate need for a generic debug trace function it's still fair to add it
* track postUnseal instead of unsealInternal
also some usability improvements from manual testing
* address PR comments
* address security review
there were concerns about using the /tmp directory because of permissions, or having a default dir at all, so now it's required to set a dir in order to generate the traces.
* add unit tests to StartDebugTrace
* move back to default dir
* document new parameters
* add tiny integration test
* avoid column in trace filename
sounds like it might be forbidden in Windows and possibly cause problems in some MacOS applications.
* address PR feedback
* add go doc to test
CI was complaining about missing comments on the new test function. It feels a bit silly to require this of tests but whatever XD
* fix tests
* Support trimming trailing slashes via a mount tuneable to support CMPv2
* changelog/
* Perform trimming in handleLoginRequest too
* Eagerly fetch the mount entry so we only test this once
* Add a mount match function that gets path and entry
* Update vault/request_handling.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* more docs
* Some patches (from ENT) didnt apply
* patch fail
* Update vault/router.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* PR feedback
* dupe
* another dupe
* Add support for enabling trim_request_trailing_slashes on mount creation
* Fix read mount api returning configuration for trim_request_trailing_slashes
* Fix test assertion
* Switch enable and tune arguments to BoolPtrVal to allow end-users to specify false flag
* Add trim-request-trailing-slashes to the auth enable API and CLI
---------
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
This PR adds the CE plumbing and stubs for forcing agent instantiation
whenever the Vault license changes.
Resolves: VAULT-28583
Enterprise PR: hashicorp/vault-enterprise#6168
* timeout 'testListenerConnFn' waiting on the server connection after 3 secs
* return the invalid upstream error so the library knows not to stop listening/serving
* update go-proxyproto to use fork/tag
* test that fails before library and code update, but passes afterwards
* VAULT-28192 fix Agent and Proxy consuming large amounts of CPU for auto-auth self-healing
* Changelog
* Update changelog
* drain incoming if we get invalid token
---------
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Add option for not mounting the KV mount in --dev
* Fix whitespace I messed up during merge conflict resolution
* Feature -> Improvement
* Fix formatting for real
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* VAULT-25848 update product code to remove mholt/archiver dependency
* VAULT-25848 replace tests, still WIP while I figure out if there's a bug caught by TestDebugCommand_PartialPermissions
* VAULT-25848 actually remove the dep
* VAULT-25848 add headers for directories, improve test
* Comment cleanup
* Typo
* Use %w
* Typo
* add reloading service configuration
* add changelog entry
* add tests
* fix typo
* check if config.ServiceRegistration is nil before signaling
* add changes for deregistering service on nil config with failing tests
* fix tests by decreasing reconcile_timeout + setting consul agent tokens
* fix races
* add comments in test
---------
Co-authored-by: Marc Boudreau <marc.boudreau@hashicorp.com>
* VAULT-25341 Address issue where having no permissions to renew caused Agent and Proxy auth to attempt to renew with no backoff
* Fiddle with go.mod changes that shouldn't have happened
* VAULT-25341 small cleanup and extra test
* VAULT-25341 backoff only in error case
* VAULT-25341 godocs
* VAULT-25342 changelog
* Update command/agent_test.go
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
* VAULT-25341 rename file audit
---------
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
* Add missing delegated_auth_accessors config field to /sys/mounts/<path> response
- The field hadn't been properly populated in the JSON struct being returned
through the API response, but had been properly set in the stored structs
in the backend.
- Add missing update to the command tune docs for the -delegated-auth-accessors
option that existed
- Add -delegated-auth-accessors to the secret enable vault command along with
a docs update
* Add cl
* Fix documentation, using a comma separated list does not work
* Apply suggestions from code review
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Drop plural on doc update
---------
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
This PR adds inmem_transactional to the map of available physical
backends for TestServerCommand. This is harmless, as tests need to opt
into the backend.
This is required to test AOP configuration on enterprise.
We have many hand-written String() methods (and similar) for enums.
These require more maintenance and are more error-prone than using
automatically generated methods. In addition, the auto-generated
versions can be more efficient.
Here, we switch to using https://github.com/loggerhead/enumer, itself
a fork of https://github.com/diegostamigni/enumer, no longer maintained,
and a fork of the mostly standard tool
https://pkg.go.dev/golang.org/x/tools/cmd/stringer.
We use this fork of enumer for Go 1.20+ compatibility and because
we require the `-transform` flag to be able to generate
constants that match our current code base.
Some enums were not targeted for this change:
The new vault.namespace config option seems to always be overridden by
the auto-auth namespace, so add an extra check to allow them to be set
separately.
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Auto Auth Healing for Proxy
* Edited changelog
* Fix failing tests and small comment change
* Readded check because proxy cache is initialized with inmem sink
* Add support for x_forwarded_for_client_cert_header
* add changelog entry
* add tests for a badly and properly formatted certs
* both conditions should be true
* handle case where r.TLS is nil
* prepend client_certs to PeerCertificates list
* Add support for x_forwarded_for_client_cert_header
* add changelog entry
* add tests for a badly and properly formatted certs
* both conditions should be true
* handle case where r.TLS is nil
* prepend client_certs to PeerCertificates list
* add option for decoders to handle different proxies
* Add support for x_forwarded_for_client_cert_header
* add changelog entry
* add tests for a badly and properly formatted certs
* both conditions should be true
* handle case where r.TLS is nil
* prepend client_certs to PeerCertificates list
* add option for decoders to handle different proxies
* fix tests
* fix typo
---------
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Improve TestAutoAuthSelfHealing_TokenFileAuth_SinkOutput to make it more robust in race test
* Tweak the sensitivity on waiting for template re-renders after triggering
Fix reloading of seal configuration when a node gains leadership.
Verify that the in-memory seal generation information is stale and only reload
seal configuration when that is the case.
When reloading seal configuration, only do it when enable_multiseal is currently
set to true, or the new configuration is attempting to set it to true.
As part of the process of becoming a leader node, check to see if the seal
configuration needs to be reloaded. Reloading may be necessary if the seal
generation information computed during start up is outdated. For example, a new
node that has just joined the cluster will have incorrect seal generation
information in memory, even if it has the correct seal configuration, since it
did not have access to the stored seal generation information.
* Add a configuration flag for enabling multiseal (Seal HA), CE side
* imports
* no quotes
* get rid of dep on ent config
* Abstract enableMultiSeal for a build time switch
* license headers
* wip
* gate physical seal gen fetch by a param
* docs tweak, remove core flag
* updates from the ent pr
* update stub
* update test fixtures for enable_multiseal
* use accessor
* add a test fixture for non-multiseal diagnose
* remove debugging crtuch
* Do handle phys seal gen info even if multiseal is off, in order to facilitate enable/disable safeties
* more enabled flag handling
* Accept seal gen info if we were previously disabled, and persist it
* update unit test
* Validation happens postUnseal, so this test is invalid
* Dont continue setting conf if seal loading fails during SIGHUP
* Update website/content/docs/configuration/seal/seal-ha.mdx
Thanks, that does sound much clearer
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* use validation if previous gen was enabled
* unit test update
* stub SetMultisealEnabled
* bring over more changes from ent
* this was an unfix
---------
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Move command/config + command/token to api/cliconfig + api/tokenhelper
* Remove unused functions and unused import
* Simplify and inline function copied from SDK
* Delete unused duplicated/forwarding config implementation from command package
* Delete unused code, unexport API surface that's only used internally to the package
* Fix up license headers
* Add changelog
* Tweak .gitignore to track hcl files in testdata/ folders
* add secret syncs to vault operator usage
* changelog
* unexport
* add godoc for test and remove t.Run invocation
* move test to separate package
* update comment
