From 9350e704c83882015376d708bbea0e1af0d2e335 Mon Sep 17 00:00:00 2001 From: Stian Thorgersen Date: Wed, 11 Feb 2026 11:12:35 +0100 Subject: [PATCH] Follow-up update to test framework docs (#46176) Closes #46073 Signed-off-by: stianst --- test-framework/docs/GITHUB.md | 14 +++ test-framework/docs/README.md | 1 + test-framework/docs/WRITING_TESTS.md | 131 +++++++++++++++++++++++++++ 3 files changed, 146 insertions(+) create mode 100644 test-framework/docs/GITHUB.md diff --git a/test-framework/docs/GITHUB.md b/test-framework/docs/GITHUB.md new file mode 100644 index 00000000000..e69666a7eed --- /dev/null +++ b/test-framework/docs/GITHUB.md @@ -0,0 +1,14 @@ +# GitHub Actions Job Summary + +The test framework has built-in support for creating GitHub Actions job summaries. It will report: + +* Failed tests +* Slow tests + +The report is enabled by default when tests are executed on GitHub Actions. If you don't want this report it can be +disabled with the `KC_TEST_GITHUB_ENABLED` environment variable. + +Slow test detection can be configured separately by setting the threshold for slow test classes and test methods. +By default, test classes that take longer than 120 seconds and test methods that take longer than 30 seconds are +reported. This can be configured with `KC_TEST_GITHUB_SLOW_CLASS` and `KC_TEST_GITHUB_SLOW_METHOD` environment +variables. \ No newline at end of file diff --git a/test-framework/docs/README.md b/test-framework/docs/README.md index 349f9826d65..8d418f1fdc0 100644 --- a/test-framework/docs/README.md +++ b/test-framework/docs/README.md @@ -15,3 +15,4 @@ Tests simply declare what they want, including specific configuration, and the f * [Using test suites](SUITES.md) - How to use tests suites with custom configuration * [Best practices](BEST_PRACTICES.md) - Best practices and common pitfalls * [Writing extensions](EXTENSIONS.md) - Writing test framework extensions +* [GitHub Actions](GITHUB.md) - Support for creating job summary on GitHub Actions \ No newline at end of file diff --git a/test-framework/docs/WRITING_TESTS.md b/test-framework/docs/WRITING_TESTS.md index 6d24b8ac4f7..fc2a81853ec 100644 --- a/test-framework/docs/WRITING_TESTS.md +++ b/test-framework/docs/WRITING_TESTS.md @@ -99,6 +99,137 @@ public class Test2 { In this example the realm from `Test1` would be destroyed and a new realm created for `Test2` since different configuration is requested. +### Injection support in config classes + +`RealmConfig`, `ClientConfig` and `UserConfig` supports injecting dependencies into the config. This can be useful +when the configuration depends on how other resources are configured. For example: + +```java +public static class MyClient implements ClientConfig { + + @InjectDependency + KeycloakUrls keycloakUrls; + + @Override + public ClientConfigBuilder configure(ClientConfigBuilder client) { + return client.redirectUris(keycloakUrls.getAdmin()); + } +} +``` + +Only dependencies (including transitive dependencies) defined by the suppliers can be injected into config classes. + +## Realm cleanup + +The test framework aims to re-use as much as possible to reduce execution time. This is especially relevant to +managed realms. By default, a managed realm has its lifecycle set to `CLASS`, which means the same realm will be +re-used for all tests methods within the same test class. + +It's also possible to change the lifecycle to `GLOBAL` where the realm will be shared for all test classes. This can +be beneficial for large and complex realms, but bear in mind that tests will need to carefully clean after +themselves. + +In the end choosing the lifecycle of the realm depends on how much (if any) cleanup tests have to perform. + +To help with cleanup `ManagedRealm` provides some convenience methods to help test clean-up after themselves. In general +the above methods should be called at the start of the test method before any changes are made. + +### `dirty()` + +If a limited number of tests require a lot of cleanup it can be expensive to do so, and result in larger and more +complex test methods. Marking the realm as dirty within a test method will cause it to be re-created after the test +method has executed: + +```java +@Test +public void testSomething() { + managedRealm.dirty(); + + // Make loads of changes to the realm +} +``` + +If most or all test methods are using `dirty()` consider using lifecycle `CLASS` instead for the managed realm. + +### `updateWithCleanup(...)` + +If a limited number of test methods require changes to the realm configuration the `updateWithCleanup(...)` method +can be used: + +```java +@Test +public void testSomethingThatRequiresRegistration() { + managedRealm.updateWithCleanup(r -> r.registrationAllowed(true)); + + // Test registration +} +``` + +The changes will then be reverted after the test method has executed. + +### `update` and `add` methods + +There are a number of utilities that allow adding or updating resources within a realm, with cleanup after the test +method has executed. This allows for example adding a user that is only required for a single test method: + +```java +@Test +public void testUser() { + managedRealm.addUser(UserConfigBuilder.create().username("myuser")); +} +``` + +There are a limit number of supported resources at the moment, and more will be added as needed, eventually +supporting the majority of resources within a realm. + +### `cleanup().add(...)` + +Adding cleanup to the realm will allow cleaning up anything within the realm: + +```java +@Test +public void testWithCleanup() { + managedRealm.cleanup().add(r -> r.roles().get("foo").remove()); +} +``` + +## Setup and Cleanup + +Typically, for a JUnit test `beforeAll` and `afterAll` are used to setup the environment for tests, but these are not +very useful when using the test framework since these need to be `static` and does not have access to any injected +resources. + +Instead, the test framework allows annotating `non-static` methods with no parameters using `@TestSetup` and +`@TestCleanup`. Methods annotated with `@TestSetup` will be executed before all tests, and methods annotated with +`@TestCleanup` after all test methods have completed. For example: + +```java +@InjectRealm(lifecycle = LifeCycle.CLASS) +ManagedRealm realm; + +@TestSetup +public void setupRealms() { + RealmRepresentation rep = realm.admin().toRepresentation(); + Assertions.assertNull(rep.getAttributes().get("test.setup")); + rep.getAttributes().put("test.setup", "myvalue"); + realm.admin().update(rep); +} + +@TestCleanup +public void cleanupRealms() { + RealmRepresentation rep = realm.admin().toRepresentation(); + Assertions.assertEquals("myvalue", rep.getAttributes().get("test.setup")); + rep.getAttributes().remove("test.setup"); + realm.admin().update(rep); +} +``` + +One thing to bear in mind when using `@TestSetup` and `@TestCleanup` is any injected resources with lifecycle `METHOD`. +As these will be re-created for each test method, any changes done in `@TestSetup` will to those resources will be +reverted after the first test has executed. + +Avoid using `@TestSetup` for anything that can be configured using `config`. + ## Multiple instances By default, all resources are granted the default reference, and other resources that depend on them don't need to