mirror of
https://github.com/redis/redis.git
synced 2026-07-11 11:05:39 -04:00
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / test-sanitizer-address (push) Waiting to run
CI / build-debian-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-32bit (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
CI / build-centos-jemalloc (push) Waiting to run
CI / build-old-chain-jemalloc (push) Waiting to run
Codecov / code-coverage (push) Waiting to run
External Server Tests / test-external-standalone (push) Waiting to run
External Server Tests / test-external-cluster (push) Waiting to run
External Server Tests / test-external-nodebug (push) Waiting to run
Spellcheck / Spellcheck (push) Waiting to run
## Summary Redis core and the external modules (Search, JSON, Bloom, TimeSeries) are currently built and released through separate flows, forcing anyone assembling a full distribution to juggle multiple repos, refs, build commands, and config files. This PR collapses that into one Redis-owned workflow: ```sh make modules-update # clone/refresh modules from modules/modules.yaml make bootstrap # install per-module build/test deps make build # build Redis core + selected modules make run # start redis-server with modules loaded make test all # run module test suites make tarball TAG=... # build a reproducible source bundle ``` ## What This Adds **1. Manifest-driven modules** — `modules/modules.yaml` is the single source of truth for bundled modules. Each entry pins `repo`/`ref` and defines its build artifact and runtime `loadmodule` path. Refs resolve dynamically (tag → branch → SHA, else fail). Current pins: RedisBloom `v8.9.81`, RediSearch `v8.9.82`, RedisJSON `v8.9.81`, RedisTimeSeries `v8.9.82`. **2. Central manifest parser** — `scripts/lib/manifest.sh` is the only YAML reader, usable both as a shell library and a CLI helper (via `modules/manifest.mk`). **3. Per-module stub Makefiles removed** — The `modules/redis*/Makefile` stubs are deleted; their data moves to `modules.yaml` and shared behavior to `modules/common.mk`. ## Make Workflow The root `Makefile` is now a thin dispatch layer over scripts in `scripts/`: | Target | Script | Purpose | |---|---|---| | `make build [args]` | `scripts/build.sh` | Build Redis core plus selected modules | | `make bootstrap [args]` | `scripts/bootstrap.sh` | Install per-module build/test prerequisites | | `make setup [args]` | `scripts/setup.sh` | Run module update plus bootstrap | | `make run [args]` | `scripts/run.sh` | Start Redis with selected modules loaded | | `make test [args]` | `scripts/test.sh` | Run Redis or module tests | | `make clean [args]` | `scripts/clean.sh` | Clean Redis and/or module outputs | | `make deploy [args]` | `scripts/deploy.sh` | Install Redis and module artifacts | | `make modules-update [args]` | `scripts/modules-update.sh` | Clone/refresh module sources from the manifest | | `make modules-shallow [args]` | `scripts/modules-shallow.sh` | Re-clone selected modules with shallow history | | `make sync-redis-conf [args]` | `scripts/sync-redis-conf.sh` | Regenerate managed module config | | `make tarball TAG=...` | `scripts/tarball.sh` | Create a reproducible Redis + modules source tarball | All accept module selectors: ```sh make build all # core + all modules make build redisjson # core + one module make run none # core only, no modules make test redisearch TEST='some:test:name' ``` ## Runtime Configuration Removes the hand-maintained `redis-full.conf` (which could drift from upstream). Config is now generated: `sync-redis-conf.sh` produces a managed Modules block appended to verbatim `redis.conf`, with `loadmodule` lines (active when built or `ASSUME_BUILT=1`) plus each module's `module.conf`. The generated file is git-ignored. ## Docker & Release - **`docker/Dockerfile.noble`** — Ubuntu 24.04 build env for core + modules (amd64/arm64). - **`scripts/tarball.sh`** — reproducible source bundle: archives core at `TAG`, clones modules at pinned refs, strips `.git`/build artifacts, generates config, and produces a deterministic gzip. `utils/releasetools/01_create_tarball.sh` now delegates to it. - **`post-release-automation.yml`** — builds the bundle on published releases, uploads it as artifact + release asset, and reports SHA256. ## Docs `modules/MODULES.md` (operator guide), `modules/README.md`, and `README.md` cover the manifest schema, Make targets, tarball flow, Docker env, and troubleshooting. ## Compatibility Notes - `BUILD_WITH_MODULES=yes` is replaced by `make build [redis|core|none|all|<module>...]`. - `redis-full.conf` is now generated and git-ignored. - Module clones live under `modules/<name>/src/` (git-ignored). - `vector-sets` is omitted — it already lives in-tree. - RediSearch builds with `LTO=1 REDISEARCH_GENERATE_HEADERS=0 INLINE_LSE_ATOMICS=0`. --------- Co-authored-by: debing.sun <debing.sun@redis.com>
102 lines
3.6 KiB
Bash
Executable file
102 lines
3.6 KiB
Bash
Executable file
#!/bin/sh
|
|
# test.sh — dispatch test execution.
|
|
#
|
|
# Usage: scripts/test.sh # Redis tests (make -C src test)
|
|
# scripts/test.sh redis | none # same — Redis only
|
|
# scripts/test.sh all | . | '*' # tests for every cloned module
|
|
# scripts/test.sh <module> # full suite for one module
|
|
# scripts/test.sh <module> <test_name> # one test (positional)
|
|
# Env: TEST=<name> single-test selector for names containing ':' (which
|
|
# Make can't pass positionally). Positional <test_name>
|
|
# wins if both are given.
|
|
|
|
set -eu
|
|
|
|
SCRIPT_DIR="$(cd -- "$(dirname -- "$0")" && pwd)" || exit 1
|
|
. "$SCRIPT_DIR/lib/manifest.sh"
|
|
cd "$REPO_ROOT"
|
|
|
|
MAKE_BIN="${MAKE:-make}"
|
|
test_var="${TEST:-}"
|
|
|
|
if [ "$#" -eq 0 ]; then
|
|
echo "==> Running Redis tests (src/)"
|
|
exec "$MAKE_BIN" -C src test
|
|
fi
|
|
|
|
target="$1"; shift
|
|
cloned="$(cloned_modules)"
|
|
|
|
case "$target" in
|
|
redis|none)
|
|
if [ "$#" -gt 0 ] || [ -n "$test_var" ]; then
|
|
echo "ERROR: cannot pass a test name together with '$target'"
|
|
echo " ('$target' selects Redis tests only — no module test name applies)"
|
|
exit 1
|
|
fi
|
|
echo "==> Running Redis tests (src/)"
|
|
exec "$MAKE_BIN" -C src test
|
|
;;
|
|
all|.|'*')
|
|
if [ "$#" -gt 0 ] || [ -n "$test_var" ]; then
|
|
echo "ERROR: cannot pass a test name together with '$target'"
|
|
echo " (select a single module when running one test)"
|
|
exit 1
|
|
fi
|
|
if [ -z "$cloned" ]; then
|
|
echo "ERROR: no cloned modules under modules/*/src"
|
|
echo " run 'make modules-update all' and 'make build' first"
|
|
exit 1
|
|
fi
|
|
echo "==> Running tests for all cloned modules: $cloned"
|
|
failed=""
|
|
for name in $cloned; do
|
|
echo
|
|
echo "==> [test] $name (modules/$name/src)"
|
|
if ! "$MAKE_BIN" -C "modules/$name/src" test \
|
|
REDIS_SERVER="$REPO_ROOT/src/redis-server"; then
|
|
failed="$failed $name"
|
|
fi
|
|
done
|
|
echo
|
|
if [ -n "$failed" ]; then
|
|
echo "==> Module tests FAILED for:$failed"
|
|
exit 1
|
|
fi
|
|
echo "==> Module tests passed for all cloned modules"
|
|
;;
|
|
*)
|
|
ok=""
|
|
case " $cloned " in *" $target "*) ok=1 ;; esac
|
|
if [ -z "$ok" ]; then
|
|
echo "ERROR: module '$target' is not cloned under modules/$target/src"
|
|
echo "Cloned modules: $cloned"
|
|
echo "Usage:"
|
|
echo " make test # run Redis tests"
|
|
echo " make test redis # run Redis tests (explicit)"
|
|
echo " make test all # run tests for every cloned module"
|
|
echo " make test <module> # run all tests for one module"
|
|
echo " make test <module> <test_name> # run a single test (positional)"
|
|
echo " make test <module> TEST=<name> # run a single test (use this for names with ':')"
|
|
exit 1
|
|
fi
|
|
tname=""
|
|
if [ "$#" -eq 1 ]; then
|
|
tname="$1"
|
|
elif [ "$#" -gt 1 ]; then
|
|
echo "ERROR: too many arguments."
|
|
echo "Usage: make test <module> [<test_name>] [TEST=<name>]"
|
|
exit 1
|
|
fi
|
|
[ -z "$tname" ] && tname="$test_var"
|
|
if [ -z "$tname" ]; then
|
|
echo "==> Running all tests for module '$target'"
|
|
exec "$MAKE_BIN" -C "modules/$target/src" test \
|
|
REDIS_SERVER="$REPO_ROOT/src/redis-server"
|
|
else
|
|
echo "==> Running test '$tname' for module '$target' (TEST=$tname)"
|
|
exec "$MAKE_BIN" -C "modules/$target/src" test TEST="$tname" \
|
|
REDIS_SERVER="$REPO_ROOT/src/redis-server"
|
|
fi
|
|
;;
|
|
esac
|