Obtain env vars from conf.sh in pytest runner

The commands executed by pytest during a system test need to have the
same environment variables set as if they were executed by the run.sh
shell script.

It was decided that for the moment, legacy way of executing system tests
with run.sh should be kept, which complicates things a bit. In order to
avoid duplicating the required variables in both conf.sh and pytest, it
was decided to use the existing conf.sh as the only authoritative
place for the variables.

It is necessary to process the environment variables from conf.sh right
when conftest.py is loaded, since they might be needed right away (e.g.
to test for feature support during test collection).

This solution is a bit hacky and is only meant to be used during the
transitory phase when both pytest and the legacy run.sh are both
supported. In the future, a superior pytest-only solution should be
used.

For discussion of other options, refer to
https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/6809#note_318889

(cherry picked from commit 2f7af791a1)

bin/tests/system/conftest.py

@@ -49,4 +49,44 @@ def control_port():
 # don't branch the code.
 
 if os.getenv("LEGACY_TEST_RUNNER", "0") == "0":
-    pass  # will be implemented in followup commits
+    import logging
+    from pathlib import Path
+    import re
+    import subprocess
+
+    # ----------------------- Globals definition -----------------------------
+
+    FILE_DIR = os.path.abspath(Path(__file__).parent)
+    ENV_RE = re.compile("([^=]+)=(.*)")
+
+    # ---------------------- Module initialization ---------------------------
+
+    def parse_env(env_text):
+        """Parse the POSIX env format into Python dictionary."""
+        out = {}
+        for line in env_text.splitlines():
+            match = ENV_RE.match(line)
+            if match:
+                out[match.groups()[0]] = match.groups()[1]
+        return out
+
+    def get_env(cmd):
+        try:
+            proc = subprocess.run(
+                [cmd],
+                shell=True,
+                check=True,
+                cwd=FILE_DIR,
+                stdout=subprocess.PIPE,
+            )
+        except subprocess.CalledProcessError as exc:
+            logging.error("failed to get shell env: %s", exc)
+            raise exc
+        env_text = proc.stdout.decode("utf-8")
+        return parse_env(env_text)
+
+    # Read common environment variables for running tests from conf.sh.
+    # FUTURE: Remove conf.sh entirely and define all variables in pytest only.
+    CONF_ENV = get_env(". ./conf.sh && env")
+    os.environ.update(CONF_ENV)
+    logging.debug("conf.sh env: %s", CONF_ENV)