These two jobs run on exactly the same triggers and are effectively variations
of each other. There is no need to have two separate workflows for them.
This fixes:
.github/matrix.py:72: SyntaxWarning: "\." is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\."? A raw string is also an option.
return re.match('^v[0-9]+(\.[0-9]+)*$', version_string)
.github/matrix.py:89: SyntaxWarning: "\." is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\."? A raw string is also an option.
return re.match('^AWS-LC-FIPS-[0-9]+(\.[0-9]+)*$', version_string)
.github/matrix.py:106: SyntaxWarning: "\." is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\."? A raw string is also an option.
return re.match('^v[0-9]+(\.[0-9]+)*-stable$', version_string)
The thread-local variable dbg_indent_level used a generic name that could
collide with identifiers in other compilation units. Renamed it to
flt_ot_dbg_indent_level so that it carried the flt_ot_ prefix consistent
with the rest of the OpenTracing filter namespace. The rename covered the
declaration, definition, and all macro references in debug.h, parser.c and
util.c.
The -Iaddons/ot/include flag in OT_CFLAGS allowed source files to use a
bare #include "include.h", which was fragile because it depended on the
compiler search path. Removed that flag from the Makefile and changed
every source file under addons/ot/src/ to use the relative include path
../include/include.h instead. This made header resolution explicit and
consistent with standard addon conventions.
After calling flt_ot_conf_group_init() and storing the result in
flt_ot_current_group, the code incorrectly checked flt_ot_current_config
for NULL instead of the newly assigned flt_ot_current_group. This meant
a failed group init was never detected and the error path was never taken.
The local variable str was declared but never assigned a value other than
NULL. The error-handling block that called flt_ot_conf_str_free(&str) on
it was therefore a no-op. Removed both the unused variable and the dead
cleanup path.
In flt_ot_vars_scope_dump(), switched from cebu64_first()/cebu64_next() to
cebu64_imm_first()/cebu64_imm_next() for iterating the variable name trees.
Since this function only reads variables under a read lock, the immutable
traversal API is the correct choice. Also updated the container_of()
member from 'node' to 'name_node' to match the current struct var layout.
Replaced the static key-value attribute storage in update-form instruments
with sample-evaluated attributes, matching the log-record attr change.
The 'attr' keyword now accepts a key and a HAProxy sample expression
evaluated at runtime.
The struct (conf.h) changed from otelc_kv/attr_len to a list of
flt_otel_conf_sample entries. The parser (parser.c) calls
flt_otel_parse_cfg_sample() with n=1 per attr keyword. At runtime
(event.c) each attribute is evaluated via flt_otel_sample_eval() and
added via flt_otel_sample_add_kv() to a bare flt_otel_scope_data_kv,
which is passed to the meter.
Updated documentation, debug macro and test configurations.
Replaced the static key-value attribute storage in log-record with
sample-evaluated attributes. The 'attr' keyword now accepts a key and a
HAProxy sample expression evaluated at runtime, instead of a static string
value.
The struct (conf.h) changed from otelc_kv/attr_len to a list of
flt_otel_conf_sample entries. The parser (parser.c) calls
flt_otel_parse_cfg_sample() with n=1 per attr keyword. At runtime
(event.c) each attribute is evaluated via flt_otel_sample_eval() and added
via flt_otel_sample_add_kv() to a bare flt_otel_scope_data_kv, which is
passed to logger->log_span().
Updated documentation, debug macro and test configurations.
Factored the sample evaluation logic out of flt_otel_sample_add() into a
new flt_otel_sample_eval() function that evaluates a sample definition
into an otelc_value. Both the log-format path and the bare sample
expression path are handled, with a flag_native parameter controlling
native type preservation for single-expression samples.
flt_otel_sample_add() now calls flt_otel_sample_eval() and dispatches the
result.
Made flt_otel_sample_add_kv() non-static so callers outside util.c can
add key-value pairs directly to a bare flt_otel_scope_data_kv without
requiring the full flt_otel_scope_data structure.
The test directory gained a speed test guide (README-test-speed)
explaining how to run performance benchmarks at various rate-limit levels,
together with benchmark result files for the standalone, composite,
context-propagation, and frontend-backend test configurations.
Added README documentation for each test configuration (sa, cmp, ctx,
fe-be, empty, full) describing event coverage, signal usage, instrument
tables, span hierarchies and run instructions.
Added the full documentation set for the OpenTelemetry filter.
The main README served as the user-facing guide covering build
instructions, core OpenTelemetry concepts, the complete filter
configuration reference, usage examples with worked scenarios,
CLI commands, and known limitations.
Supplementary documents provided a detailed configuration guide with
worked examples (README-configuration), an internal C structure reference
for developers (README-conf), a function reference organized by source
file (README-func), an architecture and implementation review
(README-implementation), and miscellaneous notes (README-misc).
Added the 'full' test configuration that exercises all 29 supported OTel
filter events with all three signal types (traces, metrics, logs). Every
instrument definition has a corresponding update.
Added "log-record" as the third OpenTelemetry signal alongside traces
(span) and metrics (instrument). This includes the
flt_otel_conf_log_record structure definition, parser keyword defines,
the otel-scope section parser with optional "id", "event", "span", and
"attr" keywords followed by sample fetch expressions or a log-format
string, init/free lifecycle, scope list wiring, log-format evaluation
in flt_otel_scope_run_instrument_record(), a test configuration example,
log-record span reference validation in flt_otel_check(), and logger
handle creation, startup, and teardown in the filter lifecycle.
Added the "instrument" keyword to otel-scope sections for recording metric
measurements alongside traces.
Introduced flt_otel_conf_instrument holding instrument type, description,
unit, sample expressions, and optional key-value attributes. The
supported synchronous integer-precision instrument types were counters,
histograms, up-down counters, and gauges.
Instruments followed a two-form design: a "create" form defined a new
instrument with its type and value expression, while an "update" form
recorded measurements against an existing instrument with per-scope
attributes.
Instrument creation was performed lazily at first use with HA_ATOMIC_CAS
to guarantee thread-safe one-time initialization. The configuration
check phase validated that every update-form had a matching create-form
definition and that create-form names were unique across all scopes.
The meter lifecycle was integrated into filter init and deinit, starting
the meter alongside the tracer and shutting it down during cleanup.
Added span link support, allowing a span to reference other spans or
extracted contexts without establishing a parent relationship.
Introduced the flt_otel_conf_link structure and added a links list to
flt_otel_conf_span. The parser accepted both an inline syntax on the span
declaration line ("span <name> link <target>") and a standalone multi-
argument form ("link <span> ..."), each creating a conf_link entry
appended to the span's link list.
At runtime, each configured link name was resolved against the active
spans and extracted contexts in the runtime context. Resolved references
were collected into flt_otel_scope_data_link entries and passed to the C
wrapper add_link API during span creation.
Initialization, cleanup, and debug dump routines were added for the link
data structures at both configuration and runtime levels.
Added a test suite under addons/otel/test/ for the OpenTelemetry filter.
Five scenarios exercise different filter capabilities: standalone (sa)
covers all hook points including idle-timeout heartbeats, metrics and log
records; compact (cmp) covers the full request/response lifecycle with
ACL-based error handling; context (ctx) tests explicit inject/extract
propagation through numbered context variables; frontend/backend (fe/be)
tests distributed tracing across two HAProxy instances; and empty tests
bare filter initialisation with no active scopes.
A performance benchmarking script (test-speed.sh) uses wrk to measure
throughput and latency at different rate-limit settings (100% through 0%,
disabled, and filter-off). Each scenario includes comprehensive YAML
exporter definitions covering OTLP file/gRPC/HTTP, ostream, memory,
Zipkin, and Elasticsearch backends.
Extended flt_otel_parse_cfg_sample() to accept log-format strings in
addition to bare sample expressions. Added lf_expr and lf_used fields
to flt_otel_conf_sample.
Extended flt_otel_sample_add() to evaluate log-format expressions when
lf_used was set.
Added the "otel-group" action keyword that allows executing a named group
of OTel scopes from HAProxy TCP and HTTP action rule contexts.
The new group.c module registers the "otel-group" keyword for all four
action contexts (tcp-request, tcp-response, http-request, http-response)
and implements the action lifecycle callbacks.
The parser flt_otel_group_parse() accepts a filter ID and group ID as
arguments, duplicates them into the action rule's argument slots, and
wires up the check, action, and release callbacks.
The post-parse validator flt_otel_group_check() resolves the filter ID and
group ID string references into direct configuration pointers by searching
the proxy's filter list for a matching OTel filter and then looking up the
named group within that filter's configuration.
The action handler flt_otel_group_action() retrieves the filter and group
configuration from the resolved rule arguments, verifies the filter is
attached to the stream and not disabled, then iterates through all scopes
in the group and executes each via flt_otel_scope_run() with a shared
timestamp pair. This allows operators to trigger OTel instrumentation
conditionally from HAProxy rules, for example applying different tracing
scopes based on ACL conditions or request properties.
Added HAProxy CLI commands that allow runtime inspection and modification
of OTel filter settings without requiring a configuration reload.
The new cli.c module registers CLI keywords under the "otel" prefix and
implements the following commands: flt_otel_cli_parse_status() displays a
comprehensive status report of all OTel filter instances including filter
ID, proxy, disabled state, hard-error mode, logging state, rate limit,
analyzer bits, and SDK diagnostic message count;
flt_otel_cli_parse_disabled() enables or disables filtering across all
instances; flt_otel_cli_parse_option() toggles the hard-error mode that
controls whether errors disable the filter for a stream or are silently
ignored; flt_otel_cli_parse_logging() manages the logging state with
support for off, on, and dontlog-normal modes; flt_otel_cli_parse_rate()
adjusts the sampling rate limit as a floating-point percentage; and
flt_otel_cli_parse_debug() sets the debug verbosity level in debug builds.
All modifications are applied atomically across every OTel filter instance
in every proxy.
The CLI initialization is called from flt_otel_ops_init() during filter
startup via flt_otel_cli_init(), which registers the keyword table through
cli_register_kw().
Supporting changes include the FLT_OTEL_U32_FLOAT macro for converting the
internal uint32_t rate representation to a human-readable percentage, the
FLT_OTEL_PROXIES_LIST_START/END iteration macros for traversing all OTel
filter instances across the proxy list, and flt_otel_filters_dump() for
debug logging of filter instances.
Introduced an alternative variable scanning strategy that directly walked
the CEB tree of HAProxy's variable store instead of maintaining a separate
tracking buffer.
The Makefile auto-detected whether struct var carried a "name" member
by inspecting include/haproxy/vars-t.h and conditionally defined
USE_OTEL_VARS_NAME. When enabled, the tracking buffer (flt_otel_ctx) and
its callback type were compiled out and replaced by direct tree walks.
flt_otel_vars_unset() walked the CEB tree for the resolved scope, removed
every variable whose normalized name matched the given prefix followed by
a dot, and adjusted the variable accounting. flt_otel_vars_get()
performed the same prefix scan under a read lock, denormalized each
matching variable name back to its original OTel form, and assembled the
results into an otelc_text_map.
A helper flt_otel_vars_get_scope() was added to resolve scope name strings
("txn", "sess", "proc", "req", "res") to the corresponding HAProxy
variable store. The set path skipped the tracking buffer update when
prefix scanning was available.
Added support for storing OTel span context in HAProxy transaction
variables as an alternative to HTTP headers, enabled by the OTEL_USE_VARS
compile flag.
The new vars.c module implements variable-based context propagation
through the HAProxy variable subsystem. Variable names are constructed
from a configurable prefix and the OTel propagation key, with dots
normalized to underscores for HAProxy variable name compatibility
and denormalized back during retrieval. The module provides
flt_otel_var_register() to pre-register variables at parse time,
flt_otel_var_set() and flt_otel_vars_unset() to store and clear context
key-value pairs in the txn scope, flt_otel_vars_get() to collect all
variables matching a prefix into an otelc_text_map for context extraction,
and flt_otel_vars_dump() for debug logging of all OTel variables.
The inject/extract keywords in the scope parser now accept an optional
"use-vars" argument alongside "use-headers", controlled by the new
FLT_OTEL_CTX_USE_VARS flag. Both storage types can be used simultaneously
on the same span context, allowing context to be propagated through both
HTTP headers and variables.
The scope runner in event.c was extended to handle variable-based
context in parallel with headers: during extraction, it reads matching
variables via flt_otel_vars_get() when FLT_OTEL_CTX_USE_VARS is set;
during injection, it stores each propagation key as a variable via
flt_otel_var_set(). The unused resource cleanup now also unsets context
variables when removing failed extraction contexts.
The filter attach callback registers and sets the sess.otel.uuid variable
with the generated session UUID, making the trace identifier available to
HAProxy log formats and ACL expressions.
The feature is conditionally compiled: the OTEL_USE_VARS flag controls
whether vars.c is included in the build and whether the "use-vars" keyword
is available in the configuration parser.
Added the HTTP header manipulation layer that enables span context
injection into and extraction from HAProxy's HTX message buffers,
completing the end-to-end context propagation path.
The new http.c module implements three public functions:
flt_otel_http_headers_get() extracts HTTP headers matching a name prefix
from the channel's HTX buffer into an otelc_text_map structure, stripping
the prefix and separator dash from header names before storage;
flt_otel_http_header_set() constructs a full header name from a prefix and
suffix joined by a dash, removes all existing occurrences, and optionally
adds the header with a new value; and flt_otel_http_headers_remove()
removes all headers matching a given prefix. A debug-only
flt_otel_http_headers_dump() logs all HTTP headers from a channel at
NOTICE level.
The scope runner in event.c now extracts propagation contexts from HTTP
headers before processing spans: for each configured extract context, it
calls flt_otel_http_headers_get() to read matching headers into a text
map, then passes the text map to flt_otel_scope_context_init() which
extracts the OTel span context from the carrier. After span execution,
the span runner injects the span context back into HTTP headers via
flt_otel_inject_http_headers() followed by flt_otel_http_header_set()
for each propagation key.
The unused resource cleanup in flt_otel_scope_free_unused() now also
removes contexts that failed extraction by deleting their associated
HTTP headers via flt_otel_http_headers_remove() before freeing the scope
context structure.
Added the span context injection and extraction layer that bridges the
OTel C wrapper's propagation API with HAProxy's HTTP headers and text map
carriers.
The new otelc.c module implements four public functions that wrap the
OTel C wrapper's context propagation methods: flt_otel_inject_text_map()
and flt_otel_inject_http_headers() serialize a span's context into
a text map or HTTP headers carrier for outbound propagation, while
flt_otel_extract_text_map() and flt_otel_extract_http_headers()
deserialize an inbound carrier into an otelc_span_context for parent
linking.
Each direction uses a pair of callbacks registered on the carrier
structure. The injection writers (flt_otel_text_map_writer_set_cb and
flt_otel_http_headers_writer_set_cb) store key-value pairs emitted by the
SDK into the carrier's text map via OTELC_TEXT_MAP_ADD(). The extraction
readers (flt_otel_text_map_reader_foreach_key_cb and
flt_otel_http_headers_reader_foreach_key_cb) iterate the carrier's text
map entries and pass each pair to the SDK's handler callback.
The scope context initialization in flt_otel_scope_context_init() now
calls flt_otel_extract_http_headers() to extract the span context from the
provided text map carrier and stores it in the scope context structure,
making extracted contexts available for parent linking in subsequent span
creation.
Implemented the scope execution engine that creates OTel spans, evaluates
sample expressions to collect telemetry data, and manages span lifecycle
during request and response processing.
The scope runner flt_otel_scope_run() was expanded from a stub into a
complete implementation that evaluates ACL conditions on the scope,
extracts span contexts from HTTP headers when configured, iterates over
the scope's span definitions calling flt_otel_scope_run_span() for each,
marks and finishes completed spans, and cleans up unused runtime
resources.
The span runner flt_otel_scope_run_span() creates OTel spans via the
tracer with optional parent references (from other spans or extracted
contexts), collects telemetry by calling flt_otel_sample_add() for each
configured attribute, event, baggage and status entry, then applies the
collected data to the span (attributes, events with their own key-value
arrays, baggage items, and status code with description) and injects the
span context into HTTP headers when configured.
The sample evaluation layer converts HAProxy sample expressions into OTel
telemetry data. flt_otel_sample_add() evaluates each sample expression
against the stream, converts the result via flt_otel_sample_to_value()
which preserves native types (booleans as OTELC_VALUE_BOOL, integers as
OTELC_VALUE_INT64, all others as strings), and routes the key-value pair
to the appropriate collector based on the sample type (attribute, event,
baggage, or status). The key-value arrays grow dynamically using the
FLT_OTEL_ATTR_INIT_SIZE and FLT_OTEL_ATTR_INC_SIZE constants.
Span finishing is handled in two phases: flt_otel_scope_finish_mark()
marks spans and contexts for completion using exact name matching or
wildcards ("*" for all, "*req*" for request-direction, "*res*" for
response-direction), and flt_otel_scope_finish_marked() ends all marked
spans with a common monotonic timestamp and destroys their contexts.
Connected the OpenTelemetry C wrapper library to the filter lifecycle by
implementing the library initialization, tracer creation, memory and
thread callbacks, shutdown sequence, and span completion.
The flt_otel_lib_init() function now verifies the C wrapper library
version against the compiled headers, calls otelc_init() with the absolute
configuration file path, and creates the tracer via otelc_tracer_create().
On success, it registers HAProxy pool-based memory callbacks
(flt_otel_mem_malloc, flt_otel_mem_free) and a thread ID callback
(flt_otel_thread_id) through otelc_ext_init(), so the C++ SDK allocates
span and context objects from pool_head_otel_span_context. A custom log
handler (flt_otel_log_handler_cb) is registered via otelc_log_set_handler()
to count OTel SDK internal diagnostic messages in the flt_otel_drop_cnt
counter.
The per-thread init callback now starts the tracer thread via
OTELC_OPS(tracer, start) instead of unconditionally returning success.
The deinit callback saves the tracer handle before freeing the
configuration, then shuts down the library via otelc_deinit() after the
pool is destroyed, ensuring the ext callbacks remain valid while the
configuration structures are still being freed. In debug builds, it logs
wrapper statistics, attach counters, and per-event HTX usage counters
before shutdown.
The runtime context cleanup in flt_otel_runtime_context_free() now ends
all active spans with a common monotonic timestamp via
OTELC_OPSR(span, end_with_options) before freeing them. The scope context
cleanup in flt_otel_scope_context_free() now destroys the underlying OTel
span context via OTELC_OPSR(context, destroy).
The parser gained static storage for the debug memory tracker
(OTELC_DBG_MEM) and its initialization in the parse entry point, used when
compiled with the OTELC_DBG_MEM flag.
Replaced the stub filter callbacks with full implementations that dispatch
OTel events through the scope execution engine, and added the supporting
debug, error handling and utility infrastructure.
The filter lifecycle callbacks (init, deinit, init_per_thread) now
initialize the OpenTelemetry C wrapper library, create the tracer from the
instrumentation configuration file, enable HTX stream filtering, and clean
up the configuration and memory pools on shutdown.
The stream callbacks (attach, stream_start, stream_set_backend,
stream_stop, detach, check_timeouts) create the per-stream runtime context
on attach with rate-limit based sampling, fire the corresponding OTel
events (on-stream-start, on-backend-set, on-stream-stop), manage the
idle timeout timer with reschedule logic in detach, and free the runtime
context in check_timeouts. The attach callback also registers the
required pre and post channel analyzers from the instrumentation
configuration.
The channel callbacks (start_analyze, pre_analyze, post_analyze,
end_analyze) register per-channel analyzers, map analyzer bits to event
indices via flt_otel_get_event(), and dispatch the matching events.
The end_analyze callback also fires the on-server-unavailable event
when response analyzers were configured but never executed.
The HTTP callbacks (http_headers, http_end, http_reply, and the debug-only
http_payload and http_reset) dispatch their respective request/response
events based on the channel direction.
The event dispatcher flt_otel_event_run() in event.c iterates over all
scopes matching a given event index and calls flt_otel_scope_run() for
each, sharing a common monotonic and wall-clock timestamp across all spans
within a single event.
Error handling is centralized in flt_otel_return_int() and
flt_otel_return_void(), which implement the hard-error/soft-error policy:
hard errors disable the filter for the stream, soft errors are silently
cleared.
The new debug.h header provides conditional debug macros
(FLT_OTEL_DBG_ARGS, FLT_OTEL_DBG_BUF) and the FLT_OTEL_LOG macro for
structured logging through the instrumentation's log server list. The
utility layer gained debug-only label functions for channel direction,
proxy mode, stream position, filter type, and analyzer bit name lookups.
Added the memory pool management and the runtime scope layer that track
per-stream OTel spans and contexts during request processing.
The pool layer in pool.c manages HAProxy memory pools for the runtime
structures used by the filter: scope spans, scope contexts, runtime
contexts, and span contexts. Each pool is conditionally compiled via
USE_POOL_OTEL_* macros defined in config.h and registered with
REGISTER_POOL(). The allocation functions (flt_otel_pool_alloc,
flt_otel_pool_strndup, flt_otel_pool_free) transparently fall back to
heap allocation when the corresponding pool is not enabled. Trash buffer
helpers (flt_otel_trash_alloc, flt_otel_trash_free) provide scratch space
using either HAProxy's trash chunk pool or direct heap allocation.
The scope layer in scope.c implements the per-stream runtime state. The
flt_otel_runtime_context structure is allocated when a stream starts and
holds the stream and filter references, hard-error/disabled/logging flags
copied from the instrumentation configuration, idle timeout state, a
generated UUID, and lists of active scope spans and extracted scope
contexts. Scope spans (flt_otel_scope_span) carry the operation name,
fetch direction, the OTel span handle, and optional parent references
resolved from other spans or extracted contexts. Scope contexts
(flt_otel_scope_context) hold an extracted span context obtained from
a carrier text map via the tracer. The scope data structures
(flt_otel_scope_data) aggregate growable key-value arrays for attributes
and baggage, a linked list of named events with their own attribute
arrays, and a span status code with description, representing the
telemetry collected during a single event execution.
Implemented the flt_otel_ops_check() callback that validates the parsed
OTel filter configuration after all HAProxy configuration sections have
been processed.
The check callback performs the following validations: resolves deferred
sample fetch arguments under full frontend and backend capabilities,
verifies uniqueness of filter IDs across all proxies, ensures the
instrumentation section and its configuration file are present, checks
for duplicate group and scope section names, verifies that groups are not
empty, resolves group-to-scope and instrumentation-to-group/scope
cross-references by linking placeholder entries to their definitions,
detects unused scopes, counts root spans and warns when the count differs
from one, and accumulates the required channel analyzer bits from all used
scopes into the instrumentation configuration.
The commit also added the flt_otel_counters structure to track per-event
diagnostic counters in debug builds, the FLT_OTEL_ALERT macro for
filter-scoped error messages, and the FLT_OTEL_DBG_LIST macro for
iterating and dumping named configuration lists.
Added the full configuration parser that reads the OTel filter's external
configuration file and the event model that maps filter events to HAProxy
channel analyzers.
The event model in event.h defines an X-macro table
(FLT_OTEL_EVENT_DEFINES) that maps each filter event to its HAProxy
channel analyzer bit, sample fetch direction, and event name. Events
cover stream lifecycle (start, stop, backend-set, idle-timeout), client
and server sessions, request analyzers (frontend and backend TCP and
HTTP inspection, switching rules, sticking rules, RDP cookie), response
analyzers (TCP inspection, HTTP response processing), and HTTP headers,
end, and reply callbacks. The event names are partially compatible with
the SPOE filter. The flt_otel_event_data[] table in event.c is generated
from the same X-macro and provides per-event metadata at runtime.
The parser in parser.c implements section parsers for the three OTel
configuration blocks: otel-instrumentation (tracer identity, log server,
config file path, groups, scopes, ACLs, rate-limit, options for
disabled/hard-errors/nolognorm, and debug-level), otel-group (group
identity and scope list), and otel-scope (scope identity, span definitions
with optional root/parent modifiers, attributes, events, baggages, status
codes, inject/extract context operations, finish lists, idle-timeout,
ACLs, and otel-event binding with optional if/unless ACL conditions).
Each section has a post-parse callback that validates the parsed state.
The top-level flt_otel_parse_cfg() temporarily registers these section
parsers, loads the external configuration file via parse_cfg(), and
handles deferred resolution of sample fetch arguments by saving them in
conf->smp_args for later resolution in flt_otel_check() when full frontend
and backend capabilities are available. The main flt_otel_parse() entry
point was extended to parse the filter ID and config file keywords, verify
that insecure-fork-wanted is enabled, and wire the parsed configuration
into the flt_conf structure.
The utility layer gained flt_otel_strtod() and flt_otel_strtoll() for
validated string-to-number conversion used by rate-limit and debug-level
parsing.
Added the configuration structures that model the OTel filter's
instrumentation hierarchy and the utility functions that support the
configuration parser.
The configuration is organized as a tree rooted at flt_otel_conf, which
holds the proxy reference, filter identity, and lists of groups and
scopes. Below it, flt_otel_conf_instr carries the instrumentation
settings: tracer handle, rate limiting, hard-error mode, logging state,
channel analyzers, and placeholder references to groups and scopes.
Groups (flt_otel_conf_group) aggregate scopes by name. Scopes
(flt_otel_conf_scope) bind an event to its ACL condition, span context
declarations, span definitions and a list of spans scheduled for
finishing. Spans (flt_otel_conf_span) carry attributes, events,
baggages and status entries, each represented as flt_otel_conf_sample
structures that pair a key with concatenated sample-expression arguments.
All configuration types share a common header macro (FLT_OTEL_CONF_HDR)
that embeds an identifier string, its length, a configuration line number,
and a list link. Their init and free functions are generated by the
FLT_OTEL_CONF_FUNC_INIT and FLT_OTEL_CONF_FUNC_FREE macros in
conf_funcs.h, with per-type custom initialization and cleanup bodies.
The utility layer in util.c provides argument counting and concatenation
for the configuration parser, sample data to string conversion covering
boolean, integer, IPv4, IPv6, string and HTTP method types, and debug
helpers for dumping argument arrays and linked list state.
The OpenTelemetry (OTel) filter enables distributed tracing of requests
across service boundaries, export of metrics such as request rates,
latencies and error counts, and structured logging tied to trace context,
giving operators a unified view of HAProxy traffic through any
OpenTelemetry-compatible backend.
The OTel filter is implemented using the standard HAProxy stream filter
API. Stream filters attach to proxies and intercept traffic at each stage
of processing: they receive callbacks on stream creation and destruction,
channel analyzer events, HTTP header and payload processing, and TCP data
forwarding. This allows the filter to collect telemetry data at every
stage of the request/response lifecycle without modifying the core proxy
logic.
This commit added the minimum set of files required for the filter to
compile: the addon Makefile with pkg-config-based detection of the
opentelemetry-c-wrapper library, header files with configuration
constants, utility macros and type definitions, and the source files
containing stub filter operation callbacks registered through
flt_otel_ops and the "opentelemetry" keyword parser entry point.
The filter uses the opentelemetry-c-wrapper library from HAProxy
Technologies, which provides a C interface to the OpenTelemetry C++ SDK.
This wrapper allows HAProxy, a C codebase, to leverage the full
OpenTelemetry observability pipeline without direct C++ dependencies
in the HAProxy source tree.
https://github.com/haproxytech/opentelemetry-c-wrapperhttps://github.com/open-telemetry/opentelemetry-cpp
Build options:
USE_OTEL - enable the OpenTelemetry filter
OTEL_DEBUG - compile the filter in debug mode
OTEL_INC - force the include path to the C wrapper
OTEL_LIB - force the library path to the C wrapper
OTEL_RUNPATH - add the C wrapper RUNPATH to the executable
Example build with OTel and debug enabled:
make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
conn_recv_qstrm() may be called several times per connection if the read
data is too short and a truncated record is received.
Previously, record length was parsed every time the function is invoked.
However, this must only be performed if record length varint is
incomplete. Once read and parsed, data are removed from the buffer via
b_quic_dec_int(). Thus, next conn_recv_qstrm() run will reread an
invalid record length this time.
This patch fixes this by only parsing record length if <rxrlen> member
is null. Prior to it, parsing of QMux transport parameters would fail in
case of a first truncated read, which would prevent the connection
initialization.
No need to backport.
A QCC connection may be flagged with QC_CF_ERRL to trigger a
CONNECTION_CLOSE emission. However, for now error reporting is not
functional with QMux, as it relies on quic_conn layer access.
To prevent a crash in qcc_io_send() when using QMux, add a
conn_is_quic() check when QC_CF_ERRL is set to ensure no access will be
performed on quic_conn layer. In the future, this should be extended so
that QMux is also able to emit CONNECTION_CLOSE for connection closure.
No need to backport.
First, we must not emit any warning if splicing is not configured and the
global maxpipes value is 0. Then we must not remove GTUNE_USE_SPLICE flag
when we fail to allocate the haterm master pipe. Instead, we test it when we
negociate with the opposite side, to properly exclude the splicing if it is
not usable.
No backport needed.
This reverts commit 8056117e98.
Moving haterm init from haproxy is not the right way to fix the issue
because it should be possible to use a haterm configuration in haproxy.
So let's revert the commit above.
Add QUIC BLOCKED frames in the list of supported types in
qstrm_parse_frm(). Nothing is really implemented for them as for QUIC,
but this prevents a crash when receiving one of them via QMux.
No need to backport.
QMux draft 01 support is mostly achieved thanks to the recent
implementation of the Record layer. This patch thus updates the link in
the documentation to the validated draft version.
This patch implements emission of the new Record layer for QMux frames.
This handles mux-quic and xprt_qstrm layers as this is performed
similarly in both cases.
Currently, the simplest approach has been prefered : each frame is
encoded in its own record. This is not the most efficient in size but it
is extremely simple to implement for a first interop testing.
This patch implements the new QMux record layer parsing for xprt_qstrm.
This is mostly similar to the MUX code from the previous patch.
Along with this change, a new xprt_qstrm layer accessor exposes the
possible remaining record length after Transport parameters parsing.
This can only occur when xprt_qstrm Rx buffer is not completely emptied
due to other following frames. If stored in the same record, MUX layer
has to know the remaining record length.
Thus, xprt_qstrm_rxrlen() is now used in qmux_init() to preinitialize
<rx.rlen> QCC field.
This is the first patch of a serie which aims to support the new Record
layer defined by the draft 01 of QMux protocol.
https://www.ietf.org/archive/id/draft-ietf-quic-qmux-01.html#name-qmux-records
This patch deals with QMux reception at the MUX layer. The function
qcc_qstrm_recv() is adapted to read record headers before frame parsing.
This requires to keep the last record length read in a new QCC field
named <rx.rlen>.
Frames are only parsed once a full record is received. One of the
advantage of the record layer is that it can only contains whole frame
without truncation.
This patch implements proper connection error handling for xprt_qstrm
layer. Basically, processing is interrupted if CO_FL_ERROR is
encountered after either rcv_buf or snd_buf operations. Connectionn
error is set to the newly defined value CO_ER_QSTRM.
