haproxy/addons/otel/include/include.h

58 lines
1.1 KiB
C
Raw Permalink Normal View History

MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#ifndef _OTEL_INCLUDE_H_
#define _OTEL_INCLUDE_H_
#include <errno.h>
#include <stdbool.h>
#include <math.h>
#include <values.h>
#ifdef USE_THREAD
# include <pthread.h>
#endif
MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
#include <haproxy/api.h>
#include <haproxy/cfgparse.h>
#include <haproxy/acl.h>
#include <haproxy/cli.h>
#include <haproxy/clock.h>
#include <haproxy/filters.h>
#include <haproxy/http_htx.h>
#include <haproxy/http_rules.h>
#include <haproxy/log.h>
#include <haproxy/proxy.h>
#include <haproxy/sample.h>
#include <haproxy/tcp_rules.h>
#include <haproxy/tools.h>
#include <haproxy/vars.h>
#include <opentelemetry-c-wrapper/include.h>
#include "config.h"
MEDIUM: otel: implemented filter callbacks and event dispatcher Replaced the stub filter callbacks with full implementations that dispatch OTel events through the scope execution engine, and added the supporting debug, error handling and utility infrastructure. The filter lifecycle callbacks (init, deinit, init_per_thread) now initialize the OpenTelemetry C wrapper library, create the tracer from the instrumentation configuration file, enable HTX stream filtering, and clean up the configuration and memory pools on shutdown. The stream callbacks (attach, stream_start, stream_set_backend, stream_stop, detach, check_timeouts) create the per-stream runtime context on attach with rate-limit based sampling, fire the corresponding OTel events (on-stream-start, on-backend-set, on-stream-stop), manage the idle timeout timer with reschedule logic in detach, and free the runtime context in check_timeouts. The attach callback also registers the required pre and post channel analyzers from the instrumentation configuration. The channel callbacks (start_analyze, pre_analyze, post_analyze, end_analyze) register per-channel analyzers, map analyzer bits to event indices via flt_otel_get_event(), and dispatch the matching events. The end_analyze callback also fires the on-server-unavailable event when response analyzers were configured but never executed. The HTTP callbacks (http_headers, http_end, http_reply, and the debug-only http_payload and http_reset) dispatch their respective request/response events based on the channel direction. The event dispatcher flt_otel_event_run() in event.c iterates over all scopes matching a given event index and calls flt_otel_scope_run() for each, sharing a common monotonic and wall-clock timestamp across all spans within a single event. Error handling is centralized in flt_otel_return_int() and flt_otel_return_void(), which implement the hard-error/soft-error policy: hard errors disable the filter for the stream, soft errors are silently cleared. The new debug.h header provides conditional debug macros (FLT_OTEL_DBG_ARGS, FLT_OTEL_DBG_BUF) and the FLT_OTEL_LOG macro for structured logging through the instrumentation's log server list. The utility layer gained debug-only label functions for channel direction, proxy mode, stream position, filter type, and analyzer bit name lookups.
2026-04-12 04:30:03 -04:00
#include "debug.h"
MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
#include "define.h"
MEDIUM: otel: added CLI commands for runtime filter management Added HAProxy CLI commands that allow runtime inspection and modification of OTel filter settings without requiring a configuration reload. The new cli.c module registers CLI keywords under the "otel" prefix and implements the following commands: flt_otel_cli_parse_status() displays a comprehensive status report of all OTel filter instances including filter ID, proxy, disabled state, hard-error mode, logging state, rate limit, analyzer bits, and SDK diagnostic message count; flt_otel_cli_parse_disabled() enables or disables filtering across all instances; flt_otel_cli_parse_option() toggles the hard-error mode that controls whether errors disable the filter for a stream or are silently ignored; flt_otel_cli_parse_logging() manages the logging state with support for off, on, and dontlog-normal modes; flt_otel_cli_parse_rate() adjusts the sampling rate limit as a floating-point percentage; and flt_otel_cli_parse_debug() sets the debug verbosity level in debug builds. All modifications are applied atomically across every OTel filter instance in every proxy. The CLI initialization is called from flt_otel_ops_init() during filter startup via flt_otel_cli_init(), which registers the keyword table through cli_register_kw(). Supporting changes include the FLT_OTEL_U32_FLOAT macro for converting the internal uint32_t rate representation to a human-readable percentage, the FLT_OTEL_PROXIES_LIST_START/END iteration macros for traversing all OTel filter instances across the proxy list, and flt_otel_filters_dump() for debug logging of filter instances.
2026-04-12 05:31:30 -04:00
#include "cli.h"
MEDIUM: otel: added configuration parser and event model Added the full configuration parser that reads the OTel filter's external configuration file and the event model that maps filter events to HAProxy channel analyzers. The event model in event.h defines an X-macro table (FLT_OTEL_EVENT_DEFINES) that maps each filter event to its HAProxy channel analyzer bit, sample fetch direction, and event name. Events cover stream lifecycle (start, stop, backend-set, idle-timeout), client and server sessions, request analyzers (frontend and backend TCP and HTTP inspection, switching rules, sticking rules, RDP cookie), response analyzers (TCP inspection, HTTP response processing), and HTTP headers, end, and reply callbacks. The event names are partially compatible with the SPOE filter. The flt_otel_event_data[] table in event.c is generated from the same X-macro and provides per-event metadata at runtime. The parser in parser.c implements section parsers for the three OTel configuration blocks: otel-instrumentation (tracer identity, log server, config file path, groups, scopes, ACLs, rate-limit, options for disabled/hard-errors/nolognorm, and debug-level), otel-group (group identity and scope list), and otel-scope (scope identity, span definitions with optional root/parent modifiers, attributes, events, baggages, status codes, inject/extract context operations, finish lists, idle-timeout, ACLs, and otel-event binding with optional if/unless ACL conditions). Each section has a post-parse callback that validates the parsed state. The top-level flt_otel_parse_cfg() temporarily registers these section parsers, loads the external configuration file via parse_cfg(), and handles deferred resolution of sample fetch arguments by saving them in conf->smp_args for later resolution in flt_otel_check() when full frontend and backend capabilities are available. The main flt_otel_parse() entry point was extended to parse the filter ID and config file keywords, verify that insecure-fork-wanted is enabled, and wire the parsed configuration into the flt_conf structure. The utility layer gained flt_otel_strtod() and flt_otel_strtoll() for validated string-to-number conversion used by rate-limit and debug-level parsing.
2026-04-12 02:41:03 -04:00
#include "event.h"
MEDIUM: otel: added configuration and utility layer Added the configuration structures that model the OTel filter's instrumentation hierarchy and the utility functions that support the configuration parser. The configuration is organized as a tree rooted at flt_otel_conf, which holds the proxy reference, filter identity, and lists of groups and scopes. Below it, flt_otel_conf_instr carries the instrumentation settings: tracer handle, rate limiting, hard-error mode, logging state, channel analyzers, and placeholder references to groups and scopes. Groups (flt_otel_conf_group) aggregate scopes by name. Scopes (flt_otel_conf_scope) bind an event to its ACL condition, span context declarations, span definitions and a list of spans scheduled for finishing. Spans (flt_otel_conf_span) carry attributes, events, baggages and status entries, each represented as flt_otel_conf_sample structures that pair a key with concatenated sample-expression arguments. All configuration types share a common header macro (FLT_OTEL_CONF_HDR) that embeds an identifier string, its length, a configuration line number, and a list link. Their init and free functions are generated by the FLT_OTEL_CONF_FUNC_INIT and FLT_OTEL_CONF_FUNC_FREE macros in conf_funcs.h, with per-type custom initialization and cleanup bodies. The utility layer in util.c provides argument counting and concatenation for the configuration parser, sample data to string conversion covering boolean, integer, IPv4, IPv6, string and HTTP method types, and debug helpers for dumping argument arrays and linked list state.
2026-04-12 01:30:57 -04:00
#include "conf.h"
#include "conf_funcs.h"
MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
#include "filter.h"
#include "group.h"
MEDIUM: otel: added HTTP header operations for context propagation Added the HTTP header manipulation layer that enables span context injection into and extraction from HAProxy's HTX message buffers, completing the end-to-end context propagation path. The new http.c module implements three public functions: flt_otel_http_headers_get() extracts HTTP headers matching a name prefix from the channel's HTX buffer into an otelc_text_map structure, stripping the prefix and separator dash from header names before storage; flt_otel_http_header_set() constructs a full header name from a prefix and suffix joined by a dash, removes all existing occurrences, and optionally adds the header with a new value; and flt_otel_http_headers_remove() removes all headers matching a given prefix. A debug-only flt_otel_http_headers_dump() logs all HTTP headers from a channel at NOTICE level. The scope runner in event.c now extracts propagation contexts from HTTP headers before processing spans: for each configured extract context, it calls flt_otel_http_headers_get() to read matching headers into a text map, then passes the text map to flt_otel_scope_context_init() which extracts the OTel span context from the carrier. After span execution, the span runner injects the span context back into HTTP headers via flt_otel_inject_http_headers() followed by flt_otel_http_header_set() for each propagation key. The unused resource cleanup in flt_otel_scope_free_unused() now also removes contexts that failed extraction by deleting their associated HTTP headers via flt_otel_http_headers_remove() before freeing the scope context structure.
2026-04-12 05:20:59 -04:00
#include "http.h"
#include "otelc.h"
MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
#include "parser.h"
MEDIUM: otel: added memory pool and runtime scope layer Added the memory pool management and the runtime scope layer that track per-stream OTel spans and contexts during request processing. The pool layer in pool.c manages HAProxy memory pools for the runtime structures used by the filter: scope spans, scope contexts, runtime contexts, and span contexts. Each pool is conditionally compiled via USE_POOL_OTEL_* macros defined in config.h and registered with REGISTER_POOL(). The allocation functions (flt_otel_pool_alloc, flt_otel_pool_strndup, flt_otel_pool_free) transparently fall back to heap allocation when the corresponding pool is not enabled. Trash buffer helpers (flt_otel_trash_alloc, flt_otel_trash_free) provide scratch space using either HAProxy's trash chunk pool or direct heap allocation. The scope layer in scope.c implements the per-stream runtime state. The flt_otel_runtime_context structure is allocated when a stream starts and holds the stream and filter references, hard-error/disabled/logging flags copied from the instrumentation configuration, idle timeout state, a generated UUID, and lists of active scope spans and extracted scope contexts. Scope spans (flt_otel_scope_span) carry the operation name, fetch direction, the OTel span handle, and optional parent references resolved from other spans or extracted contexts. Scope contexts (flt_otel_scope_context) hold an extracted span context obtained from a carrier text map via the tracer. The scope data structures (flt_otel_scope_data) aggregate growable key-value arrays for attributes and baggage, a linked list of named events with their own attribute arrays, and a span status code with description, representing the telemetry collected during a single event execution.
2026-04-12 03:47:18 -04:00
#include "pool.h"
#include "scope.h"
MEDIUM: otel: added configuration and utility layer Added the configuration structures that model the OTel filter's instrumentation hierarchy and the utility functions that support the configuration parser. The configuration is organized as a tree rooted at flt_otel_conf, which holds the proxy reference, filter identity, and lists of groups and scopes. Below it, flt_otel_conf_instr carries the instrumentation settings: tracer handle, rate limiting, hard-error mode, logging state, channel analyzers, and placeholder references to groups and scopes. Groups (flt_otel_conf_group) aggregate scopes by name. Scopes (flt_otel_conf_scope) bind an event to its ACL condition, span context declarations, span definitions and a list of spans scheduled for finishing. Spans (flt_otel_conf_span) carry attributes, events, baggages and status entries, each represented as flt_otel_conf_sample structures that pair a key with concatenated sample-expression arguments. All configuration types share a common header macro (FLT_OTEL_CONF_HDR) that embeds an identifier string, its length, a configuration line number, and a list link. Their init and free functions are generated by the FLT_OTEL_CONF_FUNC_INIT and FLT_OTEL_CONF_FUNC_FREE macros in conf_funcs.h, with per-type custom initialization and cleanup bodies. The utility layer in util.c provides argument counting and concatenation for the configuration parser, sample data to string conversion covering boolean, integer, IPv4, IPv6, string and HTTP method types, and debug helpers for dumping argument arrays and linked list state.
2026-04-12 01:30:57 -04:00
#include "util.h"
MEDIUM: otel: added HAProxy variable storage for context propagation Added support for storing OTel span context in HAProxy transaction variables as an alternative to HTTP headers, enabled by the OTEL_USE_VARS compile flag. The new vars.c module implements variable-based context propagation through the HAProxy variable subsystem. Variable names are constructed from a configurable prefix and the OTel propagation key, with dots normalized to underscores for HAProxy variable name compatibility and denormalized back during retrieval. The module provides flt_otel_var_register() to pre-register variables at parse time, flt_otel_var_set() and flt_otel_vars_unset() to store and clear context key-value pairs in the txn scope, flt_otel_vars_get() to collect all variables matching a prefix into an otelc_text_map for context extraction, and flt_otel_vars_dump() for debug logging of all OTel variables. The inject/extract keywords in the scope parser now accept an optional "use-vars" argument alongside "use-headers", controlled by the new FLT_OTEL_CTX_USE_VARS flag. Both storage types can be used simultaneously on the same span context, allowing context to be propagated through both HTTP headers and variables. The scope runner in event.c was extended to handle variable-based context in parallel with headers: during extraction, it reads matching variables via flt_otel_vars_get() when FLT_OTEL_CTX_USE_VARS is set; during injection, it stores each propagation key as a variable via flt_otel_var_set(). The unused resource cleanup now also unsets context variables when removing failed extraction contexts. The filter attach callback registers and sets the sess.otel.uuid variable with the generated session UUID, making the trace identifier available to HAProxy log formats and ACL expressions. The feature is conditionally compiled: the OTEL_USE_VARS flag controls whether vars.c is included in the build and whether the "use-vars" keyword is available in the configuration parser.
2026-04-12 05:25:14 -04:00
#include "vars.h"
MEDIUM: otel: added OpenTelemetry filter skeleton The OpenTelemetry (OTel) filter enables distributed tracing of requests across service boundaries, export of metrics such as request rates, latencies and error counts, and structured logging tied to trace context, giving operators a unified view of HAProxy traffic through any OpenTelemetry-compatible backend. The OTel filter is implemented using the standard HAProxy stream filter API. Stream filters attach to proxies and intercept traffic at each stage of processing: they receive callbacks on stream creation and destruction, channel analyzer events, HTTP header and payload processing, and TCP data forwarding. This allows the filter to collect telemetry data at every stage of the request/response lifecycle without modifying the core proxy logic. This commit added the minimum set of files required for the filter to compile: the addon Makefile with pkg-config-based detection of the opentelemetry-c-wrapper library, header files with configuration constants, utility macros and type definitions, and the source files containing stub filter operation callbacks registered through flt_otel_ops and the "opentelemetry" keyword parser entry point. The filter uses the opentelemetry-c-wrapper library from HAProxy Technologies, which provides a C interface to the OpenTelemetry C++ SDK. This wrapper allows HAProxy, a C codebase, to leverage the full OpenTelemetry observability pipeline without direct C++ dependencies in the HAProxy source tree. https://github.com/haproxytech/opentelemetry-c-wrapper https://github.com/open-telemetry/opentelemetry-cpp Build options: USE_OTEL - enable the OpenTelemetry filter OTEL_DEBUG - compile the filter in debug mode OTEL_INC - force the include path to the C wrapper OTEL_LIB - force the library path to the C wrapper OTEL_RUNPATH - add the C wrapper RUNPATH to the executable Example build with OTel and debug enabled: make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
2026-04-13 01:48:51 -04:00
#endif /* _OTEL_INCLUDE_H_ */
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*
* vi: noexpandtab shiftwidth=8 tabstop=8
*/