3302c9b0a8
There is some race condition when the `async_write()`/`async_flush()` operation
for the `icinga::Hello` message fails (connection reset by peer for example)
around the same time the connect timeout fires and calls `cancel()` on the
stream, the following call to `async_shutdown()` may block indefinitely. If
that happens, the endpoint remains in the connecting state and no new
connection attemps are initiated.
This commit fixes the issue by removing the `Defer` containing the
`async_shutdown()`. The purpose of `async_shutdown()` is to signal a clean
termination of the connection to the peer, which really isn't something that
makes sense to to in a `Defer` block that is also executed in case of errors.
For the one situation where doing a clean TLS shutdown makes some sense
(closing anonymous client connections), a call to GracefulShutdown() is added
to that specific code path.
A large part of the change is just changing the indentation of the code, given
that a now unnecessary `try`/`catch` block is removed.
The following Go code creates a TLS server that can be used to demonstrate the
issue. Note that given that a race condition is involved, this is not reliable
and the sleep duration may need some fine-tuning. For this to work,
`ApiListener.tls_handshake_timeout` needs to be set to a large-enough value
like 60s to disable the timeout for `async_handshake()` itself so that the
overall connect timeout is the one that fires. However, changing the timeout is
not a prerequisite for the problem, it just makes it easier to reproduce. The
error can also happen with the default timeouts if the TCP connect takes long
enough so that the handshake is started late enough that its timeout expires
after the connect timeout.
package main
import (
"crypto/tls"
"log"
"net"
"time"
)
func main() {
cert, err := tls.LoadX509KeyPair("bad-agent.crt", "bad-agent.key")
if err != nil {
panic(err)
}
listener, err := tls.Listen("tcp", ":1337", &tls.Config{
Certificates: []tls.Certificate{cert},
})
if err != nil {
panic(err)
}
log.Println("Listening on", listener.Addr())
for {
conn, err := listener.Accept()
if err != nil {
panic(err)
}
go handle(conn.(*tls.Conn))
}
}
func handle(conn *tls.Conn) {
addr := conn.RemoteAddr().String()
log.Println(addr, "new connection")
time.Sleep(15*time.Second - 10*time.Millisecond)
log.Println(addr, "SetLinger(0)", conn.NetConn().(*net.TCPConn).SetLinger(0))
log.Println(addr, "Handshake()", conn.Handshake())
log.Println(addr, "conn.NetConn().Close()", conn.NetConn().Close())
}
With additional logging in the `catch` block for `boost::system::system_error`
and `Defer shutdownSslConn` (both removed by this commit), this showed the
following. Note that in particular, `async_shutdown()` never returned,
indicating that it hangs in there.
[2026-04-24 17:32:56 +0200] information/ApiListener: Reconnecting to endpoint 'bad-agent' via host 'host.docker.internal' and port '1337'
[2026-04-24 17:33:11 +0200] critical/ApiListener: Timeout while reconnecting to endpoint 'bad-agent' via host 'host.docker.internal' and port '1337', cancelling attempt
[2026-04-24 17:33:11 +0200] information/ApiListener: New client connection for identity 'bad-agent' to [172.17.0.1]:1337
[2026-04-24 17:33:12 +0200] information/ApiListener: rethrowing for bad-agent: Error: Connection reset by peer [system:104 at /usr/include/boost/asio/detail/reactive_socket_send_op.hpp:137 in function 'do_complete']
[2026-04-24 17:33:12 +0200] information/ApiListener: doing async_shutdown for bad-agent
|
||
|---|---|---|
| .github | ||
| agent | ||
| choco | ||
| cmake | ||
| doc | ||
| etc | ||
| icinga-app | ||
| icinga-installer | ||
| itl | ||
| lib | ||
| plugins | ||
| test | ||
| third-party | ||
| tools | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| AUTHORS | ||
| CHANGELOG.md | ||
| CMakeLists.txt | ||
| config.h.cmake | ||
| Containerfile | ||
| CONTRIBUTING.md | ||
| icinga-spec-version.h.cmake | ||
| icinga-version.h.cmake | ||
| ICINGA2_VERSION | ||
| LICENSE.md | ||
| mkdocs.yml | ||
| NEWS | ||
| publiccode.yml | ||
| README.md | ||
Icinga 2
Table of Contents
About
Icinga is a monitoring system which checks the availability of your network resources, notifies users of outages, and generates performance data for reporting.
Scalable and extensible, Icinga can monitor large, complex environments across multiple locations.
Icinga 2 is the monitoring server and requires Icinga Web 2 on top in your Icinga Stack. The configuration can be easily managed with either the Icinga Director, config management tools or plain text within the Icinga DSL.
Installation
- Installation
- Monitoring Basics
- Configuration
- Distributed Monitoring
- Addons, Integrations and Features
- Troubleshooting
- Upgrading
Once Icinga Server and Web are running in your distributed environment, make sure to check out the many Icinga modules for even better monitoring.
Documentation
The documentation is available on icinga.com/docs.
Support
Check the project website for status updates. Join the community channels for questions or ask an Icinga partner for professional support.
License
Icinga 2 and the Icinga 2 documentation are licensed under the terms of the GNU
General Public License Version 3 or later, you will find a copy of this license in the
LICENSE.md file included in the source package.
In addition, as a special exception, the copyright holders give permission to link the code of portions of this program with the OpenSSL library under certain conditions as described in each individual source file, and distribute linked combinations including the two.
You must obey the GNU General Public License in all respects for all of the code used other than OpenSSL. If you modify file(s) with this exception, you may extend this exception to your version of the file(s), but you are not obligated to do so. If you do not wish to do so, delete this exception statement from your version. If you delete this exception statement from all source files in the program, then also delete it here.
Note
Historically, Icinga 2 has been licensed under the GNU General Public License Version 2 or later. However, due to newly introduced dependencies licensed under the Apache License 2.0 (which is not compatible with GPLv2), we have decided to upgrade the license of Icinga 2 to GPLv3+, effective from version
v2.16.0onwards. All versions prior tov2.16.0and all existing source code files remain licensed under GPLv2+ (see the license information in those files).Also, the OpenSSL linking exception is only relevant for OpenSSL 1.x. OpenSSL >= 3.0 is licensed under the Apache License version 2.0, which is compatible with GPLv3 and no longer requires an exception.
Contributing
There are many ways to contribute to Icinga -- whether it be sending patches, testing, reporting bugs, or reviewing and updating the documentation. Every contribution is appreciated!
Please continue reading in the contributing chapter.
If you are a packager, please read the development chapter for more details.
Security Issues
For reporting security issues please visit this page.