diff --git a/source/development/architecture.rst b/source/development/architecture.rst index c26dc712..1a4c52c6 100644 --- a/source/development/architecture.rst +++ b/source/development/architecture.rst @@ -22,7 +22,7 @@ This article describes how this will be achieved. High-level architecture ----------------------- -|OPNsense Components.png| +|OPNsense Components.svg| As the above model shows there are two main areas in our stack, the frontend implemented with PHP/Phalcon and the backend using a custom @@ -42,37 +42,13 @@ Manipulation of the core configuration file is handled at the frontend model; the backend service is merely a consumer of the information provided. --------------------- -Backend Architecture --------------------- -|OPNsense backend.png| - -Configd, is responsible -for the core system interaction like starting and stopping of daemons -and generating configuration files for used services and applications. - -The daemon listens on a unix domain socket and is capable of executing -actions defined in it’s own configuration directory -(“/usr/local/opnsense/service/conf/actions\_\*.conf”). - -Currently there are four types of services implemented in the daemon: - -- script : execute external (rc) scripts, report back success or failure -- script_output: execute external scripts, report back their contents, usually in json format -- stream_output: open streams to backend components -- inline : perform inline actions which are part of configd, most notable template generation and maintanance. - -| -| Template generation is handled by Jinja2 (http://jinja.pocoo.org/), - more information on how to create application templates can be found - at :doc:`/development/backend/templates`. --------------------- Frontend Architecture --------------------- -|OPNsense frontend.png| +|OPNsense frontend.svg| Routing ------- @@ -123,12 +99,71 @@ control and the repositories are split into 4 parts: | For detailed information about the development workflow see: | :doc:`OPNsense development workflow ` -.. |OPNsense Components.png| image:: images/OPNsense_Components.png + +-------------------- +Backend Architecture +-------------------- + + +Middleware +-------------------------------------- + +|OPNsense backend.svg| + +Configd, is responsible +for the core system interaction like starting and stopping of daemons +and generating configuration files for used services and applications. + +The daemon listens on a unix domain socket and is capable of executing +actions defined in it’s own configuration directory +(“/usr/local/opnsense/service/conf/actions\_\*.conf”). + +Currently there are four types of services implemented in the daemon: + +- script : execute external (rc) scripts, report back success or failure +- script_output: execute external scripts, report back their contents, usually in json format +- stream_output: open streams to backend components +- inline : perform inline actions which are part of configd, most notable template generation and maintanance. + +| +| Template generation is handled by Jinja2 (http://jinja.pocoo.org/), + more information on how to create application templates can be found + at :doc:`/development/backend/templates`. + + + +Operating stages +-------------------------------------- + +|OPNsense_operating_events.svg| + +As all earlier layers describe how user input can be persisted and service information can be exchanged, we still +have a gap in our functionality when looking at the life of a firewall. +Until this point we have means to manually manage services and devices, including ways to collect information on demand, +but still in a rather isolated way (events triggered via the api layer). + +This is where :doc:`syshooks ` and :doc:`plugins ` come into play, +these offer mechanisms to ensure different types of services can cooperate with the shared functionality available. + +Between starting and stopping our firewall, we can identify three stages. +After power-on, we are booting, when this initial stage has been reached we end up with a running firewall with +all configured services available. During time, various events can happen, for example, someone pulling an network cable +and pushing it back in, this is the running stage. +Eventually, if someone decides to power-down or reboot the machine, we are entering shutdown stage, letting services know +we are ending operation. + +Further details of each stage can be found in the :doc:`overview ` document. + +To avoid endless dependency loops, services should prevent hooking on events that are not strictly required for operating. +For example, forcing a restart of a component when a network interface has changed is usually a sign of not +following best practices for designing network services. + + +.. |OPNsense Components.svg| image:: images/OPNsense_Components.svg :width: 600px - :height: 548px -.. |OPNsense backend.png| image:: images/OPNsense_backend.png +.. |OPNsense backend.svg| image:: images/OPNsense_backend.svg + :width: 500px +.. |OPNsense frontend.svg| image:: images/OPNsense_frontend.svg :width: 600px - :height: 575px -.. |OPNsense frontend.png| image:: images/OPNsense_frontend.png - :width: 600px - :height: 461px +.. |OPNsense_operating_events.svg| image:: images/OPNsense_operating_events.svg + :width: 700px diff --git a/source/development/backend.rst b/source/development/backend.rst index 77691e18..86bc375b 100644 --- a/source/development/backend.rst +++ b/source/development/backend.rst @@ -9,7 +9,7 @@ Our core backend service (configd) is implemented using `Python ` document contains a practical write up to explain the various phases of +operation and hooks available in them. .. toctree:: :maxdepth: 2 diff --git a/source/development/backend/overview.rst b/source/development/backend/overview.rst index 913a5efd..41d6b5c0 100644 --- a/source/development/backend/overview.rst +++ b/source/development/backend/overview.rst @@ -32,7 +32,7 @@ in sequence: .................................... -Normal operation +Running stage .................................... Now the system is booted and events may take place, some of the common integration points with their purpose are listed below: diff --git a/source/development/images/OPNsense_Components.png b/source/development/images/OPNsense_Components.png deleted file mode 100644 index adbcf0ed..00000000 Binary files a/source/development/images/OPNsense_Components.png and /dev/null differ diff --git a/source/development/images/OPNsense_backend.png b/source/development/images/OPNsense_backend.png deleted file mode 100644 index a1054bd8..00000000 Binary files a/source/development/images/OPNsense_backend.png and /dev/null differ diff --git a/source/development/images/OPNsense_frontend.png b/source/development/images/OPNsense_frontend.png deleted file mode 100644 index ccf661a6..00000000 Binary files a/source/development/images/OPNsense_frontend.png and /dev/null differ diff --git a/source/development/images/OPNsense_operating_events.graphml b/source/development/images/OPNsense_operating_events.graphml new file mode 100644 index 00000000..84bde755 --- /dev/null +++ b/source/development/images/OPNsense_operating_events.graphml @@ -0,0 +1,476 @@ + + + + + + + + + + + + + + + + + + + + + + + + bootloader +/ +kernel stage + + + + + + + + + + + + + + + + + + + power on + + + + + + + + + + + + + + + + + + syshooks +[early] + + + + + + + + + + + + + + + + + + + plugins +(registration points) + + + + + + + + + + + + + + + + + + + syshooks [start] +-> rc(8) + + + + + + + + + + + + + + + + + + + running + + + + + + + + + + + + + + + + + + syshook / plugin +events + + + + + + + + + + + + + + + + + + + stop [power off] + + + + + + + + + + + + + + + + + + syshooks +[stop] + + + + + + + + + + + + + + + + + + + Simple scripts, executed early, also used by core +components to handle early-boot requirements + + + + + + + + + + + + + + + + + Structures pluggable core services, devices and +offers configure event handlers + + + + + + + + + + + + + + + + + Regular service startup point + + + + + + + + + + + + + + + + + bootup stage + + + + + + + + + + + running stage + + + + + + + + + + + shutdown stage + + + + + + + + + + + integration points for (device +related) events + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/source/development/images/OPNsense_operating_events.svg b/source/development/images/OPNsense_operating_events.svg new file mode 100644 index 00000000..b4dab716 --- /dev/null +++ b/source/development/images/OPNsense_operating_events.svg @@ -0,0 +1,286 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + bootloader + / + kernel stage + + + + + + + + + + power on + + + + + + + + + + syshooks + [early] + + + + + + + + + + plugins + (registration points) + + + + + + + + + + syshooks [start] + -> rc(8) + + + + + + + + + + running + + + + + + + + + + syshook / plugin + events + + + + + + + + + + stop [power off] + + + + + + + + + + syshooks + [stop] + + + + + + + + + + Simple scripts, executed early, also used by core + components to handle early-boot requirements + + + + + + + + + + Structures pluggable core services, devices and + offers configure event handlers + + + + + + + + + + Regular service startup point + + + + + + + bootup stage + + + + + + + running stage + + + + + + + shutdown stage + + + + + + + + + + integration points for (device + related) events + + + + + + + + + + + + + + + + + + + + + + + + + + +