opnsense-src/sys/opencrypto
Mark Johnston 49f6925ca3 ktls: Cache output buffers for software encryption
Maintain a cache of physically contiguous runs of pages for use as
output buffers when software encryption is configured and in-place
encryption is not possible.  This makes allocation and free cheaper
since in the common case we avoid touching the vm_page structures for
the buffer, and fewer calls into UMA are needed.  gallatin@ reports a
~10% absolute decrease in CPU usage with sendfile/KTLS on a Xeon after
this change.

It is possible that we will not be able to allocate these buffers if
physical memory is fragmented.  To avoid frequently calling into the
physical memory allocator in this scenario, rate-limit allocation
attempts after a failure.  In the failure case we fall back to the old
behaviour of allocating a page at a time.

N.B.: this scheme could be simplified, either by simply using malloc()
and looking up the PAs of the pages backing the buffer, or by falling
back to page by page allocation and creating a mapping in the cache
zone.  This requires some way to save a mapping of an M_EXTPG page array
in the mbuf, though.  m_data is not really appropriate.  The second
approach may be possible by saving the mapping in the plinks union of
the first vm_page structure of the array, but this would force a vm_page
access when freeing an mbuf.

Reviewed by:	gallatin, jhb
Tested by:	gallatin
Sponsored by:	Ampere Computing
Submitted by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D28556
2021-03-03 17:34:01 -05:00
..
_cryptodev.h OpenCrypto: Convert sessions to opaque handles instead of integers 2018-07-18 00:56:25 +00:00
cbc_mac.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
cbc_mac.h Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
criov.c crypto(9): add CRYPTO_BUF_VMPAGE 2020-08-26 02:37:42 +00:00
crypto.c Add an OCF algorithm for ChaCha20-Poly1305 AEAD. 2021-02-18 09:26:00 -08:00
cryptodeflate.c Replace some K&R function definitions with ANSI C. 2020-11-03 22:32:30 +00:00
cryptodev.c Add an OCF algorithm for ChaCha20-Poly1305 AEAD. 2021-02-18 09:26:00 -08:00
cryptodev.h Add an OCF algorithm for ChaCha20-Poly1305 AEAD. 2021-02-18 09:26:00 -08:00
cryptodev_if.m Refactor driver and consumer interfaces for OCF (in-kernel crypto). 2020-03-27 18:25:23 +00:00
cryptosoft.c cryptosoft: Support per-op keys for AES-GCM and AES-CCM. 2021-02-18 09:53:25 -08:00
deflate.h Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
gfmult.c
gfmult.h
gmac.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
gmac.h Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
ktls_ocf.c ktls: Cache output buffers for software encryption 2021-03-03 17:34:01 -05:00
rmd160.c Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
rmd160.h Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
xform.c Remove MD5 HMAC from OCF. 2020-05-11 22:08:08 +00:00
xform.h Remove MD5 HMAC from OCF. 2020-05-11 22:08:08 +00:00
xform_aes_icm.c Improve support for stream ciphers in the software encryption interface. 2020-05-22 16:29:09 +00:00
xform_aes_xts.c Include sys/types.h here 2020-09-15 15:21:29 +00:00
xform_auth.h Add an implementation of CHACHA20_POLY1305 to cryptosoft. 2021-02-18 09:26:23 -08:00
xform_cbc_mac.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
xform_chacha20_poly1305.c Add an implementation of CHACHA20_POLY1305 to cryptosoft. 2021-02-18 09:26:23 -08:00
xform_cml.c Various cleanups to the software encryption transform interface. 2020-05-20 21:21:01 +00:00
xform_comp.h Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
xform_deflate.c Replace some K&R function definitions with ANSI C. 2020-11-03 22:32:30 +00:00
xform_enc.h Add an OCF algorithm for ChaCha20-Poly1305 AEAD. 2021-02-18 09:26:00 -08:00
xform_gmac.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
xform_null.c Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
xform_poly1305.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
xform_poly1305.h Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
xform_rijndael.c Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
xform_rmd160.c Adjust crypto_apply function callbacks for OCF. 2020-06-10 21:18:19 +00:00
xform_sha1.c Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00
xform_sha2.c Consistently use C99 fixed-width types in the in-kernel crypto code. 2020-11-03 22:27:54 +00:00