opnsense-src/sys
Conrad Meyer eefd8f96fb geom_uzip(4), mkuzip(8): Add Zstd image mode
The Zstd format bumps the CLOOP major number to 4 to avoid incompatibility
with older systems.  Support in geom_uzip(4) is conditional on the ZSTDIO
kernel option, which is enabled in amd64 GENERIC, but not all in-tree
configurations.

mkuzip(8) was modified slightly to always initialize the nblocks + 1'th
offset in the CLOOP file format.  Previously, it was only initialized in the
case where the final compressed block happened to be unaligned w.r.t.
DEV_BSIZE.  The "Fake" last+1 block change in r298619 means that the final
compressed block's 'blen' was never correct unless the compressed uzip image
happened to be BSIZE-aligned.  This happened in about 1 out of every 512
cases.  The zlib and lzma decompressors are probably tolerant of extra trash
following the frame they were told to decode, but Zstd complains that the
input size is incorrect.

Correspondingly, geom_uzip(4) was modified slightly to avoid trashing the
nblocks + 1'th offset when it is known to be initialized to a good value.
This corrects the calculated final real cluster compressed length to match
that printed by mkuzip(8).

mkuzip(8) was refactored somewhat to reduce code duplication and increase
ease of adding other compression formats.

  * Input block size validation was pulled out of individual compression
    init routines into main().

  * Init routines now validate a user-provided compression level or select
    an algorithm-specific default, if none was provided.

  * A new interface for calculating the maximal compressed size of an
    incompressible input block was added for each driver.  The generic code
    uses it to validate against MAXPHYS as well as to allocate compression
    result buffers in the generic code.

  * Algorithm selection is now driven by a table lookup, to increase ease of
    adding other formats in the future.

mkuzip(8) gained the ability to explicitly specify a compression level with
'-C'.  The prior defaults -- 9 for zlib and 6 for lzma -- are maintained.
The new zstd default is 9, to match zlib.

Rather than select lzma or zlib with '-L' or its absense, respectively, a
new argument '-A <algorithm>' is provided to select 'zlib', 'lzma', or
'zstd'.  '-L' is considered deprecated, but will probably never be removed.

All of the new features were documented in mkuzip.8; the page was also
cleaned up slightly.

Relnotes:	yes
2019-08-13 23:32:56 +00:00
..
amd64 sys/{x86,amd64}: remove one of doubled ;s 2019-08-13 19:39:36 +00:00
arm Floppy driver really only works on x86 2019-08-12 22:58:50 +00:00
arm64 ukbdmap.h rule was identical on all platforms, so move them into sys/conf/files. 2019-08-12 23:25:14 +00:00
bsm Create new EINTEGRITY error with message "Integrity check failed". 2019-01-17 06:35:45 +00:00
cam Make camcontrol modepage support block descriptors. 2019-08-07 14:45:10 +00:00
cddl MFV r350896: 6585 sha512, skein, and edonr have an unenforced dependency on extensible dataset 2019-08-12 11:42:16 +00:00
compat Fix build with DRM and INVARIANTS enabled. 2019-08-13 21:15:59 +00:00
conf geom_uzip(4), mkuzip(8): Add Zstd image mode 2019-08-13 23:32:56 +00:00
contrib Initialize the frentry (the control block that defines a rule) checksum 2019-08-12 02:42:47 +00:00
crypto armv8crypto: add missing unlock for session (after r336439) 2019-08-08 14:34:53 +00:00
ddb Cache kernel stacks in UMA. This gives us NUMA support, better concurrency, 2019-08-06 23:15:34 +00:00
dev Make nvd(4) report NGUID or EUI64 as GEOM::lunid. 2019-08-13 21:49:07 +00:00
dts dtso: allwinner: Add an overlay for H3 i2c0 2019-07-20 17:42:46 +00:00
fs fusefs: merge from projects/fuse2 2019-08-07 00:38:26 +00:00
gdb
geom geom_uzip(4), mkuzip(8): Add Zstd image mode 2019-08-13 23:32:56 +00:00
gnu arm: dts: am33xx: Fix the region for uart0 2019-08-07 13:11:53 +00:00
i386 Start to split out the really x86 specific NOTES from the global notes file. 2019-08-12 22:58:13 +00:00
isa
kern Move scheduler state into the per-cpu area where it can be allocated on the 2019-08-13 04:54:02 +00:00
kgssapi Make the warning intervals for deprecated crypto algorithms tunable. 2019-06-11 23:00:55 +00:00
libkern Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib, 2019-08-01 06:35:33 +00:00
mips Avoid relying on header pollution from sys/refcount.h. 2019-07-29 20:26:01 +00:00
modules geom_uzip(4), mkuzip(8): Add Zstd image mode 2019-08-13 23:32:56 +00:00
net Properly validte arguments for route deletion 2019-08-03 14:42:07 +00:00
net80211 Enhance the comment ieee80211_add_channel() to avoid a 2019-06-10 14:31:18 +00:00
netgraph Remove 'dir' argument in ng_ipfw_input, since ip_fw_args now has this info. 2019-03-14 22:30:05 +00:00
netinet Save ip_ttl value and restore it after checksum calculation. 2019-08-13 12:47:53 +00:00
netinet6 frag6: move public structure into file local space. 2019-08-08 10:59:54 +00:00
netipsec Add missing new line in several log messages. 2019-08-09 08:58:09 +00:00
netpfil pf: zero (another) output buffer in pfioctl 2019-07-31 16:58:09 +00:00
netsmb Remove unused argument to priv_check_cred. 2018-12-11 19:32:16 +00:00
nfs
nfsclient
nfsserver
nlm
ofed Fix prio vs. nonprio tagged traffic in RDMACM 2019-06-04 06:21:31 +00:00
opencrypto Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib, 2019-08-01 06:35:33 +00:00
powerpc Fix powerpc LINT build 2019-08-12 23:25:21 +00:00
riscv riscv: Fix copyin/copyout 2019-07-29 14:59:14 +00:00
rpc Avoid relying on header pollution from sys/refcount.h. 2019-07-29 20:26:01 +00:00
security Fix mac_veriexec_parser build after r347938 2019-08-08 16:51:49 +00:00
sparc64 Floppy driver really only works on x86 2019-08-12 22:58:50 +00:00
sys Remove deprecated GEOM classes 2019-08-13 20:06:55 +00:00
teken Attempt to complete fixing programmable function keys for syscons. 2019-02-20 02:14:41 +00:00
tests Regularize the Netflix copyright 2019-02-04 21:28:25 +00:00
tools Add SDIO support. 2019-06-08 16:26:56 +00:00
ufs Clarify comment that describes how the FS_METACKHASH is managed. 2019-08-13 20:56:44 +00:00
vm Fix stack grow for init. 2019-08-08 16:48:19 +00:00
x86 sys/{x86,amd64}: remove one of doubled ;s 2019-08-13 19:39:36 +00:00
xdr
xen xen: introduce a new way to setup event channel upcall 2019-01-30 11:34:52 +00:00
Makefile