The illumos-specific _SC_CPUID_MAX is the largest CPU ID in the system. This
was mapped to _SC_NPROCESSORS_CONF, which is the total number of CPUs recognized
by the kernel. If CPU IDs are contiguous, as is the case on amd64 and arm64,
this value is one greater than the maximum ID. As a result, when consuming
per-CPU dtrace buffers, libdtrace tries to fetch from a non-existent CPU. This
is mostly harmless in practice, but still wrong.
As we don't have a sysconf value for the maximum CPU ID, add a wrapper which
fetches it using the kern.smp.maxid sysctl.
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D49243
(cherry picked from commit 9a30c8d347bf9aaa89277b6e5a275f737be8edce)
The zero'th index in the array is unused, so a priority queue of N elements
needs N+1 array slots. Fix the allocation.
Also fix the assertion in dt_pq_insert(): the assertion needs to be checked
after incrementing the count of items in the priority queue, otherwise it can
miss an overflow.
Reported by: CHERI
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D49242
(cherry picked from commit 7ee1bdd094d376fdc547e8ca33e472f1d37a7d79)
Make it easier to support data models other than ILP32 and LP64 by
avoiding constructs which assume that it must be one or the other.
No functional change intended.
MFC after: 2 weeks
Sponsored by: Innovate UK
(cherry picked from commit 096a5c6cd28c417456d5ce3598be15e6b656af5c)
If one of two pointers refers to a forward declaration, let the pointers
be compatible so long as the referred types have the same name.
Otherwise we can get spurious errors.
To give a specific example, this can happen when ipfw_nat.ko is loaded
before ipfw.ko and /usr/lib/dtrace/ipfw.d is processed. Currently,
ipfw_nat.ko does not have a definition for struct inpcb (i.e., none of
its files include in_pcb.h), so in the CTF type graph, struct
ip_fw_args' "inp" member refers to a forward declaration, represented in
CTF with CTF_K_FORWARD.
Then, when libdtrace processes the ipfw_match_info_t translator in
ipfw.d, it decides that the "inp" field assignment is incorrect because
the two pointers are incompatible. However, there's no harm in allowing
this assignment. Add some logic to dt_node_is_ptrcompat() to detect
this case and declare the pointers as compatible so long as the name of
the thing they refer to is the same, similar to how any pointer is
compatible with a void *.
Reported by: marck
Reviewed by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D48254
(cherry picked from commit 4196f227aa84448ef3cae9e99678fbe787726cf7)
OpenZFS release 2.2.7
Notable upstream pull request merges:
#15457022bf8637 Increase L2ARC write rate and headroom
#155271ba5b2ef8 freebsd: remove __FBSDID macro use
#157929e7af55e8 Add custom debug printing for your asserts
#15793a1ea40af8 libzfs: use zfs_strerror() in place of strerror()
#16181 -multiple zdb/ztest: improve and harmonise crash output
#16208e5e4957a5 Allow block cloning to be interrupted by a signal
#16210ba4e582a6 FreeBSD: Add const qualifier to members of struct
opensolaris_utsname
#16225a6198f34b Simplify issig()
#1633525ec9a903 zdb: fix BRT dump
#16364cf80a803d zvol: ensure device minors are properly cleaned up
#16404384b53be8 FreeBSD: Add missing memory reclamation accounting
#16492 -multiple Split "shared" userspace & kernel code into separate files
#16496f1694496a zfs_file: rename zfs_file_fallocate to zfs_file_deallocate
#16511bc0d89bfc Fix an uninitialized data access
#165292dc8529d9 Fix handling of DNS names with '-' in them for sharenfs
#1653930ea44296 zfs_log: add flex array fields to log record structs
#16546098789216 Evicting too many bytes from MFU metadata
#1655154278533a Reduce and handle EAGAIN errors on AIO label reads
#1655484ad1d536 FreeBSD: restore zfs_znode_update_vfs()
#1656521c40e6d9 FreeBSD: Sync taskq_cancel_id() returns with Linux
#1656748482bb2f Properly release key in spa_keystore_dsl_key_hold_dd()
#16584e8f4592a1 Avoid computing strlen() inside loops
#16605acc8a3186 ARC: Cache arc_c value during arc_evict()
#16650fc60e0c6e freebsd: Use compiler.h from FreeBSD's base's linuxkpi
#16667b32b35cea zdb: show bp in uberblock dump
#166841f5e1b919 Pack dmu_buf_impl_t by 16 bytes
#1668873b3e8ace Fix gcc uninitialized warning in FreeBSD zio_crypt.c
#16690727506c94 On the first vdev open ignore impossible ashift hints
#16692d83cd5307 zdb: add extra -T flag to show histograms of BRT refcounts
#1669382ab837a0 Fix gcc unused value warning in FreeBSD simd.h
#167402bba6e3c5 BRT: Don't call brt_pending_remove() on holes/embedded
#16801299da6ace Fix race in libzfs_run_process_impl
Obtained from: OpenZFS
OpenZFS commit: e269af1b3c
OpenZFS tag: zfs-2.2.7
These were reported by `mandoc -T lint ...` as errors; this commit only
handles unnecessary .El commands.
The rendered output (in ascii and html) is not affected by this commit.
Signed-off-by: Graham Percival <gperciva@tarsnap.com>
Reviewed by: mhorne
MFC after: 3 days
Sponsored by: Tarsnap Backup Inc.
Pull Request: https://github.com/freebsd/freebsd-src/pull/1447
(cherry picked from commit c801836ff43deea3c86935abcad1069bfcd3989c)
Without this patch, vdevs faulted via AUX state would not be replaced
once the appropriate drive is replaced. ZFS does not internally use
that state, but a drive can be manually forced into such a state with a
command like zinject.
Submitted by: Goran Mekić <meka@tilda.center>
Sponsored by: ConnectWise
Differential Revision: https://reviews.freebsd.org/D46866
(cherry picked from commit 618ae939b7f95a197a0518cbb390a572bb90c6d0)
OpenZFS upstream refactored the asm versions of sha2 to be usable on all
32-bit arm flavors, so it is not necessary to limit this to armv6 and
armv7.
Suggested by: jhb
(cherry picked from commit bc6051a060f7f39540e284f2934e61411590dbfd)
Similar to fb17dfa0c83c, fix libicp_rescue to include asm versions of
sha2 on armv6, to unbreak the build of rescue.
Fixes: 3494f7c019fc
(cherry picked from commit 1a4a9a50574d9b4b4db90a85bc253d340c93a8a0)
The following upstream commit:
727497ccdfcc module/icp/asm-arm/sha2: enable non-SIMD asm kernels on armv5/6
does indeed enable sha2 asm for earlier arm CPUs, but since libicp's
Makefile was not updated, this leads to:
ld: error: undefined reference due to --no-allow-shlib-undefined: zfs_sha256_block_armv7
Fix it by compiling sha256-armv7.S and sha512-armv7.S for armv6 too.
Fixes: 3494f7c019fc
(cherry picked from commit fb17dfa0c83cc213400fe7e1ed7a39253a4fcefa)
The use of an ifdef here most likely carries over from when the dtrace
port only worked on x86 platforms.
MFC after: 2 weeks
Sponsored by: Innovate UK
(cherry picked from commit 7bbcbd43c53b49360969ca82b152fd6d971e9055)
When compiling dt_lex.l, flex produces warnings of the form:
dt_lex.l:413: warning, trailing context made variable due to preceding '|' action
dt_lex.l:412: warning, dangerous trailing context
dt_lex.l:412: warning, dangerous trailing context
Here, trailing context refers to the use of "$", which expands to "/\n".
The meaning behind these warnings is described in the first two
paragraphs of the flex manual's DEFICIENCIES/BUGS section:
Some trailing context patterns cannot be properly matched and generate
warning messages ("dangerous trailing context"). These are patterns
where the ending of the first part of the rule matches the beginning of
the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at
the beginning of the trailing context. (Note that the POSIX draft
states that the text matched by such patterns is undefined.)
For some trailing context rules, parts which are actually fixed-length
are not recognized as such, leading to the above mentioned performance
loss. In particular, parts using '|' or {n} (such as "foo{3}") are
always considered variable-length.
Here, the warnings appear to be bogus in this case. The lexer has no
problem matching either of the referenced patterns, e.g.,
printf("foobar
or
# 1 "asdfasdf
Introduce a small amount of code duplication to silence the warning.
MFC after: 2 weeks
(cherry picked from commit 4bddff0833d3efee77a099b3ef447fbae1e63d21)
The old way is racy and can cause two instances, running in parallel, to
attempt to load dtrace_test, and only one will succeed. This caused
errors when running dtrace tests in parallel.
MFC after: 1 week
(cherry picked from commit cb6b5e8ab9cfe3938855826fa3de0f58dacd364e)
ZED uses vdev props for setting disk fault/degrade thresholds, this
patch enables zfsd to use the same vdev props for these same tasks.
OpenZFS on Linux is using vdev props for ZED disk fault/degrade
thresholds. Originally the thresholds supported were for io and checksum
events and recently this was updated to process slow io events as
well, see
cbe882298e
This patch enables us to use the same vdev props in zfsd as ZED uses.
After this patch is merged both OSs will use the same vdev props to set
retirement thresholds.
It's probably important to note that the threshold defaults are
different between OS. I've kept the existing defaults inside zfsd and
DID NOT match them to what ZED does.
Differential Revision: https://reviews.freebsd.org/D44043
Relnotes: yes
Reviewed by: asomers, allanjude
Sponsored by: Axcient
Submitted by: Alek Pinchuk <apinchuk@axcient.com>
(cherry picked from commit 89f4f91dbfdcabe65bc7476bc5f13dfb837870fe)
zfsd: fix unit tests after 89f4f91dbfdcabe65bc7476bc5f13dfb837870fe
Reported by: markj
Sponsored by: Axcient
Reviewed by: Alek Pinchuk <pinchuk.alek@gmail.com>
Differential Revision: https://reviews.freebsd.org/D44744
(cherry picked from commit 25038e8de6b4e5f2ffca821565b50a633eea499a)
This is similar to other translators and will be used in static
probes where the interface is not known.
Reviewed by: markj
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D43728
(cherry picked from commit e00fd24262945fdc7f16198c8abd977f743e66e4)
This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)
This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:
- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.
This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.
Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745
(cherry picked from commit 93f27766a7e1af009c5b1e4ca538632857c91aa1)
If there are no CTF sections then ctfmerge just has nothing to do; it
should not be an error.
Note that ctfmerge has an option to require CTF:
-t Make sure that all object files have a CTF section.
Before this change, this option explicitly exited without error if none
of the object files have CTF sections, with the comment:
If we're verifying that C files have CTF, it's safe to
assume that in this case, we're building only from assembly
inputs.
PR: 276930
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43878
(cherry picked from commit 95ca89cda1a6c4e0ef0b3f765c6563f1db0d23fa)
If ZFS reports that a disk had at least 8 I/O operations over 60s that
were each delayed by at least 30s (implying a queue depth > 4 or I/O
aggregation, obviously), fault that disk. Disks that respond this
slowly can degrade the entire system's performance.
Sponsored by: Axcient
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42825
(cherry picked from commit d565784a7ebaa59e26febdcfd4a60329786ea5f5)
A number of tests use it, so just require it globally as we do with
other tools.
MFC after: 1 week
(cherry picked from commit 66bfb4b2ee49e435152316434ae322d8f3f19d3d)
In particular, avoid loading the user's .profile file, since that can
have undesirable side effects. Most tests were already careful to do
this.
MFC after: 1 week
(cherry picked from commit e25922b34ed2b7b4bf0ae4de2f7ec82b04153384)
This attribute is new in DWARF 4 and supersedes DW_AT_bit_offset.
PR: 276059
Reported by: rscheff
Tested by: rscheff
MFC after: 1 week
(cherry picked from commit 78cd75393ec79565c63927bf200f06f839a1dc05)
The header gives an offset in 32-bit words, and the translator is
supposed to convert that to a byte count. But, the conversion was
incorrect.
Reviewed by: tuexen, rscheff
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43264
(cherry picked from commit c3268c23de45b75b44102a02e96f44b0ff2f6ba7)
OpenZFS release 2.2.1
Notable upstream pull request merges:
#143786d693e20a Large sync writes perform worse with slog
#1524378fd79eac Add zfs_prepare_disk script for disk firmware install
#15356b76724ae4 FreeBSD: Improve taskq wrapper
#15360f9a9aea12 Add mutex_enter_interruptible() for interruptible
sleeping IOCTLs
#15371e82e68400 DMU: Do not pre-read holes during write
#1539079f7de575 Remove lock from dsl_pool_need_dirty_delay()
#153976e41aca51 Trust ARC_BUF_SHARED() more
#15402eaa62d995 Properly pad struct tx_cpu to cache line
#154051cc1bf4fa Set spa_ccw_fail_time=0 when expanding a vdev
#15416edebca5df FreeBSD: taskq: Remove unused declaration
#154517aef672b7 Read prefetched buffers from L2ARC
#15452bd7a02c25 Tune zio buffer caches and their alignments
#154563ec4ea68d Unify arc_prune_async() code
#15465459c99ff2 Fix block cloning between unencrypted and encrypted
datasets
#1547876663fe37 Fix accounting error for pending sync IO ops in zpool
iostat
#1552987e9e8286 Add a tunable to disable BRT support
Obtained from: OpenZFS
OpenZFS commit: 55dd24c4cc
OpenZFS tag: zfs-2.2.1
Previously zfsd would crash in the presence of a pool with a
top-level-vdev that had previously been removed. The crash happened
because the configuration nvlist of such a TLV contains an empty
ZPOOL_CONFIG_CHILDREN array, which led to a pop_front from an empty
list, which has undefined behavior.
The crash only happened in stable/14 and later, probably do to
differences in libcxx, but the change should be MFCed anyway.
PR: 273663
Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
Sponsored by: Axcient
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D41818
(cherry picked from commit 0b294a386d34f6584848ed52407687df7ae59861)
$FreeBSD$ is gone in head, so this should be removed as well.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D41595
(cherry picked from commit 7a3af0a59b56078e88b8009824fdd99e8df3f625)
Approved by: re (cperciva@)
Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this
vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15161 Make zoned/jailed zfsprops(7) make more sense
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress
#15216 Relax error reporting in zpool import and zpool split
#15227 ZIL: Tune some assertions
#15228 ZIL: Revert zl_lock scope reduction
#15233 ZIL: Change ZIOs issue order
Obtained from: OpenZFS
OpenZFS commit: 32949f2560
Approved by: re (gjb)
libproc's PR_REQUESTED is not implemented on FreeBSD. Remove dead code
in dtrace that would handle it.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41225
