The tag length is included as one of the values in the flags byte of
block 0 passed to CBC_MAC, so merely copying the first N bytes is
insufficient.
To avoid adding more sideband data to the CBC MAC software context,
pull the generation of block 0, the AAD length, and AAD padding out of
cbc_mac.c and into cryptosoft.c. This matches how GCM/GMAC are
handled where the length block is constructed in cryptosoft.c and
passed as an input to the Update callback. As a result, the CBC MAC
Update() routine is now much simpler and simply performs the
XOR-and-encrypt step on each input block.
While here, avoid a copy to the staging block in the Update routine
when one or more full blocks are passed as input to the Update
callback.
Reviewed by: sef
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32120
Rather than copying crp_iv to a local array on the stack that is then
passed to xform reinit routines, pass crp_iv directly and remove the
local copy.
Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32106
Add a 'len' argument to the reinit hook in 'struct enc_xform' to
permit support for AEAD ciphers such as AES-CCM and Chacha20-Poly1305
which support different nonce lengths.
Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32105
While here, use crypto_read_iv() in a few more places in ccr(4) that I
missed previously.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32104
Drop arguments of function prototypes since the file is mixed between
listing arg names and not.
No functional changes
Reviewed by: markj
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D32329
In some contexts it is illegal to wait for memory allocation, causing
kernel panic. By default sbuf_new passes M_WAITOK to malloc,
which caused crashes when sdhci_dumpcaps or sdhci_dumpregs was callend in
non sutiable context.
Add SBUF_NOWAIT flag to sbuf_new to fix this.
Obtained from: Semihalf
Differential revision: https://reviews.freebsd.org/D32075
This is the same underlying problem as 2624598064, just for bus ranges
rather than windows. SiFive's HiFive Unmatched has the following
topology:
Root Port <---> Bridge <---> Bridge <-+-> Bridge <---> (Unused)
(pcib0) (pcib1) (pcib2) | (pcib3)
+-> Bridge <---> xHCI
| (pcib4)
+-> Bridge <---> M.2 E-key
| (pcib5)
+-> Bridge <---> M.2 M-key
| (pcib6)
+-> Bridge <---> x16 slot
(pcib7)
If a device is plugged into the x16 slot that itself has a bridge, such
as many graphics cards, we currently fail to allocate a bus number for
its child bus (and so pcib_attach_child skips adding a child bus for
further enumeration) as, when the new child bridge attaches, it attempts
to allocate a bus number from its parent (pcib7) which in turn attempts
to grow its own bus range by calling bus_adjust_resource on its own
parent (pcib2) whose bus rman cannot accommodate the request and needs
to itself be extended by calling its own parent (pcib1). Note that
pcib3-7 do not face the same issue when they attach since pcib1 ends up
managing bus numbers 1-255 from the beginning and so never needs to grow
its own range.
Reviewed by: jhb, mav
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32011
We only use nvme_completion_poll in the initialization path. The
commands they queue and wait for finish quickly as they involve no I/O
to the drive's media. These command take about 20-200 microsecnds
each. Set the wait time to 1us and then increase it by 1.5 each
successive iteration (max 1ms). This reduces initialization time by
80ms in cpervica's tests.
Use this same technique waiting for RDY state transitions. This saves
another 20ms. In total we're down from ~330ms to ~2ms.
Tested by: cperciva
Sponsored by: Netflix
Reviewed by: mav
Differential Review: https://reviews.freebsd.org/D32259
The FreeBSD nvme driver has reset the nvme controller twice on attach to
address a theoretical issue assuring the hardware is in a known
state. However, exierence has shown the second reset is unnecessary and
increases the time to boot. Eliminate the second reset. Should there be
a situation when you need a second reset (for buggy or at least somewhat
out of the mainstream hardware), the hardware option NVME_2X_RESET will
restore the old behavior. Document this in nvme(4).
If there's any trouble at all with this, I'll add a sysctl tunable to
control it.
Sponsored by: Netflix
Reviewed by: cperciva, mav
Differential Revision: https://reviews.freebsd.org/D32241
After some study of the code and the standard, I think we can just drop
the pause(), unconditionally. If we're not initialized, then there's
nothing to wait for from a software perspective. If we are initialized,
then there might be outstanding I/O. If so, then the qpair 'recovery
state' will transition to WAITING in nvme_ctrlr_disable_qpairs, which
will ignore any interrupts for items that complete before we complete
the reset by setting cc.en=0.
If we go on to fail the controller, we'll cancel the outstanding I/O
transactions. If we reset the controller, the hardware throws away
pending transactions and we retry all the pending I/O transactions. Any
transactions that happend to complete before cc.en=0 will have the same
effect in the end (doing the same transaction twice is just inefficient,
it won't affect the state of the device any differently than having done
it once).
The standard imposes no wait times here, so it isn't needed from that
perspective.
Unanswered Question: Do we may need to disable interrupts while we
disable in legacy mode since those are level-sensitive.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D32248
The don't touch the mmio of the drive after we do a EN 1->0 transition
is only for a tiny number of dirves that have this unforunate issue.
Sponsored by: Netflix
Rewrite the nested if's using the preferred FreeBSD style for branches
of ifs that return. NFC. Minor tweaks to the comments to better fit new
code layout.
Sponsored by: Netflix
Reviewed by: mav, chuck (prior rev, but comments rolled in)
Differential Revision: https://reviews.freebsd.org/D32245
Remove the 5ms delays after writing the administrative queue
registers. These delays are from the very earliest days of the driver
(they are in the first commit) and were most likely vestiges of the
Chatham NVMe prototype card that was used to create this driver. Many of
the workarounds necessary for it aren't necessary for standards
compliant cards. The original driver had other areas marked for Chatham,
but these were not. They are unneeded. There's three lines of supporting
evidence.
First, the NVMe standards make no mention of a delay time after these
registers are written. Second, the Linux driver doesn't have them, even
as an option. Third, all my nvme cards work w/o them.
To be safe, add a write barrier between setting up the admin queue and
enabling the controller.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D32247
Use correct devclass name, due to the mismatch miibus would attach
to the wrong thing causing mii_attach to silently fail.
Fixes: dfcaa2c18b (enetc_mdio: Support building the driver ...)
Match the PCI simple comm devices (or try to). Be conservative and use
legacy interrupts rather than msi messages by default for this 'catch
all' since it matches what Linux does (it has opt-in generally for MSI,
but also matches more devices because it does a catch-all like
implemented in this commit).
Sponsored by: Netflix
Reviewed by: kbowling
Differential Revision: https://reviews.freebsd.org/D32228
Some setups claim to have one MSI, but they don't actually work. Allow
these to be flagged.
Sponsored by: Netflix
Reviewed by: kbowling
Differential Revision: https://reviews.freebsd.org/D32229
Previously mgb_admin_intr printed a diagnostic message if no interrupt
status bits were set, but it's not valid to call device_printf() from a
filter. Just drop the message as it has no user-facing value.
Also return FILTER_STRAY in this case - there is nothing further for
the driver to do.
Reviewed by: kbowling
MFC after: 1 week
Fixes: 8890ab7758 ("Introduce if_mgb driver...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32231
Looking for "tsc-tsc" in the pmu tables will fail every time. Instead,
make this an alias for the static TSC event defined in pmc_events.h.
This fixes 'pmcstat -s cycles' on Intel and AMD.
Reviewed by: emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32197
Issue: Devices wont go away after the link down.
Device lost timer functionality in ocs_fc is broken,
`is_target` flag is not set in the target database and target delete is skipped.
Fix: Remove unused flags and delete the device when timer expires.
Reported by: ken@kdm.org
Reviewed by: mav, ken
Proper error recovery depends on freezing the device queue when an
error occurs, so we can recover from an error before sending
additional commands.
The ocs_fc(4) driver was not freezing the device queue for most
SCSI errors, and that broke error recovery.
sys/dev/ocs_fc/ocs_cam.c:
In ocs_scsi_initiator_io_cb(), freeze the device queue if
we're passing back status other than CAM_REQ_CMP.
Submitted by: ken@kdm.org
Reviewed by: mav, ken
In ocs_scsi_initiator_io_cb(), if the SCSI command that is
getting completed had a residual equal to the transfer length,
it was setting the CCB status to CAM_REQ_CMP.
That breaks the expected behavior for commands like READ ATTRIBUTE.
For READ ATTRIBUTE, if the first attribute requested doesn't exist,
the command is supposed to return an error (Illegal Request,
Invalid Field in CDB). The broken behavior for READ ATTRIBUTE
caused LTFS tape formatting to fail. It looks for attribute
0x1623, and expects to see an error if the attribute isn't present.
In addition, if the residual is negative (indicating an overrun),
only set the CCB status to CAM_DATA_RUN_ERR if we have not already
reported an error. The SCSI sense data will have more detail about
what went wrong.
sys/dev/ocs_fc/ocs_cam.c:
In ocs_scsi_initiator_io_cb(), don't set the status to
CAM_REQ_CMP if the residual is equal to the transfer length.
Also, only set CAM_DATA_RUN_ERR if we didn't get SCSI
status.
Submitted by: ken@kdm.org
Reviewed by: mav, ken
Use refcounting to delay the detach rather than device_busy and/or
device_unbusy. fd/fdc is one of the few consumers of device_busy in the
tree for that, and it's not a good fit. Also, nothing is waking 'fd' and
other drivers don't loop like this. Return EBUSY if we still have active
users.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D31830
Add support for ACPI device probing for SDHCI controller found on Marvell chips.
Reviewed by: mw
Sponsored by: Semihalf
Differential revision: https://reviews.freebsd.org/D31600
This patch splits driver code into two seperate files sdhci_xenon.c
and sdhci_xenon_fdt.c. This will allow future implementation of ACPI
discovery of sdhci on Xenon chips.
Reviewed by: mw
Sponsored by: Semihalf
Differential revision: https://reviews.freebsd.org/D31599
The debug register dump routine is not hooked up and is really only
useful to driver developers, so put it under an mgb-specific MGB_DEBUG
rather than general DEBUG.
MFC after: 1 week
Fixes: 8890ab7758 ("Introduce if_mgb driver...")
Sponsored by: The FreeBSD Foundation
The refactoring in 8a8166e5bc introduced a functional change that
breaks booting on the Stratix 10, hanging when it should be attaching
da0. Previously OF_getencprop was called with a pointer to host->f_max,
so if it wasn't present then the existing value was left untouched, but
after that commit it will instead clobber the value with 0. The dwmmc
driver, as used on the Stratix 10, sets a default value before calling
mmc_fdt_parse and so was broken by this functional change. It appears
that aw_mmc also does the same thing, so was presumably also broken on
some boards.
Fixes: 8a8166e5bc ("mmc: switch mmc_helper to device_ api")
Reviewed by: manu, mw
Differential Revision: https://reviews.freebsd.org/D32209
mvneta_find_ethernet_prop_switch() is file-local static to
if_mvneta_fdt.c. Normally we would not need a function declararion
but in case MVNETA_DEBUG is set it becomes public. Move the
function declaration from if_mvneta.c to if_mvneta_fdt.c to avoid
a warning during each compile.
Make sure the completion ID is in the range of [0..num_trackers) since
the values past the end of the act_tr array are never going to be valid
trackers and will lead to pain and suffering if we try to dereference
them to get the tracker or to set the tracker back to NULL as we
complete the I/O.
Sponsored by: Netflix
Reviewed by: mav, chs, chuck
Differential Revision: https://reviews.freebsd.org/D32088
Count the number of times we're asked to process completions, but that
we ignore because the state of the qpair isn't in RECOVERY_NONE.
Sponsored by: Netflix
Reviewed by: mav, chuck
Differential Revision: https://reviews.freebsd.org/D32212
The proper phase for the qpiar right after reset in the first interrupt
is 1. For it, make sure that we're not still in phase 0. This is an
illegal state to be processing interrupts and indicates that we've
failed to properly protect against a race between initializing our state
and processing interrupts. Modify stat resetting code so it resets the
number of interrpts to 1 instead of 0 so we don't trigger a false
positive panic.
Sponsored by: Netflix
Reviewed by: cperciva, mav (prior version)
Differential Revision: https://reviews.freebsd.org/D32211
An interrupt happens on the admin queue right away after the reset, so
as soon as we enable interrupts, we'll get a call to our interrupt
handler. It is safe to ignore this interrupt if we're not yet
initialized, or to process it if we are. If we are initialized, we'll
see there's no completion records and return. If we're not, we'll
process no completion records and return. Either way, nothing is
processed and nothing is lost.
Until we've completely setup the qpair, we need to avoid processing
completion records. Start the qpair in the waiting recovery state so we
return immediately when we try to process completions. The code already
sets it to 'NONE' when we're initialization is complete. It's safe to
defer completion processing here because we don't send any commands
before the initialization of the software state of the qpair is
complete. And even if we were to somehow send a command prior to that
completing, the completion record for that command would be processed
when we send commands to the admin qpair after we've setup the software
state. There's no good central point to add an assert for this last
condition.
This fixes an KASSERT "received completion for unknown cmd" panic on
boot.
Fixes: 502dc84a8b
Sponsored by: Netflix
Reviewed by: mav, cperciva, gallatin
Differential Revision: https://reviews.freebsd.org/D32210
