Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a, the
driver had to find the softc pointer from the adapter before it could
locate the ccr_session structure for a completed request. Since
Conrad's changes, the ccr_session pointer can now be obtained directly
from the crp. Add a backpoint from ccr_session back to ccr_softc and
use this in place of the ccr_softc member in cxgbe's struct adapter.
Sponsored by: Chelsio Communications
For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.
The most import one is the inpcb itself. With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa. Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that. However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort. The new intotcpcb() macro is ready for such move.
The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.
One interesting side effect is that now TCP data is allocated from
SMR-protected zone. Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.
Large part of the change was done with sed script:
s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g
Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.
Differential revision: https://reviews.freebsd.org/D37127
Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.
The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.
When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB. The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary. If so, the connection is migrated to
TLS mode and unfrozen. Otherwise, the connection is unfrozen
temporarily until more data arrives. Once more data arrives, the
receive queue is frozen again and rechecked. This continues until the
connection is paused at a record boundary. Any records received
before TLS mode is enabled are decrypted as software records.
Note that this removes the 'rx_tls_ports' sysctl. TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit. This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37351
The previous commit lost an implicit struct socket * cast. Use an
inline function instead as the macro is already rather long.
Fixes: e1401f7579 cxgbe: use standard sototcpcb() accessor macro to get socket's tcpcb
Sponsored by: Chelsio Communications
Mechanically cleanup INP_TIMEWAIT from the kernel sources. After
0d7445193a, this commit shall not cause any functional changes.
Note: this flag was very often checked together with INP_DROPPED.
If we modify in_pcblookup*() not to return INP_DROPPED pcbs, we
will be able to remove most of this checks and turn them to
assertions. Some of them can be turned into assertions right now,
but that should be carefully done on a case by case basis.
Differential revision: https://reviews.freebsd.org/D36400
This was added in 7cba15b16e in 2016 and firmwares at that time were
already setting up the iSCSI tag mask properly. Since then it has also
become possible to split the iSCSI region between multiple PCIE PFs but
the driver's calculation takes only its own PF's allocation into account
and that means this code is incorrect and not just a harmless no-op.
MFC after: 1 week
Sponsored by: Chelsio Communications
This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use the previous structure if the write to update cal_current isn't
yet visible.
Reviewed by: gallatin
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36669
This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration once a second rather than once every 30 seconds. Note
that the cxgbe driver already queries multiple registers once a second
for the statistics timers. This version also uses fewer instructions
with no branches (for the math portion) in the per-packet fast path.
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36663
Coverity flagged this in its latest run but it is not a problem in
practice as the card's core clock would have to be > 4.2GHz for any
overflow to occur.
CID 1498303: Integer handling issues (OVERFLOW_BEFORE_WIDEN)
Potentially overflowing expression "sc->params.vpd.cclk * 1000U" with type "unsigned int" (32 bits, unsigned) is evaluated using 32-bit arithmetic, and then used in a context that expects an expression of type "uint64_t" (64 bits, unsigned).
Reported by: Coverity Scan (CID 1498303)
Sponsored by: Chelsio Communications
Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.
Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315
hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.
MFC after: 2 weeks
Sponsored by: Chelsio Communications
This could only happen on systems with PAGE_SIZE < 4K and FreeBSD
doesn't support such systems.
Reviewed by: np, imp, jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D36321
o Assert that every protosw has pr_attach. Now this structure is
only for socket protocols declarations and nothing else.
o Merge struct pr_usrreqs into struct protosw. This was suggested
in 1996 by wollman@ (see 7b187005d1), and later reiterated
in 2006 by rwatson@ (see 6fbb9cf860).
o Make struct domain hold a variable sized array of protosw pointers.
For most protocols these pointers are initialized statically.
Those domains that may have loadable protocols have spacers. IPv4
and IPv6 have 8 spacers each (andre@ dff3237ee5).
o For inetsw and inet6sw leave a comment noting that many protosw
entries very likely are dead code.
o Refactor pf_proto_[un]register() into protosw_[un]register().
o Isolate pr_*_notsupp() methods into uipc_domain.c
Reviewed by: melifaro
Differential revision: https://reviews.freebsd.org/D36232
Changes since 1.26.6.0 are listed here. This list comes from the
Release Notes for "Chelsio Unified Wire 3.17.0.0 for Linux" dated
2022-07-29.
Fixes
-----
BASE:
- Enabled all MA parity interrupt bits.
- Use config file value to override number of rx channel. nrxch=1 was not
handled in the firmware.
- Replaced read only registers with new registers EDC_H_BIST_USER_WDATA0,
EDC_H_BIST_USER_WDATA1 and EDC_H_BIST_CMD_LEN to dump the uP memory parity
error status registers.
- 10G simplex module support enabled.
Obtained from: Chelsio Communications
MFC after: 1 month
Sponsored by: Chelsio Communications
Remove a few more remnants from the old pre-KTLS support and instead
assume that each work request sends a single TLS record.
Sponsored by: Chelsio Communications
If an adapter advertises support for TLS keys but an empty TLS key
storage area in on-board memory, fail the request rather than invoking
vmem_alloc on an uninitialized vmem.
Sponsored by: Chelsio Communications
With clang 15, the following -Werror warnings are produced:
sys/dev/cxgbe/t4_tracer.c:234:18: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
t4_tracer_modload()
^
void
sys/dev/cxgbe/t4_tracer.c:243:20: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
t4_tracer_modunload()
^
void
This is because t4_tracer_modload() and t4_tracer_modunload() are
declared with a (void) argument list, but defined with an empty argument
list. Make the definitions match the declarations.
MFC after: 3 days
Clang 15 warns:
sys/dev/cxgbe/cudbg/cudbg_lib.c:2949:6: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
int i = 0;
^
Apparently 'i' was meant as the current retry counter, but '1' was used
in the while loop comparison instead, making the loop potentially
infinite, if 'busy' never gets reset.
MFC after: 3 days
Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D35834
'command' is too generic for something specific to the kernel debugger;
change this so it is less likely to collide with local variable names.
Also rename struct command_table to struct db_command_table.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35367
Round down the maximim ISO payload by the current MSS. Otherwise the
round up by MSS when calculating the 16-bit maximum payload len passed
along in the FLOWC work request can overflow.
Discussed with: np
Sponsored by: Chelsio Communications
c4iw_zero_addr is supposed to check for all-zero addresses but was using
IN_ZERONET (which does something different) for IPv4 addresses. Fix it
by simply checking for 0 as is done for IPv6 addresses.
Reported by: karels@
MFC after: 3 days
Sponsored by: Chelsio Communications
Since c67f3b8b78 the sockbuf mutexes belong to the containing socket,
and socket buffers just point to it. In 74a68313b5 macros that access
this mutex directly were added. Go over the core socket code and
eliminate code that reaches the mutex by dereferencing the sockbuf
compatibility pointer.
This change requires a KPI change, as some functions were given the
sockbuf pointer only without any hint if it is a receive or send buffer.
This change doesn't cover the whole kernel, many protocols still use
compatibility pointers internally. However, it allows operation of a
protocol that doesn't use them.
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D35152
The final transmit and receive queue indices need to be positive
values. However, since txq_idx and rxq_idx are signed (to permit
using -1 to as a marker for uninitialized values), using %= with
another integer type (vi->nofld[tr]xq) yielded a sign-extended modulus
value. This resulted in negative queue indices and a buffer underrun
when arc4random() returned a value with the sign bit set. Use a
temporary unsigned variable to hold the "raw" queue index to force
unsigned modulus.
This worked previously because the modulus was previously applied
directly to the return value of arc4random() which is unsigned before
the result was assigned to txq_idx and rxq_idx.
Discussed with: np
Fixes: db28d4a0cd cxgbe/t4_tom: Support for round-robin selection of offload queues.
Sponsored by: Chelsio Communications
cxgbei needs the ability to return different limits based on the
connection (e.g. if the connection is over a T5 adapter or a T6
adapter as well as factoring in the MTU).
This change plumbs through the changes in the ioctls without changing
any of the backends. The limits callback passed to icl_register now
accepts a second socket argument which holds the integer file
descriptor. To support ABI compatiblity for old binaries, the
callback should return "global" values if the socket fd is zero.
The CTL_ISCSI_LIMITS argument used with CTL_ISCSI by ctld(8) now
accepts the socket fd in a field that was previously part of a
reserved spare field. Old binaries zero this request which results in
passing a socket fd of 0 to the limits callback.
The ISCSIDREQUEST ioctl no longer returns limits. Instead, iscsid(8)
invokes a new ISCSIDLIMITS ioctl after establishing the connection via
connect(2). For ABI compat, if the old ISCSIDREQUEST is invoked, the
global limits are still fetched (with a socket fd of 0) and returned.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34928
A COP (Connection Offload Policy) rule can now specify that the tx
and/or rx queue for a new tid should be selected in a round-robin
manner. There is no change in default behavior.
Reviewed by: jhb@
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34921
The driver queries the firmware to find out if it supports this feature
and enables it if it does. The firmware moves the iSCSI page pod region
to a lower address so that some of it is located in the faster on-chip
memory instead of external DDR.
Reviewed by: jhb@
MFC after: 3 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34895
CID 1487932: Control flow issues (NESTING_INDENT_MISMATCH).
The macro on this line expands into multiple statements, only the first
of which is nested within the preceding parent while the rest are not.
9828 ulp_region(RX_TLS_KEY);
Reported by: Coverity (CID 1487932)
Fixes: f88b31885c cxgbe(4): meminfo should get the TLS region's limits from the hardware.
MFC after: 3 days
Sponsored by: Chelsio Communications
