Most of these sysctls don't call sbuf_* while holding any locks. Of
the ones that do hold locks, all but one can be fixed to drop the lock
before calling sbuf_*.
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45186
(cherry picked from commit 61089df8147eb7109696476c891514296d543bad)
adapter->flags are guarded by a synch_op, as noted in the comment in
adapter.h where the flags are defined.
Fixes: 5241b210a4e cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
Sponsored by: Chelsio Communications
(cherry picked from commit 52e5a66eac22d24e29f204c5b65232378e71f4cf)
* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state after t4_tom is unloaded.
* Do not allow unload if tids are in use by the TOE on any adapter.
Reported by: Bimal Abraham @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit 9ba8670a8b175de79ea087688f51595b4f2db862)
Tracers have to be recreated after a restart but that's okay given that
they are used for debugging only.
Sponsored by: Chelsio Communications
(cherry picked from commit ee3da604dd016439850dae77366796313e60f0e0)
An L2 entry in the driver's hash was marked STALE unconditionally if it
changed in the kernel when its driver refcount is 0. Fix the driver to
do this for VALID entries only.
Sponsored by: Chelsio Communications
(cherry picked from commit 3883300afe0bff5c5658274c4d8cfe773d08343e)
1. Mark the L2T entry valid only if t4_write_l2e succeeds, which won't
happen if the adapter is stopped. This prevents L2T entries from
sometimes getting (re)promoted to VALID on Tx activity during stop.
2. Discard a work request immediately instead of enqueueing it to the
arp queue if the adapter is stopped.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit 07f47e8850d0639d474026b203013072aeb32c81)
The destination queue for tracing filters is destroyed during stop or
suspend and the software state needs to reflect this. A new destination
queue will be setup when the adapter resumes operation.
Sponsored by: Chelsio Communications
(cherry picked from commit d82cb5c608ae9aaced4cb459ca511e62c64dbf74)
The STALE state means the L2T entry is valid in hardware but needs to be
refreshed (ARP/NDP) in software. But stop/suspend wipes the hardware
L2T and STALE entries need to be updated just like VALID entries to match
actual hardware state.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit 171e57967b3e53f0fb48116df5003ce17163295c)
This fixes a panic where the peer's ack to the synack arrives on a
different queue and do_pass_establish tries to remove the synqe from
synqe_list before it has been added by do_pass_accept_req.
Reported by: Sony Arpita Das @ Chelsio
Fixes: 283333c0e329 cxgbe/t4_tom: Track all synq entries in a per-adapter list.
Sponsored by: Chelsio Communications
(cherry picked from commit 674cbf38f6d0a0b307e52c4265da9f077606b035)
This fixes a regression in 5241b210a4e1 where the driver stopped
reporting link down after a fatal error unless t4_reset_on_fatal_err was
also set.
Fixes: 5241b210a4e1 cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
Sponsored by: Chelsio Communications
(cherry picked from commit 7aeec281b8b2620d7d88c9abdfadd7e20a485b9c)
Stop allocating new resources when the RNIC is stopped but continue to
allow previously allocated resources to be freed. Note that t4_tom's
uld_stop tears down all TOE connections, including those being used for
iWARP, and that triggers the cleanup of iWARP resources.
Fail post_send/post_recv early too to avoid the SQ doorbell.
Sponsored by: Chelsio Communications
(cherry picked from commit 9fdb683d92b36cbd20bbd8d61f0c1138f8348dd4)
backlog is an int and not a string. While here, fix an adjacent CTR
that was spread over two lines even though it fits in one.
Sponsored by: Chelsio Communications
(cherry picked from commit 3f250bb6f05b57890215398767bbb8aa00c888f3)
Now that suspend/resume is supported by the base driver, a fatal error
isn't the only reason that the RNIC can stop abruptly. Also, this state
is no longer permanent as it's possible to resume operations after a
stop. Rename the flag and associated routines to match the new state of
affairs.
Sponsored by: Chelsio Communications
(cherry picked from commit 8254a276ad893ae2a1b35fcbbad255f06e29b8c6)
Add a reset_adapter wrapper that picks the most suitable reset routine
internally. Use it in the fatal error handler as well as the sysctl
based reset.
Sponsored by: Chelsio Communications
(cherry picked from commit d668a0b0abe4b079ac2a0d9cd0d7b71bd4a043c6)
Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.
Sponsored by: Chelsio Communications
(cherry picked from commit 0a9d1da6e6cede5e9c0ff63240d724049ad72b5b)
1. Remove toepcb from the toep_list on active open failure.
2. Purge the wr_list for an L2T entry on an adpater stop.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit fef0e39f64a1db796ded8777dbee71fc287f6107)
This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.
The LLD has already stopped the adapter hardware and all its queues by the time
these ULD routines get called. The general approach in t4_tom is to purge the
lookup tables immediately so that they are ready for operation by the time the
adapter resumes, and park all the resources left hanging by the stopped hardware
into separate "stranded" queues that can be dealt with at leisure.
Outstanding active opens, live connections, and synq entries (for connections in
the middle of the 3-way handshake) are all treated as if the hardware had
reported an abrupt error for the tid. The servers/listeners are a bit different
in that no error is reported. They're just noted as non-functional when the
hardware stops and are recreated by the driver during restart.
Sponsored by: Chelsio Communications
(cherry picked from commit c1c524852f625cf5f420653f7850d1fe3ff6b4ca)
Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change adds
a synqe_list to track the synq entries. These two lists can be used to
enumerate and iterate over all live tids.
Sponsored by: Chelsio Communications
(cherry picked from commit 283333c0e329fd7aceff16fa3bf2b9892744d883)
L2T entries are used by both filters and TOE and the L2T is shared
between the base driver (LLD) and the TOM ULD. Add a flag to indicate
that the L2T is stopped, which means:
* t4_alloc_l2e and t4_l2t_alloc_switching will not allocate new entries.
* t4_tom will ignore all ARP/NDP updates from the kernel.
* Previously allocated L2T entries can still be freed.
Sponsored by: Chelsio Communications
(cherry picked from commit cd93fdee5c8bbdb00d10f8a1fa43f30f151a1ef7)
atids are used by both filters and TOE and the atid table is in the base
driver (LLD). New atids cannot be allocated when the allocator is
stopped but existing ones can still be freed. It is expected that the
owners of outstanding atids will release them in their own stop
processing, before the adapter is restarted.
Sponsored by: Chelsio Communications
(cherry picked from commit fd3aca5b41968421f243b760ac5733f39f25cc56)
The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. This commit
adds basic infrastructure for stateful upper layer drivers (ULDs) to
participate in suspend/resume/reset.
* Add a uld_restart to indicate that the adapter has been restarted
after a stop and the ULD should resume operations.
* Move the existing functionality in t4_suspend/t4_resume to stop_lld and
restart_lld. Use these and the new uld restart routines everywhere the
adapter has to be stopped abruptly and restarted, namely:
1. PCIE bus suspend/resume/reset methods invoked by the kernel.
2. Manual internal-reset using driver sysctl.
3. Automatic internal-reset on a fatal error.
* Implement an alternate internal-reset for use in VMs and for testing.
Typical reset sequence is:
stop_adapter(sc);
stop_lld(sc);
stop_all_uld(sc);
set_adapter_hwstatus(sc, false);
/* hw reset takes place here. */
restart_adapter(sc);
restart_lld(sc);
set_adapter_hwstatus(sc, true);
restart_all_uld(sc);
Sponsored by: Chelsio Communications
(cherry picked from commit 5241b210a4e1029f3005faf82cd74965645c401b)
* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.
* There is no need to refcount ULDs so stop doing that.
* Add uld_ prefix to all members of uld_info.
* Rename async_event to uld_stop to match its actual purpose. Call it
for all ULDs and not just ULD_IWARP.
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46029
(cherry picked from commit cf5e6370f15cffabbbf508083ba7d48ec8abfa79)
Do not assume that the table starts at index 0 and is typically 4K in
size. The only thing the driver needs to verify is that its use of
F_SYNC_WR doesn't collide with the L2T hwidx range.
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46028
(cherry picked from commit cfcfd3c7bf5b60da42b13ac5d8085c762613c302)
Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no need to wait
for the kernel to detach the toepcb.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45991
(cherry picked from commit 27479403a764cf3b97194887a1f819c1e35357aa)
The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhaul in the kernel
some time ago. Catch up by having the driver perform the detach.
The hardware does not handle TIME_WAIT so it's important to detach and
let the kernel arm the 2MSL timer to deal with it.
Reported by: Sony Arpita Das @ Chelsio
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45990
(cherry picked from commit bbc326241d91ab2cee2ec2c5c0aa8a906480132f)
Align the comment for if_t with the rest and remove the declaration of
t4_l2t_set_switching, which was removed in 061bbaf7e7.
No functional change.
Sponsored by: Chelsio Communications
(cherry picked from commit b3bbc6cc79a3b76d342b2a8fd68e5266eb50ff11)
Previously, when transmitting short runs of packets via cxgbe_nm_tx(),
we would wait until a large number of packets were buffered before
scheduling a task to clean transmit buffers.
Obtained from: np
(cherry picked from commit 6af3d59962449c66688361598163dd174cc5fb2a)
This fixes a panic when multiple VIs are configured on an interface and
only the non-primary VI is up at the time of driver detach. The problem
was that the driver would queue a link state change notification for an
interface about to be freed.
To reproduce the panic, add "hw.cxgbe.num_vis=2" to loader.conf and
# kldload if_cxgbe
# ifconfig vcc0 up
# devctl detach t6nex0
trap 0x9, rip = 0xffffffff8107db70, rsp = 0xfffffe0055263d60, rbp = 0xfffffe0055263dd0
taskqueue_run_locked() at taskqueue_run_locked+0x2a0/frame 0xfffffe0055263dd0
taskqueue_run() at taskqueue_run+0x72/frame 0xfffffe0055263df0
taskqueue_swi_run() at taskqueue_swi_run+0x18/frame 0xfffffe0055263e10
intr_event_execute_handlers() at intr_event_execute_handlers+0x249/frame 0xfffffe0055263e50
ithread_execute_handlers() at ithread_execute_handlers+0x9e/frame 0xfffffe0055263e70
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45864
(cherry picked from commit dc20d49aa939caea365cbdf0341b00de69253be4)
This affects TOE operation when multiple rx c-channels are in use for
offload, which is an unusual configuration.
Sponsored by: Chelsio Communications
(cherry picked from commit c6c6d4aff90da83a292b4c2bbbe1f4d6e01cd82e)
It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sysctl MIB.
Sponsored by: Chelsio Communications
(cherry picked from commit 480ff89c67b25113515018cdcd13179229b4a0d3)
PORTVEC obtained from the firmware is the authoritative source of this
information, and nports (calculated from PORTVEC) is available by the
time t4_port_init runs.
Sponsored by: Chelsio Communications
(cherry picked from commit 4d1362cdc7375984a48f5f0048b1fe909524d21d)
All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.
Sponsored by: Chelsio Communications
(cherry picked from commit 857d74b6340e418396d79a46b264ce0eedd760e4)
Avoid a magic constant while here. No functional change intended.
Sponsored by: Chelsio Communications
(cherry picked from commit 43f6f08488046788b0ad66e9a5119f36e5de71ab)
The firmware clears the interrupts already and it has a better idea of
exactly what to clear for which generation of the ASIC. There is no
need for the driver to get involved.
Sponsored by: Chelsio Communications
(cherry picked from commit 1c7f9c8b4673abf3723be09afed4443261e0d186)
These register blocks are at different locations in different chips.
Sponsored by: Chelsio Communications
(cherry picked from commit b59c5d97edf17525405d95b1f5746c4a79a9c7c4)
The driver always uses the same modulation queue as the channel and the
table is unnecessary.
Sponsored by: Chelsio Communications
(cherry picked from commit f76effed14b25bfa0c47b10f6d8a076104c48d94)
