Future firmwares will report these types to the driver. These
transceivers work already but are misidentified as a different type.
Sponsored by: Chelsio Communications
(cherry picked from commit c22b297062e1440676973a8aa89cbad1571e22f9)
The driver uses bus_reset_child on its parent to reset itself but that
performs an FLR whereas the hardware needs a Conventional Reset[1] for
full re-initialization. Add routines that perform conventional hot
reset and use them instead. The available reset mechanisms are:
* PCIe secondary bus reset (default)
* PCIe link bounce
hw.cxgbe.reset_method can be used to override the default. The internal
PL_RST is also available but is for testing only.
[1] 6.6.1 in PCI Express® Base Specification 5.0 version 1.0
Sponsored by: Chelsio Communications
(cherry picked from commit 011e3d0b8b90a4330f14b2cb7da45ed7b805ed10)
Add a new hw_all_ok() routine and use it to avoid hardware access in the
public control interfaces (ifnet ioctls, ifmedia calls, etc.). Continue
to use hw_off_limits() in the private ioctls/sysctls and other debug
code. Retire adapter_stopped() as it's of no use by itself.
This fixes problems where ifnet slow-path operations would enter a
synch_op just before set_adapter_hwstatus(false) and touch the hardware
when it's not safe to do so.
Sponsored by: Chelsio Communications
(cherry picked from commit e19d84979a183deb37ce6d7e385c3ccf02a3c8c7)
It was always set to PCATCH because the driver tested (INTR_OK) instead
of (flags & INTR_OK). Fit a WITNESS_WARN in a single line while here.
Sponsored by: Chelsio Communications
(cherry picked from commit 04bf43505bae1bb20d315a44e977d97aed3e5733)
An L2 table entry isn't associated with a particular SMT (Source MAC
Table) entry.
Sponsored by: Chelsio Communications
(cherry picked from commit f79fba05a016d53e054d6f587213889c3e31b4db)
The driver does minimal initialization in this mode and suspend/resume
should ignore resources that aren't setup. This is for debug only.
kenv hw.cxgbe.sos="1"
kldload if_cxgbe
devctl suspend t6nex0
devctl resume t6nex0
Sponsored by: Chelsio Communications
(cherry picked from commit f4ab14044c1de35b1aefad5449bddc5a1272f8d9)
This function is not used outside of this file.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47760
(cherry picked from commit c2cd12b7ae28a34e551d1b96906620249a860bc0)
- Only two of these tunables are used for RATELIMIT without
TCP_OFFLOAD.
- Mark t4_tmr_idx_ofld and t4_pktc_idx_ofld static.
- Move hw.cxgbe.cop_managed_offloading under hw.cxgbe.toe since it is
specific to TOE.
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47765
(cherry picked from commit 90652188232458891a2922fb5a542971aa58055d)
Normally the reply to destroy_server() releases the listen context but
it is not called when the adapter is suspended. Release the context
right away in that case.
Sponsored by: Chelsio Communications
(cherry picked from commit 45d5b9f0324a13df06712b7a9df5f2fbe8475764)
The kernel source contains several definitions of an ilog2 function;
some are slower than necessary, and one of them is incorrect.
Elimininate them all and define an ilog2 macro in libkern to replace
them, in a way that is fast, correct for all argument types, and, in a
GENERIC kernel, includes a check for an invalid zero parameter.
Folks at Microsoft have verified that having a correct ilog2
definition for their MANA driver doesn't break it.
Reviewed by: alc, markj, mhorne (older version), jhibbits (older version)
Differential Revision: https://reviews.freebsd.org/D45170
Differential Revision: https://reviews.freebsd.org/D45235
(cherry picked from commit b0056b31e90029553894d17c441cbb2c06d31412)
The commit to introduce TCP_USE_DDP support had a couple of bugs that
broke support for zerocopy receive via aio_read(). First, the length
and offset arguments to mk_update_tcb_for_ddp() were reversed which
prevented DDP from working. Second, the AIO state in the toep was
initialized too late when the first aio_read() request was queued.
Reported by: Harshavardhan Tanneru @ Chelsio
Fixes: eba13bbc37ab cxgbe: Support TCP_USE_DDP on offloaded TOE connections
MFC after: 1 week
Sponsored by: Chelsio Communications
(cherry picked from commit 70693a45381b687e40ea30710aa38cb9f24b6b02)
A hardware IPv6 server needs 2 consecutive stids (server tids) starting
from a 2-aligned stid whereas an IPv4 server needs only 1 stid without
any constraint. The allocator used to grab the first free stid(s) for
both but this can fragment the stid space leaving nothing suitable for
IPv6 even when lots of stids are available. Change the allocator to
prefer stids for IPv4 from the ones that cannot be used for IPv6.
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47042
(cherry picked from commit f48fb131c2523d4915d1ad4eba002251c64f5574)
Reported by: Sony Arpita Das @ Chelsio
Fixes: 5c15094916bf cxgbe(4): Update the board names of the T6 OCP cards.
Sponsored by: Chelsio Communications
(cherry picked from commit 878413d5590c9f24fb8a92ebca00899662594874)
Most of these sysctls don't call sbuf_* while holding any locks. Of
the ones that do hold locks, all but one can be fixed to drop the lock
before calling sbuf_*.
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45186
(cherry picked from commit 61089df8147eb7109696476c891514296d543bad)
adapter->flags are guarded by a synch_op, as noted in the comment in
adapter.h where the flags are defined.
Fixes: 5241b210a4e cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
Sponsored by: Chelsio Communications
(cherry picked from commit 52e5a66eac22d24e29f204c5b65232378e71f4cf)
* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state after t4_tom is unloaded.
* Do not allow unload if tids are in use by the TOE on any adapter.
Reported by: Bimal Abraham @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit 9ba8670a8b175de79ea087688f51595b4f2db862)
Tracers have to be recreated after a restart but that's okay given that
they are used for debugging only.
Sponsored by: Chelsio Communications
(cherry picked from commit ee3da604dd016439850dae77366796313e60f0e0)
An L2 entry in the driver's hash was marked STALE unconditionally if it
changed in the kernel when its driver refcount is 0. Fix the driver to
do this for VALID entries only.
Sponsored by: Chelsio Communications
(cherry picked from commit 3883300afe0bff5c5658274c4d8cfe773d08343e)
1. Mark the L2T entry valid only if t4_write_l2e succeeds, which won't
happen if the adapter is stopped. This prevents L2T entries from
sometimes getting (re)promoted to VALID on Tx activity during stop.
2. Discard a work request immediately instead of enqueueing it to the
arp queue if the adapter is stopped.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit 07f47e8850d0639d474026b203013072aeb32c81)
The destination queue for tracing filters is destroyed during stop or
suspend and the software state needs to reflect this. A new destination
queue will be setup when the adapter resumes operation.
Sponsored by: Chelsio Communications
(cherry picked from commit d82cb5c608ae9aaced4cb459ca511e62c64dbf74)
The STALE state means the L2T entry is valid in hardware but needs to be
refreshed (ARP/NDP) in software. But stop/suspend wipes the hardware
L2T and STALE entries need to be updated just like VALID entries to match
actual hardware state.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit 171e57967b3e53f0fb48116df5003ce17163295c)
This fixes a panic where the peer's ack to the synack arrives on a
different queue and do_pass_establish tries to remove the synqe from
synqe_list before it has been added by do_pass_accept_req.
Reported by: Sony Arpita Das @ Chelsio
Fixes: 283333c0e329 cxgbe/t4_tom: Track all synq entries in a per-adapter list.
Sponsored by: Chelsio Communications
(cherry picked from commit 674cbf38f6d0a0b307e52c4265da9f077606b035)
This fixes a regression in 5241b210a4e1 where the driver stopped
reporting link down after a fatal error unless t4_reset_on_fatal_err was
also set.
Fixes: 5241b210a4e1 cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
Sponsored by: Chelsio Communications
(cherry picked from commit 7aeec281b8b2620d7d88c9abdfadd7e20a485b9c)
Stop allocating new resources when the RNIC is stopped but continue to
allow previously allocated resources to be freed. Note that t4_tom's
uld_stop tears down all TOE connections, including those being used for
iWARP, and that triggers the cleanup of iWARP resources.
Fail post_send/post_recv early too to avoid the SQ doorbell.
Sponsored by: Chelsio Communications
(cherry picked from commit 9fdb683d92b36cbd20bbd8d61f0c1138f8348dd4)
backlog is an int and not a string. While here, fix an adjacent CTR
that was spread over two lines even though it fits in one.
Sponsored by: Chelsio Communications
(cherry picked from commit 3f250bb6f05b57890215398767bbb8aa00c888f3)
Now that suspend/resume is supported by the base driver, a fatal error
isn't the only reason that the RNIC can stop abruptly. Also, this state
is no longer permanent as it's possible to resume operations after a
stop. Rename the flag and associated routines to match the new state of
affairs.
Sponsored by: Chelsio Communications
(cherry picked from commit 8254a276ad893ae2a1b35fcbbad255f06e29b8c6)
Add a reset_adapter wrapper that picks the most suitable reset routine
internally. Use it in the fatal error handler as well as the sysctl
based reset.
Sponsored by: Chelsio Communications
(cherry picked from commit d668a0b0abe4b079ac2a0d9cd0d7b71bd4a043c6)
Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.
Sponsored by: Chelsio Communications
(cherry picked from commit 0a9d1da6e6cede5e9c0ff63240d724049ad72b5b)
1. Remove toepcb from the toep_list on active open failure.
2. Purge the wr_list for an L2T entry on an adpater stop.
Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
Sponsored by: Chelsio Communications
(cherry picked from commit fef0e39f64a1db796ded8777dbee71fc287f6107)
This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.
The LLD has already stopped the adapter hardware and all its queues by the time
these ULD routines get called. The general approach in t4_tom is to purge the
lookup tables immediately so that they are ready for operation by the time the
adapter resumes, and park all the resources left hanging by the stopped hardware
into separate "stranded" queues that can be dealt with at leisure.
Outstanding active opens, live connections, and synq entries (for connections in
the middle of the 3-way handshake) are all treated as if the hardware had
reported an abrupt error for the tid. The servers/listeners are a bit different
in that no error is reported. They're just noted as non-functional when the
hardware stops and are recreated by the driver during restart.
Sponsored by: Chelsio Communications
(cherry picked from commit c1c524852f625cf5f420653f7850d1fe3ff6b4ca)
Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change adds
a synqe_list to track the synq entries. These two lists can be used to
enumerate and iterate over all live tids.
Sponsored by: Chelsio Communications
(cherry picked from commit 283333c0e329fd7aceff16fa3bf2b9892744d883)
L2T entries are used by both filters and TOE and the L2T is shared
between the base driver (LLD) and the TOM ULD. Add a flag to indicate
that the L2T is stopped, which means:
* t4_alloc_l2e and t4_l2t_alloc_switching will not allocate new entries.
* t4_tom will ignore all ARP/NDP updates from the kernel.
* Previously allocated L2T entries can still be freed.
Sponsored by: Chelsio Communications
(cherry picked from commit cd93fdee5c8bbdb00d10f8a1fa43f30f151a1ef7)
atids are used by both filters and TOE and the atid table is in the base
driver (LLD). New atids cannot be allocated when the allocator is
stopped but existing ones can still be freed. It is expected that the
owners of outstanding atids will release them in their own stop
processing, before the adapter is restarted.
Sponsored by: Chelsio Communications
(cherry picked from commit fd3aca5b41968421f243b760ac5733f39f25cc56)
The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. This commit
adds basic infrastructure for stateful upper layer drivers (ULDs) to
participate in suspend/resume/reset.
* Add a uld_restart to indicate that the adapter has been restarted
after a stop and the ULD should resume operations.
* Move the existing functionality in t4_suspend/t4_resume to stop_lld and
restart_lld. Use these and the new uld restart routines everywhere the
adapter has to be stopped abruptly and restarted, namely:
1. PCIE bus suspend/resume/reset methods invoked by the kernel.
2. Manual internal-reset using driver sysctl.
3. Automatic internal-reset on a fatal error.
* Implement an alternate internal-reset for use in VMs and for testing.
Typical reset sequence is:
stop_adapter(sc);
stop_lld(sc);
stop_all_uld(sc);
set_adapter_hwstatus(sc, false);
/* hw reset takes place here. */
restart_adapter(sc);
restart_lld(sc);
set_adapter_hwstatus(sc, true);
restart_all_uld(sc);
Sponsored by: Chelsio Communications
(cherry picked from commit 5241b210a4e1029f3005faf82cd74965645c401b)
* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.
* There is no need to refcount ULDs so stop doing that.
* Add uld_ prefix to all members of uld_info.
* Rename async_event to uld_stop to match its actual purpose. Call it
for all ULDs and not just ULD_IWARP.
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46029
(cherry picked from commit cf5e6370f15cffabbbf508083ba7d48ec8abfa79)
Do not assume that the table starts at index 0 and is typically 4K in
size. The only thing the driver needs to verify is that its use of
F_SYNC_WR doesn't collide with the L2T hwidx range.
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46028
(cherry picked from commit cfcfd3c7bf5b60da42b13ac5d8085c762613c302)
Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no need to wait
for the kernel to detach the toepcb.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45991
(cherry picked from commit 27479403a764cf3b97194887a1f819c1e35357aa)
The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhaul in the kernel
some time ago. Catch up by having the driver perform the detach.
The hardware does not handle TIME_WAIT so it's important to detach and
let the kernel arm the 2MSL timer to deal with it.
Reported by: Sony Arpita Das @ Chelsio
Reviewed by: jhb
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45990
(cherry picked from commit bbc326241d91ab2cee2ec2c5c0aa8a906480132f)
Align the comment for if_t with the rest and remove the declaration of
t4_l2t_set_switching, which was removed in 061bbaf7e7.
No functional change.
Sponsored by: Chelsio Communications
(cherry picked from commit b3bbc6cc79a3b76d342b2a8fd68e5266eb50ff11)
Previously, when transmitting short runs of packets via cxgbe_nm_tx(),
we would wait until a large number of packets were buffered before
scheduling a task to clean transmit buffers.
Obtained from: np
(cherry picked from commit 6af3d59962449c66688361598163dd174cc5fb2a)
