Commit graph

752 commits

Author SHA1 Message Date
Navdeep Parhar
09a7189fb7 cxgbe(4): Allow the driver to specify a burst size when configuring a
traffic class for rate limiting.

Add experimental knobs that allow the user to specify a default pktsize
and burstsize for traffic classes associated with a port:

dev.<ifname>.<instance>.tc.pktsize
dev.<ifname>.<instance>.tc.burstsize

Sponsored by:	Chelsio Communications
2018-08-07 22:13:03 +00:00
Navdeep Parhar
1979b51141 cxgbe(4): Allow user-configured and driver-configured traffic classes to
be used simultaneously.  Move sysctl_tc and sysctl_tc_params to
t4_sched.c while here.

MFC after:	3 weeks
Sponsored by:	Chelsio Communications
2018-08-06 23:21:13 +00:00
Navdeep Parhar
7b8f5a200a cxgbe(4): Break up sysctl_bitfield into 8 bit and 16 bit variants. Have
them display the current value of the bitfield rather than the fixed
value that was provided when the sysctl node was created.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-08-06 21:54:51 +00:00
Navdeep Parhar
564ec04ea8 Fix typo in cxgbe/t4_tom. 2018-08-06 19:09:55 +00:00
Navdeep Parhar
0c71c9ccb2 cxgbe(4): Improvements in TID management.
- Ignore any type of TID where the start/end values are not in the
  correct order.  There are situations where the firmware isn't able to
  reserve room for the number requested in the config file but doesn't
  report a failure during configuration and instead sets end <= start.

- Track start/end in tid_tab and remove some redundant copies from
  adapter->params.

- Move all the start/end and other read-only parameters to a quiet part
  of tid_tab, away from the tid locks.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-08-02 22:52:05 +00:00
Navdeep Parhar
ac8ec5fea6 cxgbe(4): Use the tx credit limit for ethofld rather than TOE when
initializing the softc for a per-flow rate limiter.  The limit happens
to be the same for both and the existing code worked by accident for
common configurations.

Reported by:	gallatin@
Sponsored by:	Chelsio Communications
2018-08-02 19:50:12 +00:00
Navdeep Parhar
aa8c29e5e7 cxgbe(4): Consider rateunit before ratemode when displaying information
about a traffic class.  This matches the order in which the firmware
evaluates unit and mode internally.

Sponsored by:	Chelsio Communications
2018-07-26 07:29:44 +00:00
Navdeep Parhar
f7c6e09244 cxgbe(4): Better defaults for all cl-rl rate limiters.
Start in "class" instead of "flow" mode.  This eliminates the need to
specify an MTU, which is not available that early anyway.  It also
allows the user to manually configure ch-rl rate limiting after attach.
This used to fail because ch-rl isn't supported if cl-rl "flow" mode is
configured.

Set all traffic classes to 1Gbps during initialization.  The goal is to
start off with _any_ valid configuration and 1Gbps works even for
gigabit cards.

Sponsored by:	Chelsio Communications
2018-07-26 06:42:09 +00:00
Navdeep Parhar
2095de1c3f cxgbe(4): Remove useless code that crept in with r336718.
MFC after:	3 days
X-MFC With:	336718
2018-07-25 17:45:43 +00:00
Navdeep Parhar
9c3b8b3c32 cxgbe(4): Validate only those parameters that are relevant to the
type of rate limiter being programmed.  Skip the ones that are not
applicable.

MFC after:	3 days
Sponsored by:	Chelsio Communications
2018-07-25 17:20:06 +00:00
Conrad Meyer
1b0909d51a OpenCrypto: Convert sessions to opaque handles instead of integers
Track session objects in the framework, and pass handles between the
framework (OCF), consumers, and drivers.  Avoid redundancy and complexity in
individual drivers by allocating session memory in the framework and
providing it to drivers in ::newsession().

Session handles are no longer integers with information encoded in various
high bits.  Use of the CRYPTO_SESID2FOO() macros should be replaced with the
appropriate crypto_ses2foo() function on the opaque session handle.

Convert OCF drivers (in particular, cryptosoft, as well as myriad others) to
the opaque handle interface.  Discard existing session tracking as much as
possible (quick pass).  There may be additional code ripe for deletion.

Convert OCF consumers (ipsec, geom_eli, krb5, cryptodev) to handle-style
interface.  The conversion is largely mechnical.

The change is documented in crypto.9.

Inspired by
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .

No objection from:	ae (ipsec portion)
Reported by:	jhb
2018-07-18 00:56:25 +00:00
Navdeep Parhar
069262a734 Fix vertical whitespace nit in cxgbe. 2018-07-10 06:09:25 +00:00
Navdeep Parhar
82df14c3ab cxgbe(4): Add a sysctl to report the chip's microprocessor's load
averages.  This works with debug or custom firmwares only.

sysctl dev.<nexus>.<instance>.loadavg
sysctl dev.t6nex.0.loadavg

MFC after:	1 month
Sponsored by:	Chelsio Communications
2018-07-10 03:03:10 +00:00
Navdeep Parhar
4d96a1b772 cxgbe(4): Assume that any unknown flash on the card is 4MB and has 64KB
sectors, instead of refusing to attach to the card.

Submitted by:	Casey Leedom @ Chelsio
MFC after:	3 days
Sponsored by:	Chelsio Communications
2018-07-06 19:33:58 +00:00
Matt Macy
6573d7580b epoch(9): allow preemptible epochs to compose
- Add tracker argument to preemptible epochs
- Inline epoch read path in kernel and tied modules
- Change in_epoch to take an epoch as argument
- Simplify tfb_tcp_do_segment to not take a ti_locked argument,
  there's no longer any benefit to dropping the pcbinfo lock
  and trying to do so just adds an error prone branchfest to
  these functions
- Remove cases of same function recursion on the epoch as
  recursing is no longer free.
- Remove the the TAILQ_ENTRY and epoch_section from struct
  thread as the tracker field is now stack or heap allocated
  as appropriate.

Tested by: pho and Limelight Networks
Reviewed by: kbowling at llnw dot com
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
Navdeep Parhar
d207040226 cxgbe/cxgbei: Fix harmful typo in the iSCSI offload driver.
Reported by:	gcc8 (via mmacy@)
MFC after:	3 days
Sponsored by:	Chelsio Communications
2018-06-27 14:29:13 +00:00
Navdeep Parhar
af8854fdc1 cxgbe(4): Do not leak the filters in the hashfilter table on module
unload.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-06-27 01:51:17 +00:00
Navdeep Parhar
b605d9cd51 cxgbe(4): Some mailbox commands require access to the Tx pipeline and
can time out if it's backed up due to a non-stop deluge of PAUSE frames
from a misbehaving peer.  Detect this situation and toggle MPS TxEn
to allow forward progress.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
2018-06-19 00:50:27 +00:00
Navdeep Parhar
0afe96c7bf cxgbe(4): Add a hw.cxgbe.starve_fl sysctl that can be used to starve the
freelists of netmap receive queues.  This is primarily to test various
congestion scenarios in the chip.

Sponsored by:	Chelsio Communications
2018-06-15 23:42:22 +00:00
Navdeep Parhar
31f494cdf5 cxgbe(4): Track the number of received frames separately from the number
of descriptors processed.  Add the ability to gather a certain maximum
number of frames in the driver's rx before waking up netmap rx.  If
there aren't enough frames then netmap rx will be woken up as usual.

hw.cxgbe.nm_rx_nframes

Sponsored by:	Chelsio Communications
2018-06-15 21:23:03 +00:00
Navdeep Parhar
6ddad9de86 cxgbe(4): sysctls to display the local and intr CPUs for the adapter.
The driver assumes the list can change (even though it does't right now)
and queries it every time the sysctl runs.

sysctl dev.<nexus>.<inst>.local_cpus
sysctl dev.<nexus>.<inst>.intr_cpus

sysctl dev.t6nex.0.local_cpus
sysctl dev.t6nex.0.intr_cpus

Sponsored by:	Chelsio Communications
2018-06-15 18:04:44 +00:00
Navdeep Parhar
2ea5b0f54a cxgbe(4): Catch up with recent changes in the kernel -- it no longer
holds non-sleepable locks around any of the driver ioctls.

Sponsored by:	Chelsio Communications
2018-06-14 01:27:35 +00:00
Navdeep Parhar
1bb577b4c2 cxgbe(4): Remove homemade version of htobe32 from the driver.
It was needed only for ia64 where it was implemented as a call to
bswapXX, which was always a real function.  htobeXX with a constant
argument is calculated at compile-time everywhere else.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-06-12 06:46:03 +00:00
Navdeep Parhar
c27fcc70cc cxgbe(4): Include full duplex mediaopt in media that can be reported as
active.  Always report full duplex in active media.

Sponsored by:	Chelsio Communications
2018-06-01 16:46:29 +00:00
Navdeep Parhar
b9330ed7a2 cxgbe(4): Retire an old check. 2018-06-01 01:05:34 +00:00
Navdeep Parhar
2c87bdc706 cxgbe(4): Add support for SMAC-rewriting filters.
Submitted by:	Krishnamraju Eraparaju @ Chelsio
Sponsored by:	Chelsio Communications
2018-05-31 21:56:57 +00:00
Navdeep Parhar
2dae2a7487 cxgbe(4): Add code to deal with the chip's source MAC table (aka SMT).
Submitted by:	Krishnamraju Eraparaju @ Chelsio
Sponsored by:	Chelsio Communications
2018-05-31 21:31:08 +00:00
Navdeep Parhar
1e3e6b634e cxgbe(4): Use ifm for ifmedia just like the rest of the kernel.
No functional change.
2018-05-31 02:22:40 +00:00
Navdeep Parhar
7cff4fd2d7 cxgbe(4): Implement ifm_change callback.
Sponsored by:	Chelsio Communications
2018-05-31 02:10:50 +00:00
Navdeep Parhar
56226f5673 cxgbe(4): Consider all supported speeds when building the ifmedia list
for a port.  Fix other related issues while here:
- Require port lock for access to link_config.
- Allow 100Mbps operation by tracking the speed in Mbps.  Yes, really.
- New port flag to indicate that the media list is immutable.  It will
  be used in future refinements.

This also fixes a bug where the driver reports incorrect media with
recent firmwares.

MFC after:	2 days
Sponsored by:	Chelsio Communications
2018-05-30 22:36:09 +00:00
Navdeep Parhar
c3fce948fb cxgbe(4): Suppress a warning about code that is used only with options
RATELIMIT.

Reported by:	mmacy@
2018-05-25 18:57:41 +00:00
Navdeep Parhar
475d42db4a cxgbe(4): Report IFCAP_TXRTLMT to kernels built with RATELIMIT if the
firmware has provisioned resources for this feature.

Sponsored by:	Chelsio Communications
2018-05-24 10:55:26 +00:00
Navdeep Parhar
786099de5e cxgbe(4): Data path for rate-limited tx.
This is hardware support for the SO_MAX_PACING_RATE sockopt (see
setsockopt(2)), which is available in kernels built with "options
RATELIMIT".

Relnotes:	Yes
Sponsored by:	Chelsio Communications
2018-05-24 10:18:14 +00:00
Navdeep Parhar
c90a8cf80a cxgbe/t4_tom: ABORT_RPL_RSS is a shared CPL and t4_tom shouldn't remove
the global handler when it's being unloaded.
2018-05-24 08:32:02 +00:00
Navdeep Parhar
9c707b3287 cxgbe(4): Make FW4_ACK a shared CPL. ETHOFLD in the base driver will
use it for per-flow rate limiting.

Sponsored by:	Chelsio Communications
2018-05-24 08:21:43 +00:00
Navdeep Parhar
1dd95f641e cxgbe(4): Fix range checks in is_etid. 2018-05-24 08:02:11 +00:00
Navdeep Parhar
a6a8ff351d cxgbe(4): Slightly simpler needs_<foo> functions. 2018-05-24 07:38:46 +00:00
Navdeep Parhar
2e09fe9116 cxgbe(4): Make sure that the egress queue's cidx is updated periodically
when the driver is writing WRs using start_wrq_wr/commit_wrq_wr all the
time.

Sponsored by:	Chelsio Communications
2018-05-24 06:44:06 +00:00
Navdeep Parhar
80259b6c12 cxgbe(4): Only valid filters are expected to have a valid tid. 2018-05-22 16:23:14 +00:00
Matt Macy
d7c5a620e2 ifnet: Replace if_addr_lock rwlock with epoch + mutex
Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
4.98   0.00   4.42   0.00 4235592     33   83.80 4720653 2149771   1235 247.32
4.73   0.00   4.20   0.00 4025260     33   82.99 4724900 2139833   1204 247.32
4.72   0.00   4.20   0.00 4035252     33   82.14 4719162 2132023   1264 247.32
4.71   0.00   4.21   0.00 4073206     33   83.68 4744973 2123317   1347 247.32
4.72   0.00   4.21   0.00 4061118     33   80.82 4713615 2188091   1490 247.32
4.72   0.00   4.21   0.00 4051675     33   85.29 4727399 2109011   1205 247.32
4.73   0.00   4.21   0.00 4039056     33   84.65 4724735 2102603   1053 247.32

After the patch

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
5.43   0.00   4.20   0.00 3313143     33   84.96 5434214 1900162   2656 245.51
5.43   0.00   4.20   0.00 3308527     33   85.24 5439695 1809382   2521 245.51
5.42   0.00   4.19   0.00 3316778     33   87.54 5416028 1805835   2256 245.51
5.42   0.00   4.19   0.00 3317673     33   90.44 5426044 1763056   2332 245.51
5.42   0.00   4.19   0.00 3314839     33   88.11 5435732 1792218   2499 245.52
5.44   0.00   4.19   0.00 3293228     33   91.84 5426301 1668597   2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
John Baldwin
24ddd0ec9c Be more robust against garbage input on a TOE TLS TX socket.
If a socket is closed or shutdown and a partial record (or what
appears to be a partial record) is waiting in the socket buffer,
discard the partial record and close the connection rather than
waiting forever for the rest of the record.

Reported by:	Harsh Jain @ Chelsio
Sponsored by:	Chelsio Communications
2018-05-18 19:09:11 +00:00
Navdeep Parhar
67e071128d cxgbe(4): Implement ifnet callbacks that deal with send tags.
An etid (ethoffload tid) is allocated for a send tag and it acquires a
reference on the traffic class that matches the send parameters
associated with the tag.

Sponsored by:	Chelsio Communications
2018-05-18 06:09:15 +00:00
Navdeep Parhar
d88a0442bb cxgbe(4): Fix s->neq miscalculation in r333698. 2018-05-17 06:04:50 +00:00
Navdeep Parhar
eff62dba61 cxgbe(4): Allocate offload Tx queues when a card has resources
provisioned for NIC_ETHOFLD and the kernel has option RATELIMIT.

It is possible to use the chip's offload queues for normal NIC Tx and
not just TOE Tx.  The difference is that these queues support out of
order processing of work requests and have a per-"flowid" mechanism for
tracking credits between the driver and hardware.  This allows Tx for
any number of flows bound to different rate limits to be submitted to a
single Tx queue and the work requests for slow flows won't cause HOL
blocking for the rest.

Sponsored by:	Chelsio Communications
2018-05-17 01:42:18 +00:00
Navdeep Parhar
93c0bfb85b cxgbe(4): Add NIC_ETHOFLD to the NIC capabilities allowed by the driver
by default.

This is the first of a series of commits that will add support for
RATELIMIT kernel option to the base if_cxgbe driver, for use with
ordinary NIC traffic "flows".  RATELIMIT is already supported by t4_tom
for the fully-offloaded TCP connections that it handles.

Sponsored by:	Chelsio Communications
2018-05-17 00:52:48 +00:00
Navdeep Parhar
47ae7a7e4f cxgbe(4): Fall back to a failsafe configuration built into the firmware
if an error is reported while pre-processing the configuration file that
the driver attempted to use.

Also, allow the user to explicitly use the built-in configuration with
hw.cxgbe.config_file="built-in"

MFC after:	2 days
Sponsored by:	Chelsio Communications
2018-05-16 17:55:16 +00:00
Navdeep Parhar
b6ce1d5b1d cxgbe(4): Add support for two more flash parts.
Obtained from:	Chelsio Communications
MFC after:	2 days
Sponsored by:	Chelsio Communications
2018-05-15 22:26:09 +00:00
Navdeep Parhar
40f242e440 cxgbe(4): Claim some more T5 and T6 boards.
MFC after:	2 days
Sponsored by:	Chelsio Communications
2018-05-15 21:54:59 +00:00
Navdeep Parhar
b3daa684d8 cxgbe(4): Filtering related features and fixes.
- Driver support for hardware NAT.
- Driver support for swapmac action.
- Validate a request to create a hashfilter against the filter mask.
- Add a hashfilter config file for T5.

Sponsored by:	Chelsio Communications
2018-05-15 04:24:38 +00:00
Navdeep Parhar
f348cdad1a cxgbe(4): Add fields to support configuration of hardware NAT and
swapmac (SMAC/DMAC switcheroo) from userspace.

Sponsored by:	Chelsio Communications
2018-05-10 20:39:04 +00:00