upstream/opnsense-src
1
0
Fork 0
mirror of https://github.com/opnsense/src.git synced 2026-04-29 18:32:49 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
opnsense-src/sys/dev/virtio/network/if_vtnet.c

4431 lines
108 KiB
C
Raw Normal View History

Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*-
sys/dev: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
2017-11-27 09:52:40 -05:00
* SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
Merge virtio changes from projects/virtio Contains projects/virtio commits: r245738: virtio: Minor man page tweaks r246060: virtio: Cleanup feature description printing r246306: virtio: Remove old debugging flag r247238: virtio: Remove PRIx64 macros from format strings r247239: virtio: Constify some fields r247240: virtio: Minor code simplifications r249962: virtio: Update to my freebsd.org email address MFC after: 1 month
2013-07-04 13:57:26 -04:00
* Copyright (c) 2011, Bryan Venteicher <bryanv@FreeBSD.org>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice unmodified, this list of conditions, and the following
* disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* Driver for VirtIO network devices. */
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
Include necessary headers that now are available due to pollution via if_var.h. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-28 03:29:16 -04:00
#include <sys/eventhandler.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <sys/systm.h>
#include <sys/kernel.h>
#include <sys/sockio.h>
#include <sys/malloc.h>
virtio: Add KMSAN hooks for network and block devices This ensures that host-written data is marked as initialized. Sponsored by: The FreeBSD Foundation
2021-08-11 16:23:25 -04:00
#include <sys/mbuf.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <sys/module.h>
virtio: Add KMSAN hooks for network and block devices This ensures that host-written data is marked as initialized. Sponsored by: The FreeBSD Foundation
2021-08-11 16:23:25 -04:00
#include <sys/msan.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <sys/socket.h>
#include <sys/sysctl.h>
#include <sys/random.h>
#include <sys/sglist.h>
#include <sys/lock.h>
#include <sys/mutex.h>
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#include <sys/taskqueue.h>
#include <sys/smp.h>
#include <machine/smp.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <vm/uma.h>
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
#include <net/debugnet.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <net/ethernet.h>
Add pfil(9) hook to vtnet(4). The patch could be simplier, using only the second chunk to vtnet_rxq_eof(), that passes full mbufs to pfil(9). Packet filter would m_free() them in case of returning PFIL_DROPPED. However, we pretend to be a hardware driver, so we first try to pass a memory buffer via PFIL_MEMPTR feature. This is mostly done for debugging purposes, so that one can experiment in bhyve with packet filters utilizing same features as a true driver.
2020-01-10 16:22:03 -05:00
#include <net/pfil.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <net/if.h>
The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare to this event, adding if_var.h to files that do need it. Also, include all includes that now are included due to implicit pollution via if_var.h Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-26 13:58:36 -04:00
#include <net/if_var.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <net/if_arp.h>
#include <net/if_dl.h>
#include <net/if_types.h>
#include <net/if_media.h>
#include <net/if_vlan_var.h>
#include <net/bpf.h>
#include <netinet/in_systm.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <netinet/ip6.h>
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#include <netinet6/ip6_var.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <netinet/udp.h>
#include <netinet/tcp.h>
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#include <netinet/tcp_lro.h>
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#include <machine/bus.h>
#include <machine/resource.h>
#include <sys/bus.h>
#include <sys/rman.h>
#include <dev/virtio/virtio.h>
#include <dev/virtio/virtqueue.h>
#include <dev/virtio/network/virtio_net.h>
#include <dev/virtio/network/if_vtnetvar.h>
#include "virtio_if.h"
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#include "opt_inet.h"
#include "opt_inet6.h"
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
#if defined(INET) || defined(INET6)
#include <machine/in_cksum.h>
#endif
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_modevent(module_t, int, void *);
static int vtnet_probe(device_t);
static int vtnet_attach(device_t);
static int vtnet_detach(device_t);
static int vtnet_suspend(device_t);
static int vtnet_resume(device_t);
static int vtnet_shutdown(device_t);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_attach_completed(device_t);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_config_change(device_t);
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
static int vtnet_negotiate_features(struct vtnet_softc *);
static int vtnet_setup_features(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_init_rxq(struct vtnet_softc *, int);
static int vtnet_init_txq(struct vtnet_softc *, int);
static int vtnet_alloc_rxtx_queues(struct vtnet_softc *);
static void vtnet_free_rxtx_queues(struct vtnet_softc *);
static int vtnet_alloc_rx_filters(struct vtnet_softc *);
static void vtnet_free_rx_filters(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_alloc_virtqueues(struct vtnet_softc *);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
static int vtnet_alloc_interface(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_setup_interface(struct vtnet_softc *);
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
static int vtnet_ioctl_mtu(struct vtnet_softc *, u_int);
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
static int vtnet_ioctl_ifflags(struct vtnet_softc *);
static int vtnet_ioctl_multi(struct vtnet_softc *);
static int vtnet_ioctl_ifcap(struct vtnet_softc *, struct ifreq *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_ioctl(struct ifnet *, u_long, caddr_t);
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
static uint64_t vtnet_get_counter(struct ifnet *, ift_counter);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_rxq_populate(struct vtnet_rxq *);
static void vtnet_rxq_free_mbufs(struct vtnet_rxq *);
static struct mbuf *
vtnet_rx_alloc_buf(struct vtnet_softc *, int , struct mbuf **);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
static int vtnet_rxq_replace_lro_nomrg_buf(struct vtnet_rxq *,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct mbuf *, int);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_rxq_replace_buf(struct vtnet_rxq *, struct mbuf *, int);
static int vtnet_rxq_enqueue_buf(struct vtnet_rxq *, struct mbuf *);
static int vtnet_rxq_new_buf(struct vtnet_rxq *);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
static int vtnet_rxq_csum_needs_csum(struct vtnet_rxq *, struct mbuf *,
uint16_t, int, struct virtio_net_hdr *);
static int vtnet_rxq_csum_data_valid(struct vtnet_rxq *, struct mbuf *,
uint16_t, int, struct virtio_net_hdr *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_rxq_csum(struct vtnet_rxq *, struct mbuf *,
struct virtio_net_hdr *);
static void vtnet_rxq_discard_merged_bufs(struct vtnet_rxq *, int);
static void vtnet_rxq_discard_buf(struct vtnet_rxq *, struct mbuf *);
static int vtnet_rxq_merged_eof(struct vtnet_rxq *, struct mbuf *, int);
static void vtnet_rxq_input(struct vtnet_rxq *, struct mbuf *,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct virtio_net_hdr *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_rxq_eof(struct vtnet_rxq *);
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
static void vtnet_rx_vq_process(struct vtnet_rxq *rxq, int tries);
Convert VirtIO to use ithreads instead of taskqueues Contains projects/virtio commits: r245709: Each VirtIO device was scheduling its own taskqueue(9) to do the off-level interrupt handling. ithreads(9) is the more nature way to do this. The primary motivation for this work to better support network multiqueue. r245710: virtio: Change virtqueue intr handlers to return void r245711: virtio_blk: Remove interrupt taskqueue r245721: vtnet: Remove interrupt taskqueue r245722: virtio_scsi: Remove interrupt taskqueue r245747: vtnet: Remove taskqueue fields missed in r245721 MFC after: 1 month
2013-07-04 13:50:11 -04:00
static void vtnet_rx_vq_intr(void *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_rxq_tq_intr(void *, int);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
static int vtnet_txq_intr_threshold(struct vtnet_txq *);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static int vtnet_txq_below_threshold(struct vtnet_txq *);
static int vtnet_txq_notify(struct vtnet_txq *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_txq_free_mbufs(struct vtnet_txq *);
static int vtnet_txq_offload_ctx(struct vtnet_txq *, struct mbuf *,
int *, int *, int *);
static int vtnet_txq_offload_tso(struct vtnet_txq *, struct mbuf *, int,
int, struct virtio_net_hdr *);
static struct mbuf *
vtnet_txq_offload(struct vtnet_txq *, struct mbuf *,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct virtio_net_hdr *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_txq_enqueue_buf(struct vtnet_txq *, struct mbuf **,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct vtnet_tx_header *);
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
static int vtnet_txq_encap(struct vtnet_txq *, struct mbuf **, int);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#ifdef VTNET_LEGACY_TX
static void vtnet_start_locked(struct vtnet_txq *, struct ifnet *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_start(struct ifnet *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#else
static int vtnet_txq_mq_start_locked(struct vtnet_txq *, struct mbuf *);
static int vtnet_txq_mq_start(struct ifnet *, struct mbuf *);
static void vtnet_txq_tq_deferred(void *, int);
#endif
Move duplicated transmit start code into a single function
2014-01-25 14:55:42 -05:00
static void vtnet_txq_start(struct vtnet_txq *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_txq_tq_intr(void *, int);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static int vtnet_txq_eof(struct vtnet_txq *);
Convert VirtIO to use ithreads instead of taskqueues Contains projects/virtio commits: r245709: Each VirtIO device was scheduling its own taskqueue(9) to do the off-level interrupt handling. ithreads(9) is the more nature way to do this. The primary motivation for this work to better support network multiqueue. r245710: virtio: Change virtqueue intr handlers to return void r245711: virtio_blk: Remove interrupt taskqueue r245721: vtnet: Remove interrupt taskqueue r245722: virtio_scsi: Remove interrupt taskqueue r245747: vtnet: Remove taskqueue fields missed in r245721 MFC after: 1 month
2013-07-04 13:50:11 -04:00
static void vtnet_tx_vq_intr(void *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_tx_start_all(struct vtnet_softc *);
#ifndef VTNET_LEGACY_TX
static void vtnet_qflush(struct ifnet *);
#endif
static int vtnet_watchdog(struct vtnet_txq *);
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
static void vtnet_accum_stats(struct vtnet_softc *,
struct vtnet_rxq_stats *, struct vtnet_txq_stats *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_tick(void *);
static void vtnet_start_taskqueues(struct vtnet_softc *);
static void vtnet_free_taskqueues(struct vtnet_softc *);
static void vtnet_drain_taskqueues(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_drain_rxtx_queues(struct vtnet_softc *);
static void vtnet_stop_rendezvous(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_stop(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_virtio_reinit(struct vtnet_softc *);
static void vtnet_init_rx_filters(struct vtnet_softc *);
static int vtnet_init_rx_queues(struct vtnet_softc *);
static int vtnet_init_tx_queues(struct vtnet_softc *);
static int vtnet_init_rxtx_queues(struct vtnet_softc *);
static void vtnet_set_active_vq_pairs(struct vtnet_softc *);
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
static void vtnet_update_rx_offloads(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_reinit(struct vtnet_softc *);
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
static void vtnet_init_locked(struct vtnet_softc *, int);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_init(void *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_free_ctrl_vq(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_exec_ctrl_cmd(struct vtnet_softc *, void *,
struct sglist *, int, int);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_ctrl_mac_cmd(struct vtnet_softc *, uint8_t *);
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
static int vtnet_ctrl_guest_offloads(struct vtnet_softc *, uint64_t);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_ctrl_mq_cmd(struct vtnet_softc *, uint16_t);
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
static int vtnet_ctrl_rx_cmd(struct vtnet_softc *, uint8_t, bool);
static int vtnet_set_promisc(struct vtnet_softc *, bool);
static int vtnet_set_allmulti(struct vtnet_softc *, bool);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_rx_filter(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_rx_filter_mac(struct vtnet_softc *);
static int vtnet_exec_vlan_filter(struct vtnet_softc *, int, uint16_t);
static void vtnet_rx_filter_vlan(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_update_vlan_filter(struct vtnet_softc *, int, uint16_t);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void vtnet_register_vlan(void *, struct ifnet *, uint16_t);
static void vtnet_unregister_vlan(void *, struct ifnet *, uint16_t);
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
static void vtnet_update_speed_duplex(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_is_link_up(struct vtnet_softc *);
static void vtnet_update_link_status(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_ifmedia_upd(struct ifnet *);
static void vtnet_ifmedia_sts(struct ifnet *, struct ifmediareq *);
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
static void vtnet_get_macaddr(struct vtnet_softc *);
static void vtnet_set_macaddr(struct vtnet_softc *);
static void vtnet_attached_set_macaddr(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_vlan_tag_remove(struct mbuf *);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static void vtnet_set_rx_process_limit(struct vtnet_softc *);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_setup_rxq_sysctl(struct sysctl_ctx_list *,
struct sysctl_oid_list *, struct vtnet_rxq *);
static void vtnet_setup_txq_sysctl(struct sysctl_ctx_list *,
struct sysctl_oid_list *, struct vtnet_txq *);
static void vtnet_setup_queue_sysctl(struct vtnet_softc *);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
static void vtnet_load_tunables(struct vtnet_softc *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void vtnet_setup_sysctl(struct vtnet_softc *);
static int vtnet_rxq_enable_intr(struct vtnet_rxq *);
static void vtnet_rxq_disable_intr(struct vtnet_rxq *);
static int vtnet_txq_enable_intr(struct vtnet_txq *);
static void vtnet_txq_disable_intr(struct vtnet_txq *);
static void vtnet_enable_rx_interrupts(struct vtnet_softc *);
static void vtnet_enable_tx_interrupts(struct vtnet_softc *);
static void vtnet_enable_interrupts(struct vtnet_softc *);
static void vtnet_disable_rx_interrupts(struct vtnet_softc *);
static void vtnet_disable_tx_interrupts(struct vtnet_softc *);
static void vtnet_disable_interrupts(struct vtnet_softc *);
static int vtnet_tunable_int(struct vtnet_softc *, const char *, int);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
DEBUGNET_DEFINE(vtnet);
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
#define vtnet_htog16(_sc, _val) virtio_htog16(vtnet_modern(_sc), _val)
#define vtnet_htog32(_sc, _val) virtio_htog32(vtnet_modern(_sc), _val)
#define vtnet_htog64(_sc, _val) virtio_htog64(vtnet_modern(_sc), _val)
#define vtnet_gtoh16(_sc, _val) virtio_gtoh16(vtnet_modern(_sc), _val)
#define vtnet_gtoh32(_sc, _val) virtio_gtoh32(vtnet_modern(_sc), _val)
#define vtnet_gtoh64(_sc, _val) virtio_gtoh64(vtnet_modern(_sc), _val)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/* Tunables. */
Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718
2020-02-26 09:26:36 -05:00
static SYSCTL_NODE(_hw, OID_AUTO, vtnet, CTLFLAG_RD | CTLFLAG_MPSAFE, 0,
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
"VirtIO Net driver parameters");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_csum_disable = 0;
Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay
2016-08-11 17:13:58 -04:00
SYSCTL_INT(_hw_vtnet, OID_AUTO, csum_disable, CTLFLAG_RDTUN,
&vtnet_csum_disable, 0, "Disables receive and send checksum offload");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
static int vtnet_fixup_needs_csum = 0;
SYSCTL_INT(_hw_vtnet, OID_AUTO, fixup_needs_csum, CTLFLAG_RDTUN,
&vtnet_fixup_needs_csum, 0,
"Calculate valid checksum for NEEDS_CSUM packets");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_tso_disable = 0;
if_vtnet: Remove unnecessary TUNABLE_INTs because of CTLFLAG_RDTUN Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27923
2021-01-18 23:55:25 -05:00
SYSCTL_INT(_hw_vtnet, OID_AUTO, tso_disable, CTLFLAG_RDTUN,
&vtnet_tso_disable, 0, "Disables TSO");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int vtnet_lro_disable = 0;
if_vtnet: Remove unnecessary TUNABLE_INTs because of CTLFLAG_RDTUN Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27923
2021-01-18 23:55:25 -05:00
SYSCTL_INT(_hw_vtnet, OID_AUTO, lro_disable, CTLFLAG_RDTUN,
&vtnet_lro_disable, 0, "Disables hardware LRO");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int vtnet_mq_disable = 0;
if_vtnet: Remove unnecessary TUNABLE_INTs because of CTLFLAG_RDTUN Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27923
2021-01-18 23:55:25 -05:00
SYSCTL_INT(_hw_vtnet, OID_AUTO, mq_disable, CTLFLAG_RDTUN,
&vtnet_mq_disable, 0, "Disables multiqueue support");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay
2016-08-11 17:13:58 -04:00
static int vtnet_mq_max_pairs = VTNET_MAX_QUEUE_PAIRS;
SYSCTL_INT(_hw_vtnet, OID_AUTO, mq_max_pairs, CTLFLAG_RDTUN,
if_vtnet: Remove unnecessary TUNABLE_INTs because of CTLFLAG_RDTUN Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27923
2021-01-18 23:55:25 -05:00
&vtnet_mq_max_pairs, 0, "Maximum number of multiqueue pairs");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if_vtnet: Set the interface max TSO values Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27917
2021-01-18 23:55:25 -05:00
static int vtnet_tso_maxlen = IP_MAXPACKET;
SYSCTL_INT(_hw_vtnet, OID_AUTO, tso_maxlen, CTLFLAG_RDTUN,
&vtnet_tso_maxlen, 0, "TSO burst limit");
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
static int vtnet_rx_process_limit = 1024;
Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay
2016-08-11 17:13:58 -04:00
SYSCTL_INT(_hw_vtnet, OID_AUTO, rx_process_limit, CTLFLAG_RDTUN,
if_vtnet: Remove unnecessary TUNABLE_INTs because of CTLFLAG_RDTUN Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27923
2021-01-18 23:55:25 -05:00
&vtnet_rx_process_limit, 0,
"Number of RX segments processed in one pass");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
static int vtnet_lro_entry_count = 128;
SYSCTL_INT(_hw_vtnet, OID_AUTO, lro_entry_count, CTLFLAG_RDTUN,
&vtnet_lro_entry_count, 0, "Software LRO entry count");
/* Enable sorted LRO, and the depth of the mbuf queue. */
static int vtnet_lro_mbufq_depth = 0;
SYSCTL_UINT(_hw_vtnet, OID_AUTO, lro_mbufq_depth, CTLFLAG_RDTUN,
&vtnet_lro_mbufq_depth, 0, "Depth of software LRO mbuf queue");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static uma_zone_t vtnet_tx_header_zone;
static struct virtio_feature_desc vtnet_feature_desc[] = {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
{ VIRTIO_NET_F_CSUM, "TxChecksum" },
{ VIRTIO_NET_F_GUEST_CSUM, "RxChecksum" },
{ VIRTIO_NET_F_CTRL_GUEST_OFFLOADS, "CtrlRxOffloads" },
{ VIRTIO_NET_F_MAC, "MAC" },
{ VIRTIO_NET_F_GSO, "TxGSO" },
{ VIRTIO_NET_F_GUEST_TSO4, "RxLROv4" },
{ VIRTIO_NET_F_GUEST_TSO6, "RxLROv6" },
{ VIRTIO_NET_F_GUEST_ECN, "RxLROECN" },
{ VIRTIO_NET_F_GUEST_UFO, "RxUFO" },
{ VIRTIO_NET_F_HOST_TSO4, "TxTSOv4" },
{ VIRTIO_NET_F_HOST_TSO6, "TxTSOv6" },
{ VIRTIO_NET_F_HOST_ECN, "TxTSOECN" },
{ VIRTIO_NET_F_HOST_UFO, "TxUFO" },
{ VIRTIO_NET_F_MRG_RXBUF, "MrgRxBuf" },
{ VIRTIO_NET_F_STATUS, "Status" },
{ VIRTIO_NET_F_CTRL_VQ, "CtrlVq" },
{ VIRTIO_NET_F_CTRL_RX, "CtrlRxMode" },
{ VIRTIO_NET_F_CTRL_VLAN, "CtrlVLANFilter" },
{ VIRTIO_NET_F_CTRL_RX_EXTRA, "CtrlRxModeExtra" },
{ VIRTIO_NET_F_GUEST_ANNOUNCE, "GuestAnnounce" },
{ VIRTIO_NET_F_MQ, "Multiqueue" },
{ VIRTIO_NET_F_CTRL_MAC_ADDR, "CtrlMacAddr" },
{ VIRTIO_NET_F_SPEED_DUPLEX, "SpeedDuplex" },
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{ 0, NULL }
};
static device_method_t vtnet_methods[] = {
/* Device methods. */
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
DEVMETHOD(device_probe, vtnet_probe),
DEVMETHOD(device_attach, vtnet_attach),
DEVMETHOD(device_detach, vtnet_detach),
DEVMETHOD(device_suspend, vtnet_suspend),
DEVMETHOD(device_resume, vtnet_resume),
DEVMETHOD(device_shutdown, vtnet_shutdown),
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/* VirtIO methods. */
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
DEVMETHOD(virtio_attach_completed, vtnet_attach_completed),
DEVMETHOD(virtio_config_change, vtnet_config_change),
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Catch up with Bryan Venteicher's virtio git repo: a8af6270bd96be6ccd86f70b60fa6512b710e4f0 virtio_blk: Include function name in panic string cbdb03a694b76c5253d7ae3a59b9995b9afbb67a virtio_balloon: Do the notify outside of the lock By the time we return from virtqueue_notify(), the descriptor will be in the used ring so we shouldn't have to sleep. 10ba392e60692529a5cbc1e9987e4064e0128447 virtio: Use DEVMETHOD_END 80cbcc4d6552cac758be67f0c99c36f23ce62110 virtqueue: Add support for VIRTIO_F_RING_EVENT_IDX This can be used to reduce the number of guest/host and host/guest interrupts by delaying the interrupt until a certain index value is reached. Actual use by the network driver will come along later. 8fc465969acc0c58477153e4c3530390db436c02 virtqueue: Simplify virtqueue_nused() Since the values just wrap naturally at UINT16_MAX, we can just subtract the two values directly, rather than doing 2's complement math. a8aa22f25959e2767d006cd621b69050e7ffb0ae virtio_blk: Remove debugging crud from 75dd732a There seems to be an issue with Qemu (or FreeBSD VirtIO) that sets the PCI register space for the device config to bogus values. This only seems to happen after unloading and reloading the module. d404800661cb2a9769c033f8a50b2133934501aa virtio_blk: Use better variable name 75dd732a97743d96e7c63f7ced3c2169696dadd3 virtio_blk: Partially revert 92ba40e65 Just use the virtqueue to determine if any requests are still inflight. 06661ed66b7a9efaea240f99f414c368f1bbcdc7 virtio_blk: error if allowed too few segments Should never happen unless the host provides use with a bogus seg_max value. 4b33e5085bc87a818433d7e664a0a2c8f56a1a89 virtio_blk: Sort function declarations 426b9f5cac892c9c64cc7631966461514f7e08c6 virtio_blk: Cleanup whitespace 617c23e12c61e3c2233d942db713c6b8ff0bd112 virtio_blk: Call disk_err() on error'd completed requests 081a5712d4b2e0abf273be4d26affcf3870263a9 virtio_blk: ASSERT the ready and inflight request queues are empty a9be2631a4f770a84145c18ee03a3f103bed4ca8 virtio_blk: Simplify check for too many segments At the cost of a small style violation. e00ec09da014f2e60cc75542d0ab78898672d521 virtio_blk: Add beginnings of suspend/resume Still not sure if we need to virtio_stop()/virtio_reinit() the device before/after a suspend. Don't start additional IO when marked as suspending. 47c71dc6ce8c238aa59ce8afd4bda5aa294bc884 virtio_blk: Panic when dealt an unhandled BIO cmd 1055544f90fb8c0cc6a2395f5b6104039606aafe virtio_blk: Add VQ enqueue/dequeue wrappers Wrapper functions managed the added/removing to the in-flight list of requests. Normally biodone() any completed IO when draining the virtqueue. 92ba40e65b3bb5e4acb9300ece711f1ea8f3f7f4 virtio_blk: Add in-flight list of requests 74f6d260e075443544522c0833dc2712dd93f49b virtio_blk: Rename VTBLK_FLAG_DETACHING to VTBLK_FLAG_DETACH 7aa549050f6fc6551c09c6362ed6b2a0728956ef virtio_blk: Finish all BIOs through vtblk_finish_bio() Also properly set bio_resid in the case of errors. Most geom_disk providers seem to do the same. 9eef6d0e6f7e5dd362f71ba097f2e2e4c3744882 Added function to translate VirtIO status to error code ef06adc337f31e1129d6d5f26de6d8d1be27bcd2 Reset dumping flag when given unexpected parameters 393b3e390c644193a2e392220dcc6a6c50b212d9 Added missing VTBLK_LOCK() in dump handler Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2012-04-14 01:48:04 -04:00
DEVMETHOD_END
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
};
Update to the current version of netmap. Mostly bugfixes or features developed in the past 6 months, so this is a 10.1 candidate. Basically no user API changes (some bugfixes in sys/net/netmap_user.h). In detail: 1. netmap support for virtio-net, including in netmap mode. Under bhyve and with a netmap backend [2] we reach over 1Mpps with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode. 2. (kernel) add support for multiple memory allocators, so we can better partition physical and virtual interfaces giving access to separate users. The most visible effect is one additional argument to the various kernel functions to compute buffer addresses. All netmap-supported drivers are affected, but changes are mechanical and trivial 3. (kernel) simplify the prototype for *txsync() and *rxsync() driver methods. All netmap drivers affected, changes mostly mechanical. 4. add support for netmap-monitor ports. Think of it as a mirroring port on a physical switch: a netmap monitor port replicates traffic present on the main port. Restrictions apply. Drive carefully. 5. if_lem.c: support for various paravirtualization features, experimental and disabled by default. Most of these are described in our ANCS'13 paper [1]. Paravirtualized support in netmap mode is new, and beats the numbers in the paper by a large factor (under qemu-kvm, we measured gues-host throughput up to 10-12 Mpps). A lot of refactoring and additional documentation in the files in sys/dev/netmap, but apart from #2 and #3 above, almost nothing of this stuff is visible to other kernel parts. Example programs in tools/tools/netmap have been updated with bugfixes and to support more of the existing features. This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline. A lot of this code has been contributed by my colleagues at UNIPI, including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella. MFC after: 3 days.
2014-08-16 11:00:01 -04:00
#ifdef DEV_NETMAP
#include <dev/netmap/if_vtnet_netmap.h>
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
#endif
Update to the current version of netmap. Mostly bugfixes or features developed in the past 6 months, so this is a 10.1 candidate. Basically no user API changes (some bugfixes in sys/net/netmap_user.h). In detail: 1. netmap support for virtio-net, including in netmap mode. Under bhyve and with a netmap backend [2] we reach over 1Mpps with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode. 2. (kernel) add support for multiple memory allocators, so we can better partition physical and virtual interfaces giving access to separate users. The most visible effect is one additional argument to the various kernel functions to compute buffer addresses. All netmap-supported drivers are affected, but changes are mechanical and trivial 3. (kernel) simplify the prototype for *txsync() and *rxsync() driver methods. All netmap drivers affected, changes mostly mechanical. 4. add support for netmap-monitor ports. Think of it as a mirroring port on a physical switch: a netmap monitor port replicates traffic present on the main port. Restrictions apply. Drive carefully. 5. if_lem.c: support for various paravirtualization features, experimental and disabled by default. Most of these are described in our ANCS'13 paper [1]. Paravirtualized support in netmap mode is new, and beats the numbers in the paper by a large factor (under qemu-kvm, we measured gues-host throughput up to 10-12 Mpps). A lot of refactoring and additional documentation in the files in sys/dev/netmap, but apart from #2 and #3 above, almost nothing of this stuff is visible to other kernel parts. Example programs in tools/tools/netmap have been updated with bugfixes and to support more of the existing features. This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline. A lot of this code has been contributed by my colleagues at UNIPI, including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella. MFC after: 3 days.
2014-08-16 11:00:01 -04:00
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static driver_t vtnet_driver = {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
.name = "vtnet",
.methods = vtnet_methods,
.size = sizeof(struct vtnet_softc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
};
static devclass_t vtnet_devclass;
virtio: Reduce boilerplate for device driver module definitions Rather than have every device register itself for both virtio_pci and virtio_mmio, provide a VIRTIO_DRIVER_MODULE wrapper to declare both, merge VIRTIO_SIMPLE_PNPTABLE with VIRTIO_SIMPLE_PNPINFO and make the latter register for both buses. This also has the benefit of abstracting away the available transports and their names. Reviewed by: bryanv Differential Revision: https://reviews.freebsd.org/D28073
2021-01-20 20:07:23 -05:00
VIRTIO_DRIVER_MODULE(vtnet, vtnet_driver, vtnet_devclass,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
vtnet_modevent, 0);
MODULE_VERSION(vtnet, 1);
MODULE_DEPEND(vtnet, virtio, 1, 1, 1);
add netmap dependency when compiled as a module
2015-07-10 03:13:14 -04:00
#ifdef DEV_NETMAP
MODULE_DEPEND(vtnet, netmap, 1, 1, 1);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
#endif
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
virtio: Reduce boilerplate for device driver module definitions Rather than have every device register itself for both virtio_pci and virtio_mmio, provide a VIRTIO_DRIVER_MODULE wrapper to declare both, merge VIRTIO_SIMPLE_PNPTABLE with VIRTIO_SIMPLE_PNPINFO and make the latter register for both buses. This also has the benefit of abstracting away the available transports and their names. Reviewed by: bryanv Differential Revision: https://reviews.freebsd.org/D28073
2021-01-20 20:07:23 -05:00
VIRTIO_SIMPLE_PNPINFO(vtnet, VIRTIO_ID_NETWORK, "VirtIO Networking Adapter");
virtio(4): Add PNP match metadata for virtio devices Register MODULE_PNP_INFO for virtio devices using the newbus PNP information provided by the previous commit. Matching can be quite simple; existing probe routines only matched on bus (implicit) and device_type. The same matching criteria are retained exactly, but is now also available to devmatch(8). Reviewed by: bryanv, markj; imp (earlier version) Differential Revision: https://reviews.freebsd.org/D20407
2019-06-03 22:37:11 -04:00
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_modevent(module_t mod __unused, int type, void *unused __unused)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
vtnet: fix panic on unload Since r276367 added the virtio_mmio support vtnet_modevent() gets called twice. This resulted in a memory leak during load and a panic on unload. Count the loads so we only initialise once (just like cxgbe(4)), and only clean up in the final unload. PR: 209428 Submitted by: novel@FreeBSD.org MFC after: 1 week
2016-05-14 02:07:15 -04:00
int error = 0;
static int loaded = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
switch (type) {
case MOD_LOAD:
vtnet: Pre-allocate debugnet data immediately Don't wait until the vtnet_debugnet_init() call happens, because at that point we might already have allocated something from vtnet_tx_header_zone. Some systems showed this panic: vtnet0: link state changed to UP panic: keg vtnet_tx_hdr initialization after use. cpuid = 5 time = 1578427700 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe004db427f0 vpanic() at vpanic+0x17e/frame 0xfffffe004db42850 panic() at panic+0x43/frame 0xfffffe004db428b0 uma_zone_reserve() at uma_zone_reserve+0xf6/frame 0xfffffe004db428f0 vtnet_debugnet_init() at vtnet_debugnet_init+0x77/frame 0xfffffe004db42930 debugnet_any_ifnet_update() at debugnet_any_ifnet_update+0x42/frame 0xfffffe004db42980 do_link_state_change() at do_link_state_change+0x1b3/frame 0xfffffe004db429d0 taskqueue_run_locked() at taskqueue_run_locked+0x178/frame 0xfffffe004db42a30 taskqueue_run() at taskqueue_run+0x4d/frame 0xfffffe004db42a50 ithread_loop() at ithread_loop+0x1d6/frame 0xfffffe004db42ab0 fork_exit() at fork_exit+0x80/frame 0xfffffe004db42af0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe004db42af0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 12 tid 100011 ] Stopped at kdb_enter+0x37: movq $0,0x1084eb6(%rip) db> Reviewed by: cem, markj Differential Revision: https://reviews.freebsd.org/D23073
2020-01-08 05:06:32 -05:00
if (loaded++ == 0) {
vtnet: fix panic on unload Since r276367 added the virtio_mmio support vtnet_modevent() gets called twice. This resulted in a memory leak during load and a panic on unload. Count the loads so we only initialise once (just like cxgbe(4)), and only clean up in the final unload. PR: 209428 Submitted by: novel@FreeBSD.org MFC after: 1 week
2016-05-14 02:07:15 -04:00
vtnet_tx_header_zone = uma_zcreate("vtnet_tx_hdr",
sizeof(struct vtnet_tx_header),
NULL, NULL, NULL, NULL, 0, 0);
vtnet: Pre-allocate debugnet data immediately Don't wait until the vtnet_debugnet_init() call happens, because at that point we might already have allocated something from vtnet_tx_header_zone. Some systems showed this panic: vtnet0: link state changed to UP panic: keg vtnet_tx_hdr initialization after use. cpuid = 5 time = 1578427700 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe004db427f0 vpanic() at vpanic+0x17e/frame 0xfffffe004db42850 panic() at panic+0x43/frame 0xfffffe004db428b0 uma_zone_reserve() at uma_zone_reserve+0xf6/frame 0xfffffe004db428f0 vtnet_debugnet_init() at vtnet_debugnet_init+0x77/frame 0xfffffe004db42930 debugnet_any_ifnet_update() at debugnet_any_ifnet_update+0x42/frame 0xfffffe004db42980 do_link_state_change() at do_link_state_change+0x1b3/frame 0xfffffe004db429d0 taskqueue_run_locked() at taskqueue_run_locked+0x178/frame 0xfffffe004db42a30 taskqueue_run() at taskqueue_run+0x4d/frame 0xfffffe004db42a50 ithread_loop() at ithread_loop+0x1d6/frame 0xfffffe004db42ab0 fork_exit() at fork_exit+0x80/frame 0xfffffe004db42af0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe004db42af0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 12 tid 100011 ] Stopped at kdb_enter+0x37: movq $0,0x1084eb6(%rip) db> Reviewed by: cem, markj Differential Revision: https://reviews.freebsd.org/D23073
2020-01-08 05:06:32 -05:00
#ifdef DEBUGNET
/*
* We need to allocate from this zone in the transmit path, so ensure
* that we have at least one item per header available.
* XXX add a separate zone like we do for mbufs? otherwise we may alloc
* buckets
*/
uma_zone_reserve(vtnet_tx_header_zone, DEBUGNET_MAX_IN_FLIGHT * 2);
uma_prealloc(vtnet_tx_header_zone, DEBUGNET_MAX_IN_FLIGHT * 2);
#endif
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
case MOD_QUIESCE:
if (uma_zone_get_cur(vtnet_tx_header_zone) > 0)
error = EBUSY;
vtnet: fix panic on unload Since r276367 added the virtio_mmio support vtnet_modevent() gets called twice. This resulted in a memory leak during load and a panic on unload. Count the loads so we only initialise once (just like cxgbe(4)), and only clean up in the final unload. PR: 209428 Submitted by: novel@FreeBSD.org MFC after: 1 week
2016-05-14 02:07:15 -04:00
break;
case MOD_UNLOAD:
if (--loaded == 0) {
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
uma_zdestroy(vtnet_tx_header_zone);
vtnet_tx_header_zone = NULL;
}
break;
case MOD_SHUTDOWN:
break;
default:
error = EOPNOTSUPP;
break;
}
return (error);
}
static int
vtnet_probe(device_t dev)
{
virtio(4): Add PNP match metadata for virtio devices Register MODULE_PNP_INFO for virtio devices using the newbus PNP information provided by the previous commit. Matching can be quite simple; existing probe routines only matched on bus (implicit) and device_type. The same matching criteria are retained exactly, but is now also available to devmatch(8). Reviewed by: bryanv, markj; imp (earlier version) Differential Revision: https://reviews.freebsd.org/D20407
2019-06-03 22:37:11 -04:00
return (VIRTIO_SIMPLE_PROBE(dev, vtnet));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
vtnet_attach(device_t dev)
{
struct vtnet_softc *sc;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
sc = device_get_softc(dev);
sc->vtnet_dev = dev;
virtio_set_feature_desc(dev, vtnet_feature_desc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_INIT(sc);
callout_init_mtx(&sc->vtnet_tick_ch, VTNET_CORE_MTX(sc), 0);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
vtnet_load_tunables(sc);
error = vtnet_alloc_interface(sc);
if (error) {
device_printf(dev, "cannot allocate interface\n");
goto fail;
}
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_setup_sysctl(sc);
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
error = vtnet_setup_features(sc);
if (error) {
device_printf(dev, "cannot setup features\n");
goto fail;
}
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_alloc_rx_filters(sc);
if (error) {
device_printf(dev, "cannot allocate Rx filters\n");
goto fail;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_alloc_rxtx_queues(sc);
if (error) {
device_printf(dev, "cannot allocate queues\n");
goto fail;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
error = vtnet_alloc_virtqueues(sc);
if (error) {
device_printf(dev, "cannot allocate virtqueues\n");
goto fail;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_setup_interface(sc);
if (error) {
device_printf(dev, "cannot setup interface\n");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
}
error = virtio_setup_intr(dev, INTR_TYPE_NET);
if (error) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
device_printf(dev, "cannot setup interrupts\n");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ether_ifdetach(sc->vtnet_ifp);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
}
Update to the current version of netmap. Mostly bugfixes or features developed in the past 6 months, so this is a 10.1 candidate. Basically no user API changes (some bugfixes in sys/net/netmap_user.h). In detail: 1. netmap support for virtio-net, including in netmap mode. Under bhyve and with a netmap backend [2] we reach over 1Mpps with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode. 2. (kernel) add support for multiple memory allocators, so we can better partition physical and virtual interfaces giving access to separate users. The most visible effect is one additional argument to the various kernel functions to compute buffer addresses. All netmap-supported drivers are affected, but changes are mechanical and trivial 3. (kernel) simplify the prototype for *txsync() and *rxsync() driver methods. All netmap drivers affected, changes mostly mechanical. 4. add support for netmap-monitor ports. Think of it as a mirroring port on a physical switch: a netmap monitor port replicates traffic present on the main port. Restrictions apply. Drive carefully. 5. if_lem.c: support for various paravirtualization features, experimental and disabled by default. Most of these are described in our ANCS'13 paper [1]. Paravirtualized support in netmap mode is new, and beats the numbers in the paper by a large factor (under qemu-kvm, we measured gues-host throughput up to 10-12 Mpps). A lot of refactoring and additional documentation in the files in sys/dev/netmap, but apart from #2 and #3 above, almost nothing of this stuff is visible to other kernel parts. Example programs in tools/tools/netmap have been updated with bugfixes and to support more of the existing features. This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline. A lot of this code has been contributed by my colleagues at UNIPI, including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella. MFC after: 3 days.
2014-08-16 11:00:01 -04:00
#ifdef DEV_NETMAP
vtnet_netmap_attach(sc);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_start_taskqueues(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
fail:
if (error)
vtnet_detach(dev);
return (error);
}
static int
vtnet_detach(device_t dev)
{
struct vtnet_softc *sc;
struct ifnet *ifp;
sc = device_get_softc(dev);
ifp = sc->vtnet_ifp;
if (device_is_attached(dev)) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
vtnet_stop(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
callout_drain(&sc->vtnet_tick_ch);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_drain_taskqueues(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
ether_ifdetach(ifp);
}
Update to the current version of netmap. Mostly bugfixes or features developed in the past 6 months, so this is a 10.1 candidate. Basically no user API changes (some bugfixes in sys/net/netmap_user.h). In detail: 1. netmap support for virtio-net, including in netmap mode. Under bhyve and with a netmap backend [2] we reach over 1Mpps with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode. 2. (kernel) add support for multiple memory allocators, so we can better partition physical and virtual interfaces giving access to separate users. The most visible effect is one additional argument to the various kernel functions to compute buffer addresses. All netmap-supported drivers are affected, but changes are mechanical and trivial 3. (kernel) simplify the prototype for *txsync() and *rxsync() driver methods. All netmap drivers affected, changes mostly mechanical. 4. add support for netmap-monitor ports. Think of it as a mirroring port on a physical switch: a netmap monitor port replicates traffic present on the main port. Restrictions apply. Drive carefully. 5. if_lem.c: support for various paravirtualization features, experimental and disabled by default. Most of these are described in our ANCS'13 paper [1]. Paravirtualized support in netmap mode is new, and beats the numbers in the paper by a large factor (under qemu-kvm, we measured gues-host throughput up to 10-12 Mpps). A lot of refactoring and additional documentation in the files in sys/dev/netmap, but apart from #2 and #3 above, almost nothing of this stuff is visible to other kernel parts. Example programs in tools/tools/netmap have been updated with bugfixes and to support more of the existing features. This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline. A lot of this code has been contributed by my colleagues at UNIPI, including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella. MFC after: 3 days.
2014-08-16 11:00:01 -04:00
#ifdef DEV_NETMAP
netmap_detach(ifp);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
#endif
Update to the current version of netmap. Mostly bugfixes or features developed in the past 6 months, so this is a 10.1 candidate. Basically no user API changes (some bugfixes in sys/net/netmap_user.h). In detail: 1. netmap support for virtio-net, including in netmap mode. Under bhyve and with a netmap backend [2] we reach over 1Mpps with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode. 2. (kernel) add support for multiple memory allocators, so we can better partition physical and virtual interfaces giving access to separate users. The most visible effect is one additional argument to the various kernel functions to compute buffer addresses. All netmap-supported drivers are affected, but changes are mechanical and trivial 3. (kernel) simplify the prototype for *txsync() and *rxsync() driver methods. All netmap drivers affected, changes mostly mechanical. 4. add support for netmap-monitor ports. Think of it as a mirroring port on a physical switch: a netmap monitor port replicates traffic present on the main port. Restrictions apply. Drive carefully. 5. if_lem.c: support for various paravirtualization features, experimental and disabled by default. Most of these are described in our ANCS'13 paper [1]. Paravirtualized support in netmap mode is new, and beats the numbers in the paper by a large factor (under qemu-kvm, we measured gues-host throughput up to 10-12 Mpps). A lot of refactoring and additional documentation in the files in sys/dev/netmap, but apart from #2 and #3 above, almost nothing of this stuff is visible to other kernel parts. Example programs in tools/tools/netmap have been updated with bugfixes and to support more of the existing features. This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline. A lot of this code has been contributed by my colleagues at UNIPI, including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella. MFC after: 3 days.
2014-08-16 11:00:01 -04:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_free_taskqueues(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (sc->vtnet_vlan_attach != NULL) {
EVENTHANDLER_DEREGISTER(vlan_config, sc->vtnet_vlan_attach);
sc->vtnet_vlan_attach = NULL;
}
if (sc->vtnet_vlan_detach != NULL) {
Fix typo when deregistering the VLAN unconfig event handler Submitted by: Masao Uebayashi <uebayasi@tombiinc.com> MFC after: 3 days
2015-06-13 12:13:31 -04:00
EVENTHANDLER_DEREGISTER(vlan_unconfig, sc->vtnet_vlan_detach);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
sc->vtnet_vlan_detach = NULL;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifmedia_removeall(&sc->vtnet_media);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
if (ifp != NULL) {
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_free(ifp);
sc->vtnet_ifp = NULL;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_free_rxtx_queues(sc);
vtnet_free_rx_filters(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (sc->vtnet_ctrl_vq != NULL)
vtnet_free_ctrl_vq(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_DESTROY(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (0);
}
static int
vtnet_suspend(device_t dev)
{
struct vtnet_softc *sc;
sc = device_get_softc(dev);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
vtnet_stop(sc);
sc->vtnet_flags |= VTNET_FLAG_SUSPENDED;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (0);
}
static int
vtnet_resume(device_t dev)
{
struct vtnet_softc *sc;
struct ifnet *ifp;
sc = device_get_softc(dev);
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (ifp->if_flags & IFF_UP)
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
vtnet_init_locked(sc, 0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
sc->vtnet_flags &= ~VTNET_FLAG_SUSPENDED;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (0);
}
static int
vtnet_shutdown(device_t dev)
{
/*
* Suspend already does all of what we need to
* do here; we just never expect to be resumed.
*/
return (vtnet_suspend(dev));
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_attach_completed(device_t dev)
{
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
struct vtnet_softc *sc;
sc = device_get_softc(dev);
VTNET_CORE_LOCK(sc);
vtnet_attached_set_macaddr(sc);
VTNET_CORE_UNLOCK(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int
vtnet_config_change(device_t dev)
{
struct vtnet_softc *sc;
sc = device_get_softc(dev);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Convert VirtIO to use ithreads instead of taskqueues Contains projects/virtio commits: r245709: Each VirtIO device was scheduling its own taskqueue(9) to do the off-level interrupt handling. ithreads(9) is the more nature way to do this. The primary motivation for this work to better support network multiqueue. r245710: virtio: Change virtqueue intr handlers to return void r245711: virtio_blk: Remove interrupt taskqueue r245721: vtnet: Remove interrupt taskqueue r245722: virtio_scsi: Remove interrupt taskqueue r245747: vtnet: Remove taskqueue fields missed in r245721 MFC after: 1 month
2013-07-04 13:50:11 -04:00
vtnet_update_link_status(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (sc->vtnet_link_active != 0)
vtnet_tx_start_all(sc);
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Convert VirtIO to use ithreads instead of taskqueues Contains projects/virtio commits: r245709: Each VirtIO device was scheduling its own taskqueue(9) to do the off-level interrupt handling. ithreads(9) is the more nature way to do this. The primary motivation for this work to better support network multiqueue. r245710: virtio: Change virtqueue intr handlers to return void r245711: virtio_blk: Remove interrupt taskqueue r245721: vtnet: Remove interrupt taskqueue r245722: virtio_scsi: Remove interrupt taskqueue r245747: vtnet: Remove taskqueue fields missed in r245721 MFC after: 1 month
2013-07-04 13:50:11 -04:00
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
static int
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
vtnet_negotiate_features(struct vtnet_softc *sc)
{
device_t dev;
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
uint64_t features, negotiated_features;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
int no_csum;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
dev = sc->vtnet_dev;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
features = virtio_bus_is_modern(dev) ? VTNET_MODERN_FEATURES :
VTNET_LEGACY_FEATURES;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
* TSO and LRO are only available when their corresponding checksum
* offload feature is also negotiated.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
no_csum = vtnet_tunable_int(sc, "csum_disable", vtnet_csum_disable);
if (no_csum)
features &= ~(VIRTIO_NET_F_CSUM | VIRTIO_NET_F_GUEST_CSUM);
if (no_csum || vtnet_tunable_int(sc, "tso_disable", vtnet_tso_disable))
features &= ~VTNET_TSO_FEATURES;
if (no_csum || vtnet_tunable_int(sc, "lro_disable", vtnet_lro_disable))
features &= ~VTNET_LRO_FEATURES;
Make the feature negotiation code easier to follow MFC after: 1 week
2014-06-16 00:29:28 -04:00
#ifndef VTNET_LEGACY_TX
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (vtnet_tunable_int(sc, "mq_disable", vtnet_mq_disable))
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
features &= ~VIRTIO_NET_F_MQ;
Make the feature negotiation code easier to follow MFC after: 1 week
2014-06-16 00:29:28 -04:00
#else
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
features &= ~VIRTIO_NET_F_MQ;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#endif
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
negotiated_features = virtio_negotiate_features(dev, features);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Disable F_MTU feature if MTU is invalid Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27931
2021-01-18 23:55:25 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MTU)) {
uint16_t mtu;
mtu = virtio_read_dev_config_2(dev,
offsetof(struct virtio_net_config, mtu));
if (mtu < VTNET_MIN_MTU /* || mtu > VTNET_MAX_MTU */) {
device_printf(dev, "Invalid MTU value: %d. "
"MTU feature disabled.\n", mtu);
features &= ~VIRTIO_NET_F_MTU;
negotiated_features =
virtio_negotiate_features(dev, features);
}
}
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MQ)) {
uint16_t npairs;
npairs = virtio_read_dev_config_2(dev,
offsetof(struct virtio_net_config, max_virtqueue_pairs));
if (npairs < VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN ||
npairs > VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX) {
device_printf(dev, "Invalid max_virtqueue_pairs value: "
"%d. Multiqueue feature disabled.\n", npairs);
features &= ~VIRTIO_NET_F_MQ;
negotiated_features =
virtio_negotiate_features(dev, features);
}
}
Make the feature negotiation code easier to follow MFC after: 1 week
2014-06-16 00:29:28 -04:00
if (virtio_with_feature(dev, VTNET_LRO_FEATURES) &&
virtio_with_feature(dev, VIRTIO_NET_F_MRG_RXBUF) == 0) {
/*
* LRO without mergeable buffers requires special care. This
* is not ideal because every receive buffer must be large
* enough to hold the maximum TCP packet, the Ethernet header,
* and the header. This requires up to 34 descriptors with
* MCLBYTES clusters. If we do not have indirect descriptors,
* LRO is disabled since the virtqueue will not contain very
* many receive buffers.
*/
if (!virtio_with_feature(dev, VIRTIO_RING_F_INDIRECT_DESC)) {
device_printf(dev,
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
"Host LRO disabled since both mergeable buffers "
"and indirect descriptors were not negotiated\n");
Make the feature negotiation code easier to follow MFC after: 1 week
2014-06-16 00:29:28 -04:00
features &= ~VTNET_LRO_FEATURES;
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
negotiated_features =
Make the feature negotiation code easier to follow MFC after: 1 week
2014-06-16 00:29:28 -04:00
virtio_negotiate_features(dev, features);
} else
sc->vtnet_flags |= VTNET_FLAG_LRO_NOMRG;
}
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
sc->vtnet_features = negotiated_features;
sc->vtnet_negotiated_features = negotiated_features;
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
return (virtio_finalize_features(dev));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_setup_features(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
device_t dev;
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
dev = sc->vtnet_dev;
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
error = vtnet_negotiate_features(sc);
if (error)
return (error);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (virtio_with_feature(dev, VIRTIO_F_VERSION_1))
sc->vtnet_flags |= VTNET_FLAG_MODERN;
Add softc flag for when the indirect descriptor feature was negotiated MFC after: 2 weeks
2014-12-31 21:06:00 -05:00
if (virtio_with_feature(dev, VIRTIO_RING_F_INDIRECT_DESC))
sc->vtnet_flags |= VTNET_FLAG_INDIRECT;
Complete any pending Tx frames before attempting the next transmit Also complete pending frames in the watchdog function when the EVENT_IDX feature was negotiated just in case the completion interrupt was postponed.
2013-09-02 22:28:31 -04:00
if (virtio_with_feature(dev, VIRTIO_RING_F_EVENT_IDX))
sc->vtnet_flags |= VTNET_FLAG_EVENT_IDX;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MAC)) {
/* This feature should always be negotiated. */
sc->vtnet_flags |= VTNET_FLAG_MAC;
}
if_vtnet: Support VIRTIO_NET_F_MTU This feature lets the guest driver know the maximum MTU size supported by the host device. If set, use this to limit the acceptable MTUs, and improve how the receive mbuf cluster size then is selected. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27907
2021-01-18 23:55:24 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MTU)) {
sc->vtnet_max_mtu = virtio_read_dev_config_2(dev,
offsetof(struct virtio_net_config, mtu));
} else
sc->vtnet_max_mtu = VTNET_MAX_MTU;
Revert: virtio: Support non-legacy network device and queue And subsequent fix 576b099a. By adding the mergable header to the vtnet_rx_header structure, the size was increased by 2 bytes, breaking the alignment of this structure as described the in preceding comments. Furthermore, the mergable header does not belong the structure. With the mergable feature, the header is placed in line with the data, so there is no need for a separate segment, and misleading to follow the mergable header with any padding. The V1 header is effectively identical to mergable header, and the driver has long supported the mergable feature. Revert this so the later changes that add V1 support can show how V1 is derived from the existing mergable buffers support, and to facilitate a later MFC. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27855
2021-01-18 23:55:23 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MRG_RXBUF)) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_flags |= VTNET_FLAG_MRG_RXBUFS;
sc->vtnet_hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
} else if (vtnet_modern(sc)) {
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
/* This is identical to the mergeable header. */
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sc->vtnet_hdr_size = sizeof(struct virtio_net_hdr_v1);
Revert: virtio: Support non-legacy network device and queue And subsequent fix 576b099a. By adding the mergable header to the vtnet_rx_header structure, the size was increased by 2 bytes, breaking the alignment of this structure as described the in preceding comments. Furthermore, the mergable header does not belong the structure. With the mergable feature, the header is placed in line with the data, so there is no need for a separate segment, and misleading to follow the mergable header with any padding. The V1 header is effectively identical to mergable header, and the driver has long supported the mergable feature. Revert this so the later changes that add V1 support can show how V1 is derived from the existing mergable buffers support, and to facilitate a later MFC. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27855
2021-01-18 23:55:23 -05:00
} else
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_hdr_size = sizeof(struct virtio_net_hdr);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (vtnet_modern(sc) || sc->vtnet_flags & VTNET_FLAG_MRG_RXBUFS)
sc->vtnet_rx_nsegs = VTNET_RX_SEGS_HDR_INLINE;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
else if (sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG)
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_rx_nsegs = VTNET_RX_SEGS_LRO_NOMRG;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
else
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_rx_nsegs = VTNET_RX_SEGS_HDR_SEPARATE;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
/*
* Favor "hardware" LRO if negotiated, but support software LRO as
* a fallback; there is usually little benefit (or worse) with both.
*/
if (virtio_with_feature(dev, VIRTIO_NET_F_GUEST_TSO4) == 0 &&
virtio_with_feature(dev, VIRTIO_NET_F_GUEST_TSO6) == 0)
sc->vtnet_flags |= VTNET_FLAG_SW_LRO;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_GSO) ||
virtio_with_feature(dev, VIRTIO_NET_F_HOST_TSO4) ||
virtio_with_feature(dev, VIRTIO_NET_F_HOST_TSO6))
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_tx_nsegs = VTNET_TX_SEGS_MAX;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
else
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_tx_nsegs = VTNET_TX_SEGS_MIN;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
sc->vtnet_req_vq_pairs = 1;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sc->vtnet_max_vq_pairs = 1;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (virtio_with_feature(dev, VIRTIO_NET_F_CTRL_VQ)) {
sc->vtnet_flags |= VTNET_FLAG_CTRL_VQ;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (virtio_with_feature(dev, VIRTIO_NET_F_CTRL_RX))
sc->vtnet_flags |= VTNET_FLAG_CTRL_RX;
if (virtio_with_feature(dev, VIRTIO_NET_F_CTRL_VLAN))
sc->vtnet_flags |= VTNET_FLAG_VLAN_FILTER;
if (virtio_with_feature(dev, VIRTIO_NET_F_CTRL_MAC_ADDR))
sc->vtnet_flags |= VTNET_FLAG_CTRL_MAC;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_MQ)) {
sc->vtnet_max_vq_pairs = virtio_read_dev_config_2(dev,
offsetof(struct virtio_net_config,
max_virtqueue_pairs));
}
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay
2016-08-11 17:13:58 -04:00
if (sc->vtnet_max_vq_pairs > 1) {
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
int req;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
* Limit the maximum number of requested queue pairs to the
* number of CPUs and the configured maximum.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
req = vtnet_tunable_int(sc, "mq_max_pairs", vtnet_mq_max_pairs);
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
if (req < 0)
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
req = 1;
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
if (req == 0)
req = mp_ncpus;
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
if (req > sc->vtnet_max_vq_pairs)
req = sc->vtnet_max_vq_pairs;
if (req > mp_ncpus)
req = mp_ncpus;
if (req > 1) {
sc->vtnet_req_vq_pairs = req;
sc->vtnet_flags |= VTNET_FLAG_MQ;
Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay
2016-08-11 17:13:58 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
virtio: Handle possible failure of virtio_finalize_features() Try to standardize how drivers negotiate feature and the function names Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27930
2021-01-18 23:55:26 -05:00
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_init_rxq(struct vtnet_softc *sc, int id)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_rxq *rxq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq = &sc->vtnet_rxqs[id];
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
snprintf(rxq->vtnrx_name, sizeof(rxq->vtnrx_name), "%s-rx%d",
device_get_nameunit(sc->vtnet_dev), id);
mtx_init(&rxq->vtnrx_mtx, rxq->vtnrx_name, NULL, MTX_DEF);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_sc = sc;
rxq->vtnrx_id = id;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
rxq->vtnrx_sg = sglist_alloc(sc->vtnet_rx_nsegs, M_NOWAIT);
if (rxq->vtnrx_sg == NULL)
return (ENOMEM);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#if defined(INET) || defined(INET6)
if (vtnet_software_lro(sc)) {
if (tcp_lro_init_args(&rxq->vtnrx_lro, sc->vtnet_ifp,
sc->vtnet_lro_entry_count, sc->vtnet_lro_mbufq_depth) != 0)
return (ENOMEM);
}
#endif
Use NET_TASK_INIT() and NET_GROUPTASK_INIT() for drivers that process incoming packets in taskqueue context. Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D23518
2020-02-11 13:57:07 -05:00
NET_TASK_INIT(&rxq->vtnrx_intrtask, 0, vtnet_rxq_tq_intr, rxq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_tq = taskqueue_create(rxq->vtnrx_name, M_NOWAIT,
taskqueue_thread_enqueue, &rxq->vtnrx_tq);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (rxq->vtnrx_tq == NULL ? ENOMEM : 0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_init_txq(struct vtnet_softc *sc, int id)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_txq *txq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq = &sc->vtnet_txqs[id];
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
snprintf(txq->vtntx_name, sizeof(txq->vtntx_name), "%s-tx%d",
device_get_nameunit(sc->vtnet_dev), id);
mtx_init(&txq->vtntx_mtx, txq->vtntx_name, NULL, MTX_DEF);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq->vtntx_sc = sc;
txq->vtntx_id = id;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
txq->vtntx_sg = sglist_alloc(sc->vtnet_tx_nsegs, M_NOWAIT);
if (txq->vtntx_sg == NULL)
return (ENOMEM);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#ifndef VTNET_LEGACY_TX
txq->vtntx_br = buf_ring_alloc(VTNET_DEFAULT_BUFRING_SIZE, M_DEVBUF,
M_NOWAIT, &txq->vtntx_mtx);
if (txq->vtntx_br == NULL)
return (ENOMEM);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
TASK_INIT(&txq->vtntx_defrtask, 0, vtnet_txq_tq_deferred, txq);
#endif
TASK_INIT(&txq->vtntx_intrtask, 0, vtnet_txq_tq_intr, txq);
txq->vtntx_tq = taskqueue_create(txq->vtntx_name, M_NOWAIT,
taskqueue_thread_enqueue, &txq->vtntx_tq);
if (txq->vtntx_tq == NULL)
return (ENOMEM);
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_alloc_rxtx_queues(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int i, npairs, error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
npairs = sc->vtnet_max_vq_pairs;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Revert r327828, r327949, r327953, r328016-r328026, r328041: Uses of mallocarray(9). The use of mallocarray(9) has rocketed the required swap to build FreeBSD. This is likely caused by the allocation size attributes which put extra pressure on the compiler. Given that most of these checks are superfluous we have to choose better where to use mallocarray(9). We still have more uses of mallocarray(9) but hopefully this is enough to bring swap usage to a reasonable level. Reported by: wosch PR: 225197
2018-01-21 10:42:36 -05:00
sc->vtnet_rxqs = malloc(sizeof(struct vtnet_rxq) * npairs, M_DEVBUF,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
M_NOWAIT | M_ZERO);
Revert r327828, r327949, r327953, r328016-r328026, r328041: Uses of mallocarray(9). The use of mallocarray(9) has rocketed the required swap to build FreeBSD. This is likely caused by the allocation size attributes which put extra pressure on the compiler. Given that most of these checks are superfluous we have to choose better where to use mallocarray(9). We still have more uses of mallocarray(9) but hopefully this is enough to bring swap usage to a reasonable level. Reported by: wosch PR: 225197
2018-01-21 10:42:36 -05:00
sc->vtnet_txqs = malloc(sizeof(struct vtnet_txq) * npairs, M_DEVBUF,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
M_NOWAIT | M_ZERO);
if (sc->vtnet_rxqs == NULL || sc->vtnet_txqs == NULL)
return (ENOMEM);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
for (i = 0; i < npairs; i++) {
error = vtnet_init_rxq(sc, i);
if (error)
return (error);
error = vtnet_init_txq(sc, i);
if (error)
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
vtnet_set_rx_process_limit(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_setup_queue_sysctl(sc);
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_destroy_rxq(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_sc = NULL;
rxq->vtnrx_id = -1;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#if defined(INET) || defined(INET6)
tcp_lro_free(&rxq->vtnrx_lro);
#endif
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
if (rxq->vtnrx_sg != NULL) {
sglist_free(rxq->vtnrx_sg);
rxq->vtnrx_sg = NULL;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (mtx_initialized(&rxq->vtnrx_mtx) != 0)
mtx_destroy(&rxq->vtnrx_mtx);
}
static void
vtnet_destroy_txq(struct vtnet_txq *txq)
{
txq->vtntx_sc = NULL;
txq->vtntx_id = -1;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
if (txq->vtntx_sg != NULL) {
sglist_free(txq->vtntx_sg);
txq->vtntx_sg = NULL;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#ifndef VTNET_LEGACY_TX
if (txq->vtntx_br != NULL) {
buf_ring_free(txq->vtntx_br, M_DEVBUF);
txq->vtntx_br = NULL;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (mtx_initialized(&txq->vtntx_mtx) != 0)
mtx_destroy(&txq->vtntx_mtx);
}
static void
vtnet_free_rxtx_queues(struct vtnet_softc *sc)
{
int i;
if (sc->vtnet_rxqs != NULL) {
for (i = 0; i < sc->vtnet_max_vq_pairs; i++)
vtnet_destroy_rxq(&sc->vtnet_rxqs[i]);
free(sc->vtnet_rxqs, M_DEVBUF);
sc->vtnet_rxqs = NULL;
}
if (sc->vtnet_txqs != NULL) {
for (i = 0; i < sc->vtnet_max_vq_pairs; i++)
vtnet_destroy_txq(&sc->vtnet_txqs[i]);
free(sc->vtnet_txqs, M_DEVBUF);
sc->vtnet_txqs = NULL;
}
}
static int
vtnet_alloc_rx_filters(struct vtnet_softc *sc)
{
if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX) {
sc->vtnet_mac_filter = malloc(sizeof(struct vtnet_mac_filter),
M_DEVBUF, M_NOWAIT | M_ZERO);
if (sc->vtnet_mac_filter == NULL)
return (ENOMEM);
}
if (sc->vtnet_flags & VTNET_FLAG_VLAN_FILTER) {
sc->vtnet_vlan_filter = malloc(sizeof(uint32_t) *
VTNET_VLAN_FILTER_NWORDS, M_DEVBUF, M_NOWAIT | M_ZERO);
if (sc->vtnet_vlan_filter == NULL)
return (ENOMEM);
}
return (0);
}
static void
vtnet_free_rx_filters(struct vtnet_softc *sc)
{
if (sc->vtnet_mac_filter != NULL) {
free(sc->vtnet_mac_filter, M_DEVBUF);
sc->vtnet_mac_filter = NULL;
}
if (sc->vtnet_vlan_filter != NULL) {
free(sc->vtnet_vlan_filter, M_DEVBUF);
sc->vtnet_vlan_filter = NULL;
}
}
static int
vtnet_alloc_virtqueues(struct vtnet_softc *sc)
{
device_t dev;
struct vq_alloc_info *info;
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
int i, idx, flags, nvqs, error;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
dev = sc->vtnet_dev;
flags = 0;
nvqs = sc->vtnet_max_vq_pairs * 2;
if (sc->vtnet_flags & VTNET_FLAG_CTRL_VQ)
nvqs++;
Revert r327828, r327949, r327953, r328016-r328026, r328041: Uses of mallocarray(9). The use of mallocarray(9) has rocketed the required swap to build FreeBSD. This is likely caused by the allocation size attributes which put extra pressure on the compiler. Given that most of these checks are superfluous we have to choose better where to use mallocarray(9). We still have more uses of mallocarray(9) but hopefully this is enough to bring swap usage to a reasonable level. Reported by: wosch PR: 225197
2018-01-21 10:42:36 -05:00
info = malloc(sizeof(struct vq_alloc_info) * nvqs, M_TEMP, M_NOWAIT);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (info == NULL)
return (ENOMEM);
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0, idx = 0; i < sc->vtnet_req_vq_pairs; i++, idx += 2) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq = &sc->vtnet_rxqs[i];
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
VQ_ALLOC_INFO_INIT(&info[idx], sc->vtnet_rx_nsegs,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rx_vq_intr, rxq, &rxq->vtnrx_vq,
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
"%s-rx%d", device_get_nameunit(dev), rxq->vtnrx_id);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq = &sc->vtnet_txqs[i];
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
VQ_ALLOC_INFO_INIT(&info[idx+1], sc->vtnet_tx_nsegs,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_tx_vq_intr, txq, &txq->vtntx_vq,
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
"%s-tx%d", device_get_nameunit(dev), txq->vtntx_id);
}
/* These queues will not be used so allocate the minimum resources. */
for (/**/; i < sc->vtnet_max_vq_pairs; i++, idx += 2) {
rxq = &sc->vtnet_rxqs[i];
VQ_ALLOC_INFO_INIT(&info[idx], 0, NULL, rxq, &rxq->vtnrx_vq,
"%s-rx%d", device_get_nameunit(dev), rxq->vtnrx_id);
txq = &sc->vtnet_txqs[i];
VQ_ALLOC_INFO_INIT(&info[idx+1], 0, NULL, txq, &txq->vtntx_vq,
"%s-tx%d", device_get_nameunit(dev), txq->vtntx_id);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
if (sc->vtnet_flags & VTNET_FLAG_CTRL_VQ) {
VQ_ALLOC_INFO_INIT(&info[idx], 0, NULL, NULL,
&sc->vtnet_ctrl_vq, "%s ctrl", device_get_nameunit(dev));
}
/*
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
* TODO: Enable interrupt binding if this is multiqueue. This will
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* only matter when per-virtqueue MSIX is available.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (sc->vtnet_flags & VTNET_FLAG_MQ)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
flags |= 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = virtio_alloc_virtqueues(dev, flags, nvqs, info);
free(info, M_TEMP);
return (error);
}
static int
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
vtnet_alloc_interface(struct vtnet_softc *sc)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
device_t dev;
struct ifnet *ifp;
dev = sc->vtnet_dev;
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
ifp = if_alloc(IFT_ETHER);
if (ifp == NULL)
return (ENOMEM);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
sc->vtnet_ifp = ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_softc = sc;
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
if_initname(ifp, device_get_name(dev), device_get_unit(dev));
return (0);
}
static int
vtnet_setup_interface(struct vtnet_softc *sc)
{
device_t dev;
struct pfil_head_args pa;
struct ifnet *ifp;
dev = sc->vtnet_dev;
ifp = sc->vtnet_ifp;
Although most of the NIC drivers are epoch ready, due to peer pressure switch over to opt-in instead of opt-out for epoch. Instead of IFF_NEEDSEPOCH, provide IFF_KNOWSEPOCH. If driver marks itself with IFF_KNOWSEPOCH, then ether_input() would not enter epoch when processing its packets. Now this will create recursive entrance in epoch in >90% network drivers, but will guarantee safeness of the transition. Mark several tested drivers as IFF_KNOWSEPOCH. Reviewed by: hselasky, jeff, bz, gallatin Differential Revision: https://reviews.freebsd.org/D23674
2020-02-24 16:07:30 -05:00
ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST |
IFF_KNOWSEPOCH;
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
ifp->if_baudrate = IF_Gbps(10);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_init = vtnet_init;
ifp->if_ioctl = vtnet_ioctl;
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
ifp->if_get_counter = vtnet_get_counter;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#ifndef VTNET_LEGACY_TX
ifp->if_transmit = vtnet_txq_mq_start;
ifp->if_qflush = vtnet_qflush;
#else
struct virtqueue *vq = sc->vtnet_txqs[0].vtntx_vq;
ifp->if_start = vtnet_start;
IFQ_SET_MAXLEN(&ifp->if_snd, virtqueue_size(vq) - 1);
ifp->if_snd.ifq_drv_maxlen = virtqueue_size(vq) - 1;
IFQ_SET_READY(&ifp->if_snd);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
vtnet_get_macaddr(sc);
if_vtnet: Only set IFCAP_JUMBO_MTU when jumbo MTU is supported Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27912
2021-01-18 23:55:24 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_STATUS))
ifp->if_capabilities |= IFCAP_LINKSTATE;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
ifmedia_init(&sc->vtnet_media, 0, vtnet_ifmedia_upd, vtnet_ifmedia_sts);
ifmedia_add(&sc->vtnet_media, IFM_ETHER | IFM_AUTO, 0, NULL);
ifmedia_set(&sc->vtnet_media, IFM_ETHER | IFM_AUTO);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (virtio_with_feature(dev, VIRTIO_NET_F_CSUM)) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
int gso;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_capabilities |= IFCAP_TXCSUM | IFCAP_TXCSUM_IPV6;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
gso = virtio_with_feature(dev, VIRTIO_NET_F_GSO);
if (gso || virtio_with_feature(dev, VIRTIO_NET_F_HOST_TSO4))
ifp->if_capabilities |= IFCAP_TSO4;
if (gso || virtio_with_feature(dev, VIRTIO_NET_F_HOST_TSO6))
ifp->if_capabilities |= IFCAP_TSO6;
if (gso || virtio_with_feature(dev, VIRTIO_NET_F_HOST_ECN))
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_flags |= VTNET_FLAG_TSO_ECN;
if_vtnet: Set the interface max TSO values Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27917
2021-01-18 23:55:25 -05:00
if (ifp->if_capabilities & (IFCAP_TSO4 | IFCAP_TSO6)) {
int tso_maxlen;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_capabilities |= IFCAP_VLAN_HWTSO;
if_vtnet: Set the interface max TSO values Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27917
2021-01-18 23:55:25 -05:00
tso_maxlen = vtnet_tunable_int(sc, "tso_maxlen",
vtnet_tso_maxlen);
ifp->if_hw_tsomax = tso_maxlen -
(ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
ifp->if_hw_tsomaxsegcount = sc->vtnet_tx_nsegs - 1;
ifp->if_hw_tsomaxsegsize = PAGE_SIZE;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Enable LRO by default when available on vtnet interfaces The prior change to not enable LRO by default has confused several people. The configurations where LRO is problematic is not the typical use case for VirtIO, and due to other issues, this often requires checksum offloading to be disabled anyways. PR: 185864 MFC after: 2 weeks
2014-11-09 15:04:12 -05:00
if (virtio_with_feature(dev, VIRTIO_NET_F_GUEST_CSUM)) {
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
ifp->if_capabilities |= IFCAP_RXCSUM;
#ifdef notyet
/* BMV: Rx checksums not distinguished between IPv4 and IPv6. */
ifp->if_capabilities |= IFCAP_RXCSUM_IPV6;
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (vtnet_tunable_int(sc, "fixup_needs_csum",
vtnet_fixup_needs_csum) != 0)
sc->vtnet_flags |= VTNET_FLAG_FIXUP_NEEDS_CSUM;
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
/* Support either "hardware" or software LRO. */
ifp->if_capabilities |= IFCAP_LRO;
Enable LRO by default when available on vtnet interfaces The prior change to not enable LRO by default has confused several people. The configurations where LRO is problematic is not the typical use case for VirtIO, and due to other issues, this often requires checksum offloading to be disabled anyways. PR: 185864 MFC after: 2 weeks
2014-11-09 15:04:12 -05:00
}
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
if (ifp->if_capabilities & (IFCAP_HWCSUM | IFCAP_HWCSUM_IPV6)) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* VirtIO does not support VLAN tagging, but we can fake
* it by inserting and removing the 802.1Q header during
* transmit and receive. We are then able to do checksum
* offloading of VLAN frames.
*/
ifp->if_capabilities |=
IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_HWCSUM;
}
if_vtnet: Only set IFCAP_JUMBO_MTU when jumbo MTU is supported Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27912
2021-01-18 23:55:24 -05:00
if (sc->vtnet_max_mtu >= ETHERMTU_JUMBO)
ifp->if_capabilities |= IFCAP_JUMBO_MTU;
ifp->if_capabilities |= IFCAP_VLAN_MTU;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* Capabilities after here are not enabled by default.
*/
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
ifp->if_capenable = ifp->if_capabilities;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (sc->vtnet_flags & VTNET_FLAG_VLAN_FILTER) {
ifp->if_capabilities |= IFCAP_VLAN_HWFILTER;
sc->vtnet_vlan_attach = EVENTHANDLER_REGISTER(vlan_config,
vtnet_register_vlan, sc, EVENTHANDLER_PRI_FIRST);
sc->vtnet_vlan_detach = EVENTHANDLER_REGISTER(vlan_unconfig,
vtnet_unregister_vlan, sc, EVENTHANDLER_PRI_FIRST);
}
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
ether_ifattach(ifp, sc->vtnet_hwaddr);
/* Tell the upper layer(s) we support long frames. */
ifp->if_hdrlen = sizeof(struct ether_vlan_header);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
DEBUGNET_SET(ifp, vtnet);
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
Add pfil(9) hook to vtnet(4). The patch could be simplier, using only the second chunk to vtnet_rxq_eof(), that passes full mbufs to pfil(9). Packet filter would m_free() them in case of returning PFIL_DROPPED. However, we pretend to be a hardware driver, so we first try to pass a memory buffer via PFIL_MEMPTR feature. This is mostly done for debugging purposes, so that one can experiment in bhyve with packet filters utilizing same features as a true driver.
2020-01-10 16:22:03 -05:00
pa.pa_version = PFIL_VERSION;
pa.pa_flags = PFIL_IN;
pa.pa_type = PFIL_TYPE_ETHERNET;
pa.pa_headname = ifp->if_xname;
sc->vtnet_pfil = pfil_head_register(&pa);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
static int
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
vtnet_rx_cluster_size(struct vtnet_softc *sc, int mtu)
{
int framesz;
if (sc->vtnet_flags & VTNET_FLAG_MRG_RXBUFS)
return (MJUMPAGESIZE);
else if (sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG)
return (MCLBYTES);
/*
if_vtnet: Support VIRTIO_NET_F_MTU This feature lets the guest driver know the maximum MTU size supported by the host device. If set, use this to limit the acceptable MTUs, and improve how the receive mbuf cluster size then is selected. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27907
2021-01-18 23:55:24 -05:00
* Try to scale the receive mbuf cluster size from the MTU. We
* could also use the VQ size to influence the selected size,
* but that would only matter for very small queues.
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
*/
if (vtnet_modern(sc)) {
MPASS(sc->vtnet_hdr_size == sizeof(struct virtio_net_hdr_v1));
framesz = sizeof(struct virtio_net_hdr_v1);
} else
framesz = sizeof(struct vtnet_rx_header);
framesz += sizeof(struct ether_vlan_header) + mtu;
if (framesz <= MCLBYTES)
return (MCLBYTES);
else if (framesz <= MJUMPAGESIZE)
return (MJUMPAGESIZE);
else if (framesz <= MJUM9BYTES)
return (MJUM9BYTES);
/* Sane default; avoid 16KB clusters. */
return (MCLBYTES);
}
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_ioctl_mtu(struct vtnet_softc *sc, u_int mtu)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
struct ifnet *ifp;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int clustersz;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp = sc->vtnet_ifp;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
VTNET_CORE_LOCK_ASSERT(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (ifp->if_mtu == mtu)
return (0);
if_vtnet: Support VIRTIO_NET_F_MTU This feature lets the guest driver know the maximum MTU size supported by the host device. If set, use this to limit the acceptable MTUs, and improve how the receive mbuf cluster size then is selected. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27907
2021-01-18 23:55:24 -05:00
else if (mtu < ETHERMIN || mtu > sc->vtnet_max_mtu)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (EINVAL);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
ifp->if_mtu = mtu;
clustersz = vtnet_rx_cluster_size(sc, mtu);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (clustersz != sc->vtnet_rx_clustersz &&
ifp->if_drv_flags & IFF_DRV_RUNNING) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
vtnet_init_locked(sc, 0);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
static int
vtnet_ioctl_ifflags(struct vtnet_softc *sc)
{
struct ifnet *ifp;
int drv_running;
ifp = sc->vtnet_ifp;
drv_running = (ifp->if_drv_flags & IFF_DRV_RUNNING) != 0;
VTNET_CORE_LOCK_ASSERT(sc);
if ((ifp->if_flags & IFF_UP) == 0) {
if (drv_running)
vtnet_stop(sc);
goto out;
}
if (!drv_running) {
vtnet_init_locked(sc, 0);
goto out;
}
if ((ifp->if_flags ^ sc->vtnet_if_flags) &
(IFF_PROMISC | IFF_ALLMULTI)) {
if ((sc->vtnet_flags & VTNET_FLAG_CTRL_RX) == 0)
return (ENOTSUP);
vtnet_rx_filter(sc);
}
out:
sc->vtnet_if_flags = ifp->if_flags;
return (0);
}
static int
vtnet_ioctl_multi(struct vtnet_softc *sc)
{
struct ifnet *ifp;
ifp = sc->vtnet_ifp;
VTNET_CORE_LOCK_ASSERT(sc);
if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX &&
ifp->if_drv_flags & IFF_DRV_RUNNING)
vtnet_rx_filter_mac(sc);
return (0);
}
static int
vtnet_ioctl_ifcap(struct vtnet_softc *sc, struct ifreq *ifr)
{
struct ifnet *ifp;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
int mask, reinit, update;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
ifp = sc->vtnet_ifp;
mask = (ifr->ifr_reqcap & ifp->if_capabilities) ^ ifp->if_capenable;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
reinit = update = 0;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
VTNET_CORE_LOCK_ASSERT(sc);
if (mask & IFCAP_TXCSUM)
ifp->if_capenable ^= IFCAP_TXCSUM;
if (mask & IFCAP_TXCSUM_IPV6)
ifp->if_capenable ^= IFCAP_TXCSUM_IPV6;
if (mask & IFCAP_TSO4)
ifp->if_capenable ^= IFCAP_TSO4;
if (mask & IFCAP_TSO6)
ifp->if_capenable ^= IFCAP_TSO6;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if (mask & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6 | IFCAP_LRO)) {
/*
* These Rx features require the negotiated features to
* be updated. Avoid a full reinit if possible.
*/
if (sc->vtnet_features & VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)
update = 1;
else
reinit = 1;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
/* BMV: Avoid needless renegotiation for just software LRO. */
if ((mask & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6 | IFCAP_LRO)) ==
IFCAP_LRO && vtnet_software_lro(sc))
reinit = update = 0;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
if (mask & IFCAP_RXCSUM)
ifp->if_capenable ^= IFCAP_RXCSUM;
if (mask & IFCAP_RXCSUM_IPV6)
ifp->if_capenable ^= IFCAP_RXCSUM_IPV6;
if (mask & IFCAP_LRO)
ifp->if_capenable ^= IFCAP_LRO;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
/*
* VirtIO does not distinguish between IPv4 and IPv6 checksums
* so treat them as a pair. Guest TSO (LRO) requires receive
* checksums.
*/
if (ifp->if_capenable & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)) {
ifp->if_capenable |= IFCAP_RXCSUM;
#ifdef notyet
ifp->if_capenable |= IFCAP_RXCSUM_IPV6;
#endif
} else
ifp->if_capenable &=
~(IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6 | IFCAP_LRO);
}
if (mask & IFCAP_VLAN_HWFILTER) {
/* These Rx features require renegotiation. */
reinit = 1;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
if (mask & IFCAP_VLAN_HWFILTER)
ifp->if_capenable ^= IFCAP_VLAN_HWFILTER;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
}
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
if (mask & IFCAP_VLAN_HWTSO)
ifp->if_capenable ^= IFCAP_VLAN_HWTSO;
if (mask & IFCAP_VLAN_HWTAGGING)
ifp->if_capenable ^= IFCAP_VLAN_HWTAGGING;
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
if (reinit) {
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
vtnet_init_locked(sc, 0);
} else if (update)
vtnet_update_rx_offloads(sc);
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
}
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int
vtnet_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data)
{
struct vtnet_softc *sc;
struct ifreq *ifr;
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
sc = ifp->if_softc;
ifr = (struct ifreq *) data;
error = 0;
switch (cmd) {
case SIOCSIFMTU:
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
VTNET_CORE_LOCK(sc);
error = vtnet_ioctl_mtu(sc, ifr->ifr_mtu);
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
case SIOCSIFFLAGS:
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
error = vtnet_ioctl_ifflags(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
case SIOCADDMULTI:
case SIOCDELMULTI:
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
error = vtnet_ioctl_multi(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
case SIOCSIFMEDIA:
case SIOCGIFMEDIA:
error = ifmedia_ioctl(ifp, ifr, &sc->vtnet_media, cmd);
break;
case SIOCSIFCAP:
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
if_vtnet: Move ioctl handlers into separate functions Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27914 https://reviews.freebsd.org/D27915
2021-01-18 23:55:25 -05:00
error = vtnet_ioctl_ifcap(sc, ifr);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
VLAN_CAPABILITIES(ifp);
break;
default:
error = ether_ioctl(ifp, cmd, data);
break;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_ASSERT_NOTOWNED(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (error);
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_populate(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct virtqueue *vq;
int nbufs, error;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#ifdef DEV_NETMAP
error = vtnet_netmap_rxq_populate(rxq);
if (error >= 0)
return (error);
#endif /* DEV_NETMAP */
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vq = rxq->vtnrx_vq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
error = ENOSPC;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
for (nbufs = 0; !virtqueue_full(vq); nbufs++) {
error = vtnet_rxq_new_buf(rxq);
if (error)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
}
if (nbufs > 0) {
virtqueue_notify(vq);
/*
* EMSGSIZE signifies the virtqueue did not have enough
* entries available to hold the last mbuf. This is not
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
* an error.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
if (error == EMSGSIZE)
error = 0;
}
return (error);
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_free_mbufs(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct virtqueue *vq;
struct mbuf *m;
int last;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#ifdef DEV_NETMAP
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
struct netmap_kring *kring = netmap_kring_on(NA(rxq->vtnrx_sc->vtnet_ifp),
rxq->vtnrx_id, NR_RX);
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#else /* !DEV_NETMAP */
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
void *kring = NULL;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#endif /* !DEV_NETMAP */
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vq = rxq->vtnrx_vq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
last = 0;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
while ((m = virtqueue_drain(vq, &last)) != NULL) {
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
if (kring == NULL)
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
m_freem(m);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
KASSERT(virtqueue_empty(vq),
("%s: mbufs remaining in rx queue %p", __func__, rxq));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static struct mbuf *
vtnet_rx_alloc_buf(struct vtnet_softc *sc, int nbufs, struct mbuf **m_tailp)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct mbuf *m_head, *m_tail, *m;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int i, size;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
m_head = NULL;
size = sc->vtnet_rx_clustersz;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
KASSERT(nbufs == 1 || sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG,
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
("%s: mbuf %d chain requested without LRO_NOMRG", __func__, nbufs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
for (i = 0; i < nbufs; i++) {
m = m_getjcl(M_NOWAIT, MT_DATA, i == 0 ? M_PKTHDR : 0, size);
if (m == NULL) {
sc->vtnet_stats.mbuf_alloc_failed++;
m_freem(m_head);
return (NULL);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
m->m_len = size;
if (m_head != NULL) {
m_tail->m_next = m;
m_tail = m;
} else
m_head = m_tail = m;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
if (m_tailp != NULL)
*m_tailp = m_tail;
return (m_head);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* Slow path for when LRO without mergeable buffers is negotiated.
*/
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
vtnet_rxq_replace_lro_nomrg_buf(struct vtnet_rxq *rxq, struct mbuf *m0,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int len0)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct mbuf *m, *m_prev, *m_new, *m_tail;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
int len, clustersz, nreplace, error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
clustersz = sc->vtnet_rx_clustersz;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m_prev = NULL;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m_tail = NULL;
nreplace = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m = m0;
len = len0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* Since these mbuf chains are so large, avoid allocating a complete
* replacement when the received frame did not consume the entire
* chain. Unused mbufs are moved to the tail of the replacement mbuf.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
while (len > 0) {
if (m == NULL) {
sc->vtnet_stats.rx_frame_too_large++;
return (EMSGSIZE);
}
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
/*
* Every mbuf should have the expected cluster size since that
* is also used to allocate the replacements.
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
KASSERT(m->m_len == clustersz,
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
("%s: mbuf size %d not expected cluster size %d", __func__,
m->m_len, clustersz));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m->m_len = MIN(m->m_len, len);
len -= m->m_len;
m_prev = m;
m = m->m_next;
nreplace++;
}
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
KASSERT(nreplace > 0 && nreplace <= sc->vtnet_rx_nmbufs,
("%s: invalid replacement mbuf count %d max %d", __func__,
nreplace, sc->vtnet_rx_nmbufs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m_new = vtnet_rx_alloc_buf(sc, nreplace, &m_tail);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (m_new == NULL) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
m_prev->m_len = clustersz;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (ENOBUFS);
}
/*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* Move any unused mbufs from the received mbuf chain onto the
* end of the replacement chain.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
if (m_prev->m_next != NULL) {
m_tail->m_next = m_prev->m_next;
m_prev->m_next = NULL;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_rxq_enqueue_buf(rxq, m_new);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (error) {
/*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* The replacement is suppose to be an copy of the one
* dequeued so this is a very unexpected error.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* Restore the m0 chain to the original state if it was
* modified so we can then discard it.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
if (m_tail->m_next != NULL) {
m_prev->m_next = m_tail->m_next;
m_tail->m_next = NULL;
}
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
m_prev->m_len = clustersz;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_stats.rx_enq_replacement_failed++;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m_freem(m_new);
}
return (error);
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_replace_buf(struct vtnet_rxq *rxq, struct mbuf *m, int len)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
struct mbuf *m_new;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
int error;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG)
return (vtnet_rxq_replace_lro_nomrg_buf(rxq, m, len));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
MPASS(m->m_next == NULL);
if (m->m_len < len)
return (EMSGSIZE);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
m_new = vtnet_rx_alloc_buf(sc, 1, NULL);
if (m_new == NULL)
return (ENOBUFS);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
error = vtnet_rxq_enqueue_buf(rxq, m_new);
if (error) {
sc->vtnet_stats.rx_enq_replacement_failed++;
m_freem(m_new);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
} else
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
m->m_len = len;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_enqueue_buf(struct vtnet_rxq *rxq, struct mbuf *m)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
struct sglist *sg;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int header_inlined, error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
sg = rxq->vtnrx_sg;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
KASSERT(m->m_next == NULL || sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG,
("%s: mbuf chain without LRO_NOMRG", __func__));
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_RXQ_LOCK_ASSERT(rxq);
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
sglist_reset(sg);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
header_inlined = vtnet_modern(sc) ||
(sc->vtnet_flags & VTNET_FLAG_MRG_RXBUFS) != 0; /* TODO: ANY_LAYOUT */
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (header_inlined)
error = sglist_append_mbuf(sg, m);
else {
struct vtnet_rx_header *rxhdr =
mtod(m, struct vtnet_rx_header *);
Revert: virtio: Support non-legacy network device and queue And subsequent fix 576b099a. By adding the mergable header to the vtnet_rx_header structure, the size was increased by 2 bytes, breaking the alignment of this structure as described the in preceding comments. Furthermore, the mergable header does not belong the structure. With the mergable feature, the header is placed in line with the data, so there is no need for a separate segment, and misleading to follow the mergable header with any padding. The V1 header is effectively identical to mergable header, and the driver has long supported the mergable feature. Revert this so the later changes that add V1 support can show how V1 is derived from the existing mergable buffers support, and to facilitate a later MFC. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27855
2021-01-18 23:55:23 -05:00
MPASS(sc->vtnet_hdr_size == sizeof(struct virtio_net_hdr));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
/* Append the header and remaining mbuf data. */
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = sglist_append(sg, &rxhdr->vrh_hdr, sc->vtnet_hdr_size);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (error)
return (error);
error = sglist_append(sg, &rxhdr[1],
m->m_len - sizeof(struct vtnet_rx_header));
if (error)
return (error);
if (m->m_next != NULL)
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = sglist_append_mbuf(sg, m->m_next);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error)
return (error);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
return (virtqueue_enqueue(rxq->vtnrx_vq, m, sg, 0, sg->sg_nseg));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_rxq_new_buf(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
struct mbuf *m;
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m = vtnet_rx_alloc_buf(sc, sc->vtnet_rx_nmbufs, NULL);
if (m == NULL)
return (ENOBUFS);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_rxq_enqueue_buf(rxq, m);
if (error)
m_freem(m);
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
vtnet_rxq_csum_needs_csum(struct vtnet_rxq *rxq, struct mbuf *m, uint16_t etype,
int hoff, struct virtio_net_hdr *hdr)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
/*
* NEEDS_CSUM corresponds to Linux's CHECKSUM_PARTIAL, but FreeBSD does
* not have an analogous CSUM flag. The checksum has been validated,
* but is incomplete (TCP/UDP pseudo header).
*
* The packet is likely from another VM on the same host that itself
* performed checksum offloading so Tx/Rx is basically a memcpy and
* the checksum has little value.
*
* Default to receiving the packet as-is for performance reasons, but
* this can cause issues if the packet is to be forwarded because it
* does not contain a valid checksum. This patch may be helpful:
* https://reviews.freebsd.org/D6611. In the meantime, have the driver
* compute the checksum if requested.
*
* BMV: Need to add an CSUM_PARTIAL flag?
*/
if ((sc->vtnet_flags & VTNET_FLAG_FIXUP_NEEDS_CSUM) == 0) {
error = vtnet_rxq_csum_data_valid(rxq, m, etype, hoff, hdr);
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* Compute the checksum in the driver so the packet will contain a
* valid checksum. The checksum is at csum_offset from csum_start.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
switch (etype) {
#if defined(INET) || defined(INET6)
case ETHERTYPE_IP:
case ETHERTYPE_IPV6: {
int csum_off, csum_end;
uint16_t csum;
csum_off = hdr->csum_start + hdr->csum_offset;
csum_end = csum_off + sizeof(uint16_t);
/* Assume checksum will be in the first mbuf. */
if (m->m_len < csum_end || m->m_pkthdr.len < csum_end)
return (1);
/*
* Like in_delayed_cksum()/in6_delayed_cksum(), compute the
* checksum and write it at the specified offset. We could
* try to verify the packet: csum_start should probably
* correspond to the start of the TCP/UDP header.
*
* BMV: Need to properly handle UDP with zero checksum. Is
* the IPv4 header checksum implicitly validated?
*/
csum = in_cksum_skip(m, m->m_pkthdr.len, hdr->csum_start);
*(uint16_t *)(mtodo(m, csum_off)) = csum;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m->m_pkthdr.csum_flags |= CSUM_DATA_VALID | CSUM_PSEUDO_HDR;
m->m_pkthdr.csum_data = 0xFFFF;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
break;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
}
#endif
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
default:
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
sc->vtnet_stats.rx_csum_bad_ethtype++;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (1);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
vtnet_rxq_csum_data_valid(struct vtnet_rxq *rxq, struct mbuf *m,
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
uint16_t etype, int hoff, struct virtio_net_hdr *hdr __unused)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
struct vtnet_softc *sc;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int protocol;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
switch (etype) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#if defined(INET)
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
case ETHERTYPE_IP:
if (__predict_false(m->m_len < hoff + sizeof(struct ip)))
protocol = IPPROTO_DONE;
else {
struct ip *ip = (struct ip *)(m->m_data + hoff);
protocol = ip->ip_p;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
break;
#endif
#if defined(INET6)
case ETHERTYPE_IPV6:
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (__predict_false(m->m_len < hoff + sizeof(struct ip6_hdr))
|| ip6_lasthdr(m, hoff, IPPROTO_IPV6, &protocol) < 0)
protocol = IPPROTO_DONE;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
break;
#endif
default:
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
protocol = IPPROTO_DONE;
break;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
switch (protocol) {
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
case IPPROTO_TCP:
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
case IPPROTO_UDP:
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m->m_pkthdr.csum_flags |= CSUM_DATA_VALID | CSUM_PSEUDO_HDR;
m->m_pkthdr.csum_data = 0xFFFF;
break;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
default:
/*
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* FreeBSD does not support checksum offloading of this
* protocol. Let the stack re-verify the checksum later
* if the protocol is supported.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
#if 0
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if_printf(sc->vtnet_ifp,
"%s: checksum offload of unsupported protocol "
"etype=%#x protocol=%d csum_start=%d csum_offset=%d\n",
__func__, etype, protocol, hdr->csum_start,
hdr->csum_offset);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#endif
break;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
return (0);
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_csum(struct vtnet_rxq *rxq, struct mbuf *m,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct virtio_net_hdr *hdr)
{
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
const struct ether_header *eh;
int hoff;
uint16_t etype;
eh = mtod(m, const struct ether_header *);
etype = ntohs(eh->ether_type);
if (etype == ETHERTYPE_VLAN) {
/* TODO BMV: Handle QinQ. */
const struct ether_vlan_header *evh =
mtod(m, const struct ether_vlan_header *);
etype = ntohs(evh->evl_proto);
hoff = sizeof(struct ether_vlan_header);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
} else
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
hoff = sizeof(struct ether_header);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM)
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
return (vtnet_rxq_csum_needs_csum(rxq, m, etype, hoff, hdr));
else /* VIRTIO_NET_HDR_F_DATA_VALID */
return (vtnet_rxq_csum_data_valid(rxq, m, etype, hoff, hdr));
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_rxq_discard_merged_bufs(struct vtnet_rxq *rxq, int nbufs)
{
struct mbuf *m;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
while (--nbufs > 0) {
m = virtqueue_dequeue(rxq->vtnrx_vq, NULL);
if (m == NULL)
break;
vtnet_rxq_discard_buf(rxq, m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_rxq_discard_buf(struct vtnet_rxq *rxq, struct mbuf *m)
{
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* Requeue the discarded mbuf. This should always be successful
* since it was just dequeued.
*/
error = vtnet_rxq_enqueue_buf(rxq, m);
KASSERT(error == 0,
("%s: cannot requeue discarded mbuf %d", __func__, error));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_merged_eof(struct vtnet_rxq *rxq, struct mbuf *m_head, int nbufs)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct virtqueue *vq;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct mbuf *m_tail;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
vq = rxq->vtnrx_vq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m_tail = m_head;
while (--nbufs > 0) {
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct mbuf *m;
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
uint32_t len;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m = virtqueue_dequeue(vq, &len);
if (m == NULL) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_stats.vrxs_ierrors++;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (vtnet_rxq_new_buf(rxq) != 0) {
rxq->vtnrx_stats.vrxs_iqdrops++;
vtnet_rxq_discard_buf(rxq, m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (nbufs > 1)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_discard_merged_bufs(rxq, nbufs);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
}
if (m->m_len < len)
len = m->m_len;
m->m_len = len;
m->m_flags &= ~M_PKTHDR;
m_head->m_pkthdr.len += len;
m_tail->m_next = m;
m_tail = m;
}
return (0);
fail:
sc->vtnet_stats.rx_mergeable_failed++;
m_freem(m_head);
return (1);
}
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#if defined(INET) || defined(INET6)
static int
vtnet_lro_rx(struct vtnet_rxq *rxq, struct mbuf *m)
{
struct lro_ctrl *lro;
lro = &rxq->vtnrx_lro;
if (lro->lro_mbuf_max != 0) {
tcp_lro_queue_mbuf(lro, m);
return (0);
}
return (tcp_lro_rx(lro, m, 0));
}
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_rxq_input(struct vtnet_rxq *rxq, struct mbuf *m,
struct virtio_net_hdr *hdr)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct ifnet *ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (ifp->if_capenable & IFCAP_VLAN_HWTAGGING) {
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct ether_header *eh = mtod(m, struct ether_header *);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (eh->ether_type == htons(ETHERTYPE_VLAN)) {
vtnet_vlan_tag_remove(m);
/*
* With the 802.1Q header removed, update the
* checksum starting location accordingly.
*/
if (hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM)
hdr->csum_start -= ETHER_VLAN_ENCAP_LEN;
}
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m->m_pkthdr.flowid = rxq->vtnrx_id;
Start process of removing the use of the deprecated "M_FLOWID" flag from the FreeBSD network code. The flag is still kept around in the "sys/mbuf.h" header file, but does no longer have any users. Instead the "m_pkthdr.rsstype" field in the mbuf structure is now used to decide the meaning of the "m_pkthdr.flowid" field. To modify the "m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX" macros as defined in the "sys/mbuf.h" header file. This patch introduces new behaviour in the transmit direction. Previously network drivers checked if "M_FLOWID" was set in "m_flags" before using the "m_pkthdr.flowid" field. This check has now now been replaced by checking if "M_HASHTYPE_GET(m)" is different from "M_HASHTYPE_NONE". In the future more hashtypes will be added, for example hashtypes for hardware dedicated flows. "M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is valid and has no particular type. This change removes the need for an "if" statement in TCP transmit code checking for the presence of a valid flowid value. The "if" statement mentioned above is now a direct variable assignment which is then later checked by the respective network drivers like before. Additional notes: - The SCTP code changes will be committed as a separate patch. - Removal of the "M_FLOWID" flag will also be done separately. - The FreeBSD version has been bumped. MFC after: 1 month Sponsored by: Mellanox Technologies
2014-12-01 06:45:24 -05:00
M_HASHTYPE_SET(m, M_HASHTYPE_OPAQUE);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (hdr->flags &
(VIRTIO_NET_HDR_F_NEEDS_CSUM | VIRTIO_NET_HDR_F_DATA_VALID)) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (vtnet_rxq_csum(rxq, m, hdr) == 0)
rxq->vtnrx_stats.vrxs_csum++;
else
rxq->vtnrx_stats.vrxs_csum_failed++;
}
if_vtnet: Set lro_nsegs for host LRO packets Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27933
2021-01-18 23:55:25 -05:00
if (hdr->gso_size != 0) {
switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) {
case VIRTIO_NET_HDR_GSO_TCPV4:
case VIRTIO_NET_HDR_GSO_TCPV6:
m->m_pkthdr.lro_nsegs =
howmany(m->m_pkthdr.len, hdr->gso_size);
if_vtnet: Add counter for received host LRO Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27928
2021-01-18 23:55:26 -05:00
rxq->vtnrx_stats.vrxs_host_lro++;
if_vtnet: Set lro_nsegs for host LRO packets Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27933
2021-01-18 23:55:25 -05:00
break;
}
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_stats.vrxs_ipackets++;
rxq->vtnrx_stats.vrxs_ibytes += m->m_pkthdr.len;
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#if defined(INET) || defined(INET6)
if (vtnet_software_lro(sc) && ifp->if_capenable & IFCAP_LRO) {
if (vtnet_lro_rx(rxq, m) == 0)
return;
}
#endif
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
(*ifp->if_input)(ifp, m);
}
static int
vtnet_rxq_eof(struct vtnet_rxq *rxq)
{
struct virtio_net_hdr lhdr, *hdr;
struct vtnet_softc *sc;
struct ifnet *ifp;
struct virtqueue *vq;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
int deq, count;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
vq = rxq->vtnrx_vq;
ifp = sc->vtnet_ifp;
deq = 0;
count = sc->vtnet_rx_process_limit;
VTNET_RXQ_LOCK_ASSERT(rxq);
while (count-- > 0) {
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct mbuf *m;
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
uint32_t len, nbufs, adjsz;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
m = virtqueue_dequeue(vq, &len);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (m == NULL)
break;
deq++;
if (len < sc->vtnet_hdr_size + ETHER_HDR_LEN) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq->vtnrx_stats.vrxs_ierrors++;
vtnet_rxq_discard_buf(rxq, m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
continue;
}
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (sc->vtnet_flags & VTNET_FLAG_MRG_RXBUFS) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct virtio_net_hdr_mrg_rxbuf *mhdr =
mtod(m, struct virtio_net_hdr_mrg_rxbuf *);
nbufs = vtnet_htog16(sc, mhdr->num_buffers);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
adjsz = sizeof(struct virtio_net_hdr_mrg_rxbuf);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
} else if (vtnet_modern(sc)) {
nbufs = 1; /* num_buffers is always 1 */
adjsz = sizeof(struct virtio_net_hdr_v1);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
} else {
nbufs = 1;
adjsz = sizeof(struct vtnet_rx_header);
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
/*
* Account for our gap between the header and start of
* data to keep the segments separated.
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
len += VTNET_RX_HEADER_PAD;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (vtnet_rxq_replace_buf(rxq, m, len) != 0) {
rxq->vtnrx_stats.vrxs_iqdrops++;
vtnet_rxq_discard_buf(rxq, m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (nbufs > 1)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_discard_merged_bufs(rxq, nbufs);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
continue;
}
m->m_pkthdr.len = len;
m->m_pkthdr.rcvif = ifp;
m->m_pkthdr.csum_flags = 0;
if (nbufs > 1) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/* Dequeue the rest of chain. */
if (vtnet_rxq_merged_eof(rxq, m, nbufs) != 0)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
continue;
}
virtio: Add KMSAN hooks for network and block devices This ensures that host-written data is marked as initialized. Sponsored by: The FreeBSD Foundation
2021-08-11 16:23:25 -04:00
kmsan_mark_mbuf(m, KMSAN_STATE_INITED);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
* Save an endian swapped version of the header prior to it
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
* being stripped. The header is always at the start of the
* mbuf data. num_buffers was already saved (and not needed)
* so use the standard header.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
hdr = mtod(m, struct virtio_net_hdr *);
lhdr.flags = hdr->flags;
lhdr.gso_type = hdr->gso_type;
lhdr.hdr_len = vtnet_htog16(sc, hdr->hdr_len);
lhdr.gso_size = vtnet_htog16(sc, hdr->gso_size);
lhdr.csum_start = vtnet_htog16(sc, hdr->csum_start);
lhdr.csum_offset = vtnet_htog16(sc, hdr->csum_offset);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m_adj(m, adjsz);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (PFIL_HOOKED_IN(sc->vtnet_pfil)) {
pfil_return_t pfil;
Add pfil(9) hook to vtnet(4). The patch could be simplier, using only the second chunk to vtnet_rxq_eof(), that passes full mbufs to pfil(9). Packet filter would m_free() them in case of returning PFIL_DROPPED. However, we pretend to be a hardware driver, so we first try to pass a memory buffer via PFIL_MEMPTR feature. This is mostly done for debugging purposes, so that one can experiment in bhyve with packet filters utilizing same features as a true driver.
2020-01-10 16:22:03 -05:00
pfil = pfil_run_hooks(sc->vtnet_pfil, &m, ifp, PFIL_IN,
NULL);
switch (pfil) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
case PFIL_REALLOCED:
m = pfil_mem2mbuf(m->m_data);
break;
Add pfil(9) hook to vtnet(4). The patch could be simplier, using only the second chunk to vtnet_rxq_eof(), that passes full mbufs to pfil(9). Packet filter would m_free() them in case of returning PFIL_DROPPED. However, we pretend to be a hardware driver, so we first try to pass a memory buffer via PFIL_MEMPTR feature. This is mostly done for debugging purposes, so that one can experiment in bhyve with packet filters utilizing same features as a true driver.
2020-01-10 16:22:03 -05:00
case PFIL_DROPPED:
case PFIL_CONSUMED:
continue;
default:
KASSERT(pfil == PFIL_PASS,
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
("Filter returned %d!", pfil));
Add pfil(9) hook to vtnet(4). The patch could be simplier, using only the second chunk to vtnet_rxq_eof(), that passes full mbufs to pfil(9). Packet filter would m_free() them in case of returning PFIL_DROPPED. However, we pretend to be a hardware driver, so we first try to pass a memory buffer via PFIL_MEMPTR feature. This is mostly done for debugging purposes, so that one can experiment in bhyve with packet filters utilizing same features as a true driver.
2020-01-10 16:22:03 -05:00
}
}
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
vtnet_rxq_input(rxq, m, &lhdr);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
if (deq > 0) {
#if defined(INET) || defined(INET6)
Fix vtnet TCP lro panic Differential Revision: https://reviews.freebsd.org/D29900 Reviewed by: hps, kp
2021-04-19 11:58:41 -04:00
if (vtnet_software_lro(sc))
tcp_lro_flush_all(&rxq->vtnrx_lro);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
#endif
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
virtqueue_notify(vq);
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (count > 0 ? 0 : EAGAIN);
}
static void
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
vtnet_rx_vq_process(struct vtnet_rxq *rxq, int tries)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct vtnet_softc *sc;
struct ifnet *ifp;
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
u_int more;
netmap: vtnet: honor NM_IRQ_RESCHED The netmap_rx_irq() function may return NM_IRQ_RESCHED to inform the driver that more work is pending, and that netmap expects netmap_rx_irq() to be called again as soon as possible. This change implements this behaviour in the vtnet driver. MFC after: 1 week
2020-06-03 01:09:33 -04:00
#ifdef DEV_NETMAP
int nmirq;
#endif /* DEV_NETMAP */
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = rxq->vtnrx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (__predict_false(rxq->vtnrx_id >= sc->vtnet_act_vq_pairs)) {
/*
* Ignore this interrupt. Either this is a spurious interrupt
* or multiqueue without per-VQ MSIX so every queue needs to
* be polled (a brain dead configuration we could try harder
* to avoid).
*/
vtnet_rxq_disable_intr(rxq);
return;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
netmap: vtnet: call netmap_rx_irq() under VQ lock The netmap_rx_irq() function normally wakes up user-space threads waiting for more packets. In this case, it is not necessary to call it under the driver queue lock. However, if the interface is attached to a VALE switch, netmap_rx_irq() ends up calling rxsync on the interface (see netmap_bwrap_intr_notify()). Although concurrent rxsyncs are serialized through the kring lock (see nm_kr_tryget()), the lock acquire operation is not blocking. As a result, it may happen that netmap_rx_irq() is called on an RX ring while another instance is running, causing the second call to fail, and received packets stall in the receive VQ. We fix this issue by calling netmap_irx_irq() under the VQ lock. MFC after: 1 week
2020-06-03 01:27:29 -04:00
VTNET_RXQ_LOCK(rxq);
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#ifdef DEV_NETMAP
netmap: vtnet: call netmap_rx_irq() under VQ lock The netmap_rx_irq() function normally wakes up user-space threads waiting for more packets. In this case, it is not necessary to call it under the driver queue lock. However, if the interface is attached to a VALE switch, netmap_rx_irq() ends up calling rxsync on the interface (see netmap_bwrap_intr_notify()). Although concurrent rxsyncs are serialized through the kring lock (see nm_kr_tryget()), the lock acquire operation is not blocking. As a result, it may happen that netmap_rx_irq() is called on an RX ring while another instance is running, causing the second call to fail, and received packets stall in the receive VQ. We fix this issue by calling netmap_irx_irq() under the VQ lock. MFC after: 1 week
2020-06-03 01:27:29 -04:00
/*
* We call netmap_rx_irq() under lock to prevent concurrent calls.
* This is not necessary to serialize the access to the RX vq, but
* rather to avoid races that may happen if this interface is
* attached to a VALE switch, which would cause received packets
* to stall in the RX queue (nm_kr_tryget() could find the kring
* busy when called from netmap_bwrap_intr_notify()).
*/
netmap: vtnet: honor NM_IRQ_RESCHED The netmap_rx_irq() function may return NM_IRQ_RESCHED to inform the driver that more work is pending, and that netmap expects netmap_rx_irq() to be called again as soon as possible. This change implements this behaviour in the vtnet driver. MFC after: 1 week
2020-06-03 01:09:33 -04:00
nmirq = netmap_rx_irq(ifp, rxq->vtnrx_id, &more);
if (nmirq != NM_IRQ_PASS) {
netmap: vtnet: call netmap_rx_irq() under VQ lock The netmap_rx_irq() function normally wakes up user-space threads waiting for more packets. In this case, it is not necessary to call it under the driver queue lock. However, if the interface is attached to a VALE switch, netmap_rx_irq() ends up calling rxsync on the interface (see netmap_bwrap_intr_notify()). Although concurrent rxsyncs are serialized through the kring lock (see nm_kr_tryget()), the lock acquire operation is not blocking. As a result, it may happen that netmap_rx_irq() is called on an RX ring while another instance is running, causing the second call to fail, and received packets stall in the receive VQ. We fix this issue by calling netmap_irx_irq() under the VQ lock. MFC after: 1 week
2020-06-03 01:27:29 -04:00
VTNET_RXQ_UNLOCK(rxq);
netmap: vtnet: honor NM_IRQ_RESCHED The netmap_rx_irq() function may return NM_IRQ_RESCHED to inform the driver that more work is pending, and that netmap expects netmap_rx_irq() to be called again as soon as possible. This change implements this behaviour in the vtnet driver. MFC after: 1 week
2020-06-03 01:09:33 -04:00
if (nmirq == NM_IRQ_RESCHED) {
taskqueue_enqueue(rxq->vtnrx_tq, &rxq->vtnrx_intrtask);
}
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
return;
netmap: vtnet: honor NM_IRQ_RESCHED The netmap_rx_irq() function may return NM_IRQ_RESCHED to inform the driver that more work is pending, and that netmap expects netmap_rx_irq() to be called again as soon as possible. This change implements this behaviour in the vtnet driver. MFC after: 1 week
2020-06-03 01:09:33 -04:00
}
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#endif /* DEV_NETMAP */
Avoid queue unlock followed by relock when the enable interrupt race is lost This already happens infrequently, and the hold time is still bounded since we defer to a taskqueue after a few tries.
2014-01-25 14:57:30 -05:00
again:
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) {
VTNET_RXQ_UNLOCK(rxq);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
more = vtnet_rxq_eof(rxq);
if (more || vtnet_rxq_enable_intr(rxq) != 0) {
if (!more)
vtnet_rxq_disable_intr(rxq);
/*
* This is an occasional condition or race (when !more),
* so retry a few times before scheduling the taskqueue.
*/
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
if (tries-- > 0)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
goto again;
Avoid queue unlock followed by relock when the enable interrupt race is lost This already happens infrequently, and the hold time is still bounded since we defer to a taskqueue after a few tries.
2014-01-25 14:57:30 -05:00
rxq->vtnrx_stats.vrxs_rescheduled++;
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
VTNET_RXQ_UNLOCK(rxq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
taskqueue_enqueue(rxq->vtnrx_tq, &rxq->vtnrx_intrtask);
} else
VTNET_RXQ_UNLOCK(rxq);
}
static void
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
vtnet_rx_vq_intr(void *xrxq)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
struct vtnet_rxq *rxq;
rxq = xrxq;
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
vtnet_rx_vq_process(rxq, VTNET_INTR_DISABLE_RETRIES);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
static void
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_rxq_tq_intr(void *xrxq, int pending __unused)
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
{
struct vtnet_rxq *rxq;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: let vtnet_rx_vq_intr() and vtnet_rxq_tq_intr() share code Since the two functions are similar, introduce a common function (vtnet_rx_vq_process()) to share common code. This also improves locking, by ensuring vrxs_rescheduled is accessed under the RXQ lock, and taskqueue_enqueue() is not called under the lock (therefore avoiding a spurious duplicate lock warning). Reported by: jrtc27 MFC after: 2 weeks
2020-06-15 15:46:34 -04:00
rxq = xrxq;
vtnet_rx_vq_process(rxq, 0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static int
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
vtnet_txq_intr_threshold(struct vtnet_txq *txq)
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
{
struct vtnet_softc *sc;
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
int threshold;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
sc = txq->vtntx_sc;
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
/*
* The Tx interrupt is disabled until the queue free count falls
* below our threshold. Completed frames are drained from the Tx
* virtqueue before transmitting new frames and in the watchdog
* callout, so the frequency of Tx interrupts is greatly reduced,
* at the cost of not freeing mbufs as quickly as they otherwise
* would be.
*/
threshold = virtqueue_size(txq->vtntx_vq) / 4;
/*
* Without indirect descriptors, leave enough room for the most
* segments we handle.
*/
if ((sc->vtnet_flags & VTNET_FLAG_INDIRECT) == 0 &&
threshold < sc->vtnet_tx_nsegs)
threshold = sc->vtnet_tx_nsegs;
return (threshold);
}
static int
vtnet_txq_below_threshold(struct vtnet_txq *txq)
{
struct virtqueue *vq;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
vq = txq->vtntx_vq;
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
return (virtqueue_nfree(vq) <= txq->vtntx_intr_threshold);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
}
static int
vtnet_txq_notify(struct vtnet_txq *txq)
{
struct virtqueue *vq;
vq = txq->vtntx_vq;
txq->vtntx_watchdog = VTNET_TX_TIMEOUT;
virtqueue_notify(vq);
if (vtnet_txq_enable_intr(txq) == 0)
return (0);
/*
* Drain frames that were completed since last checked. If this
* causes the queue to go above the threshold, the caller should
* continue transmitting.
*/
if (vtnet_txq_eof(txq) != 0 && vtnet_txq_below_threshold(txq) == 0) {
virtqueue_disable_intr(vq);
return (1);
}
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_txq_free_mbufs(struct vtnet_txq *txq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct virtqueue *vq;
struct vtnet_tx_header *txhdr;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int last;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#ifdef DEV_NETMAP
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
struct netmap_kring *kring = netmap_kring_on(NA(txq->vtntx_sc->vtnet_ifp),
txq->vtntx_id, NR_TX);
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#else /* !DEV_NETMAP */
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
void *kring = NULL;
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#endif /* !DEV_NETMAP */
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vq = txq->vtntx_vq;
last = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
while ((txhdr = virtqueue_drain(vq, &last)) != NULL) {
netmap: introduce netmap_kring_on() This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week
2020-06-11 16:35:28 -04:00
if (kring == NULL) {
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
m_freem(txhdr->vth_mbuf);
uma_zfree(vtnet_tx_header_zone, txhdr);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
KASSERT(virtqueue_empty(vq),
("%s: mbufs remaining in tx queue %p", __func__, txq));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
* BMV: This can go away once we finally have offsets in the mbuf header.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
static int
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
vtnet_txq_offload_ctx(struct vtnet_txq *txq, struct mbuf *m, int *etype,
int *proto, int *start)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct ether_vlan_header *evh;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int offset;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
evh = mtod(m, struct ether_vlan_header *);
if (evh->evl_encap_proto == htons(ETHERTYPE_VLAN)) {
/* BMV: We should handle nested VLAN tags too. */
*etype = ntohs(evh->evl_proto);
offset = sizeof(struct ether_vlan_header);
} else {
*etype = ntohs(evh->evl_encap_proto);
offset = sizeof(struct ether_header);
}
switch (*etype) {
#if defined(INET)
case ETHERTYPE_IP: {
struct ip *ip, iphdr;
if (__predict_false(m->m_len < offset + sizeof(struct ip))) {
m_copydata(m, offset, sizeof(struct ip),
(caddr_t) &iphdr);
ip = &iphdr;
} else
ip = (struct ip *)(m->m_data + offset);
*proto = ip->ip_p;
*start = offset + (ip->ip_hl << 2);
break;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#endif
#if defined(INET6)
case ETHERTYPE_IPV6:
*proto = -1;
*start = ip6_lasthdr(m, offset, IPPROTO_IPV6, proto);
/* Assert the network stack sent us a valid packet. */
KASSERT(*start > offset,
("%s: mbuf %p start %d offset %d proto %d", __func__, m,
*start, offset, *proto));
break;
#endif
default:
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
sc->vtnet_stats.tx_csum_unknown_ethtype++;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (EINVAL);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet: fix TSO for TCP/IPv6 The decision whether a TCP packet is sent over IPv4 or IPv6 was based on ethertype, which works correctly. In D27926 the criteria was changed to checking if the CSUM_IP_TSO flag is set in the csum-flags and then considering it to be TCP/IPv4. However, the TCP stack sets the flag to CSUM_TSO for IPv4 and IPv6, where CSUM_TSO is defined as CSUM_IP_TSO|CSUM_IP6_TSO. Therefore TCP/IPv6 packets gets mis-classified as TCP/IPv4, which breaks TSO for TCP/IPv6. This patch bases the check again on the ethertype. This fix will be MFC instantly as discussed with re(gjb). MFC after: instantly PR: 254366 Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D29331
2021-03-18 16:25:47 -04:00
vtnet_txq_offload_tso(struct vtnet_txq *txq, struct mbuf *m, int eth_type,
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int offset, struct virtio_net_hdr *hdr)
{
static struct timeval lastecn;
static int curecn;
struct vtnet_softc *sc;
struct tcphdr *tcp, tcphdr;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
if (__predict_false(m->m_len < offset + sizeof(struct tcphdr))) {
m_copydata(m, offset, sizeof(struct tcphdr), (caddr_t) &tcphdr);
tcp = &tcphdr;
} else
tcp = (struct tcphdr *)(m->m_data + offset);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
hdr->hdr_len = vtnet_gtoh16(sc, offset + (tcp->th_off << 2));
hdr->gso_size = vtnet_gtoh16(sc, m->m_pkthdr.tso_segsz);
vtnet: fix TSO for TCP/IPv6 The decision whether a TCP packet is sent over IPv4 or IPv6 was based on ethertype, which works correctly. In D27926 the criteria was changed to checking if the CSUM_IP_TSO flag is set in the csum-flags and then considering it to be TCP/IPv4. However, the TCP stack sets the flag to CSUM_TSO for IPv4 and IPv6, where CSUM_TSO is defined as CSUM_IP_TSO|CSUM_IP6_TSO. Therefore TCP/IPv6 packets gets mis-classified as TCP/IPv4, which breaks TSO for TCP/IPv6. This patch bases the check again on the ethertype. This fix will be MFC instantly as discussed with re(gjb). MFC after: instantly PR: 254366 Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D29331
2021-03-18 16:25:47 -04:00
hdr->gso_type = eth_type == ETHERTYPE_IP ? VIRTIO_NET_HDR_GSO_TCPV4 :
VIRTIO_NET_HDR_GSO_TCPV6;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
if (__predict_false(tcp->th_flags & TH_CWR)) {
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
* Drop if VIRTIO_NET_F_HOST_ECN was not negotiated. In
* FreeBSD, ECN support is not on a per-interface basis,
* but globally via the net.inet.tcp.ecn.enable sysctl
* knob. The default is off.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if ((sc->vtnet_flags & VTNET_FLAG_TSO_ECN) == 0) {
if (ppsratecheck(&lastecn, &curecn, 1))
if_printf(sc->vtnet_ifp,
"TSO with ECN not negotiated with host\n");
return (ENOTSUP);
}
hdr->gso_type |= VIRTIO_NET_HDR_GSO_ECN;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq->vtntx_stats.vtxs_tso++;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static struct mbuf *
vtnet_txq_offload(struct vtnet_txq *txq, struct mbuf *m,
struct virtio_net_hdr *hdr)
{
struct vtnet_softc *sc;
int flags, etype, csum_start, proto, error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
flags = m->m_pkthdr.csum_flags;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_txq_offload_ctx(txq, m, &etype, &proto, &csum_start);
if (error)
goto drop;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
if (flags & (VTNET_CSUM_OFFLOAD | VTNET_CSUM_OFFLOAD_IPV6)) {
/* Sanity check the parsed mbuf matches the offload flags. */
if (__predict_false((flags & VTNET_CSUM_OFFLOAD &&
etype != ETHERTYPE_IP) || (flags & VTNET_CSUM_OFFLOAD_IPV6
&& etype != ETHERTYPE_IPV6))) {
sc->vtnet_stats.tx_csum_proto_mismatch++;
goto drop;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
hdr->flags |= VIRTIO_NET_HDR_F_NEEDS_CSUM;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
hdr->csum_start = vtnet_gtoh16(sc, csum_start);
hdr->csum_offset = vtnet_gtoh16(sc, m->m_pkthdr.csum_data);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq->vtntx_stats.vtxs_csum++;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
if (flags & (CSUM_IP_TSO | CSUM_IP6_TSO)) {
/*
* Sanity check the parsed mbuf IP protocol is TCP, and
* VirtIO TSO reqires the checksum offloading above.
*/
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (__predict_false(proto != IPPROTO_TCP)) {
sc->vtnet_stats.tx_tso_not_tcp++;
goto drop;
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
} else if (__predict_false((hdr->flags &
VIRTIO_NET_HDR_F_NEEDS_CSUM) == 0)) {
sc->vtnet_stats.tx_tso_without_csum++;
goto drop;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
vtnet: fix TSO for TCP/IPv6 The decision whether a TCP packet is sent over IPv4 or IPv6 was based on ethertype, which works correctly. In D27926 the criteria was changed to checking if the CSUM_IP_TSO flag is set in the csum-flags and then considering it to be TCP/IPv4. However, the TCP stack sets the flag to CSUM_TSO for IPv4 and IPv6, where CSUM_TSO is defined as CSUM_IP_TSO|CSUM_IP6_TSO. Therefore TCP/IPv6 packets gets mis-classified as TCP/IPv4, which breaks TSO for TCP/IPv6. This patch bases the check again on the ethertype. This fix will be MFC instantly as discussed with re(gjb). MFC after: instantly PR: 254366 Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D29331
2021-03-18 16:25:47 -04:00
error = vtnet_txq_offload_tso(txq, m, etype, csum_start, hdr);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (error)
goto drop;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
return (m);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
drop:
m_freem(m);
return (NULL);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_txq_enqueue_buf(struct vtnet_txq *txq, struct mbuf **m_head,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct vtnet_tx_header *txhdr)
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_softc *sc;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct virtqueue *vq;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
struct sglist *sg;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct mbuf *m;
Use m_defrag() instead of m_collapse() to compact a long mbuf chain This should be an infrequent occurrence, so remove the per-queue counters in favor of just global counters in the softc.
2014-02-02 00:20:46 -05:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
vq = txq->vtntx_vq;
sg = txq->vtntx_sg;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m = *m_head;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
sglist_reset(sg);
error = sglist_append(sg, &txhdr->vth_uhdr, sc->vtnet_hdr_size);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error != 0 || sg->sg_nseg != 1) {
KASSERT(0, ("%s: cannot add header to sglist error %d nseg %d",
__func__, error, sg->sg_nseg));
goto fail;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
error = sglist_append_mbuf(sg, m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (error) {
Use m_defrag() instead of m_collapse() to compact a long mbuf chain This should be an infrequent occurrence, so remove the per-queue counters in favor of just global counters in the softc.
2014-02-02 00:20:46 -05:00
m = m_defrag(m, M_NOWAIT);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (m == NULL)
goto fail;
*m_head = m;
Use m_defrag() instead of m_collapse() to compact a long mbuf chain This should be an infrequent occurrence, so remove the per-queue counters in favor of just global counters in the softc.
2014-02-02 00:20:46 -05:00
sc->vtnet_stats.tx_defragged++;
error = sglist_append_mbuf(sg, m);
if (error)
goto fail;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
txhdr->vth_mbuf = m;
Do not place the sglist used for Rx/Tx on the stack The sglist segment array has grown to a bit over 512 bytes (on 64-bit system) which is more than ideally should be put on the stack. Instead allocate an appropriately sized sglist and hang it off each Rx/Tx queue structure. Bump the maximum number of Tx segments to 64 to make it unlikely we'll have defragment an mbuf chain. Our previous count was rounded up to this value since it is the next power of two, so effective memory usage should not change. Also only allocate the maximum number of Tx segments if TSO was negotiated.
2014-02-02 00:15:36 -05:00
error = virtqueue_enqueue(vq, txhdr, sg, sg->sg_nseg, 0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
fail:
Use m_defrag() instead of m_collapse() to compact a long mbuf chain This should be an infrequent occurrence, so remove the per-queue counters in favor of just global counters in the softc.
2014-02-02 00:20:46 -05:00
sc->vtnet_stats.tx_defrag_failed++;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m_freem(*m_head);
*m_head = NULL;
return (ENOBUFS);
}
static int
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
vtnet_txq_encap(struct vtnet_txq *txq, struct mbuf **m_head, int flags)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct vtnet_tx_header *txhdr;
struct virtio_net_hdr *hdr;
struct mbuf *m;
int error;
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
m = *m_head;
Merge vtnet changes from projects/virtio Minor changes to the network driver. A multiqueue driver that is a significant rewrite will be in merged shortly. Contains projects/virtio commits: r246058: vtnet: Move an mbuf ASSERT to the calling function r246059: vtnet: Tweak ASSERT message MFC after: 1 month
2013-07-04 13:55:58 -04:00
M_ASSERTPKTHDR(m);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
txhdr = uma_zalloc(vtnet_tx_header_zone, flags | M_ZERO);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
if (txhdr == NULL) {
m_freem(m);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*m_head = NULL;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return (ENOMEM);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
* Always use the non-mergeable header, regardless if mergable headers
* were negotiated, because for transmit num_buffers is always zero.
* The vtnet_hdr_size is used to enqueue the right header size segment.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
hdr = &txhdr->vth_uhdr.hdr;
if (m->m_flags & M_VLANTAG) {
m = ether_vlanencap(m, m->m_pkthdr.ether_vtag);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
if ((*m_head = m) == NULL) {
error = ENOBUFS;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
m->m_flags &= ~M_VLANTAG;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (m->m_pkthdr.csum_flags & VTNET_CSUM_ALL_OFFLOAD) {
m = vtnet_txq_offload(txq, m, hdr);
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
if ((*m_head = m) == NULL) {
error = ENOBUFS;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto fail;
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_txq_enqueue_buf(txq, m_head, txhdr);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
fail:
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error)
uma_zfree(vtnet_tx_header_zone, txhdr);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (error);
}
#ifdef VTNET_LEGACY_TX
static void
vtnet_start_locked(struct vtnet_txq *txq, struct ifnet *ifp)
{
struct vtnet_softc *sc;
struct virtqueue *vq;
struct mbuf *m0;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
int tries, enq;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
vq = txq->vtntx_vq;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
tries = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_LOCK_ASSERT(txq);
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0 ||
sc->vtnet_link_active == 0)
return;
Complete any pending Tx frames before attempting the next transmit Also complete pending frames in the watchdog function when the EVENT_IDX feature was negotiated just in case the completion interrupt was postponed.
2013-09-02 22:28:31 -04:00
vtnet_txq_eof(txq);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
again:
enq = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
while (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) {
if (virtqueue_full(vq))
break;
IFQ_DRV_DEQUEUE(&ifp->if_snd, m0);
if (m0 == NULL)
break;
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
if (vtnet_txq_encap(txq, &m0, M_NOWAIT) != 0) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (m0 != NULL)
IFQ_DRV_PREPEND(&ifp->if_snd, m0);
break;
}
enq++;
ETHER_BPF_MTAP(ifp, m0);
}
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
if (enq > 0 && vtnet_txq_notify(txq) != 0) {
if (tries++ < VTNET_NOTIFY_RETRIES)
goto again;
txq->vtntx_stats.vtxs_rescheduled++;
taskqueue_enqueue(txq->vtntx_tq, &txq->vtntx_intrtask);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
}
static void
vtnet_start(struct ifnet *ifp)
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
sc = ifp->if_softc;
txq = &sc->vtnet_txqs[0];
VTNET_TXQ_LOCK(txq);
vtnet_start_locked(txq, ifp);
VTNET_TXQ_UNLOCK(txq);
}
#else /* !VTNET_LEGACY_TX */
static int
vtnet_txq_mq_start_locked(struct vtnet_txq *txq, struct mbuf *m)
{
struct vtnet_softc *sc;
struct virtqueue *vq;
struct buf_ring *br;
struct ifnet *ifp;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
int enq, tries, error;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc = txq->vtntx_sc;
vq = txq->vtntx_vq;
br = txq->vtntx_br;
ifp = sc->vtnet_ifp;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
tries = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = 0;
VTNET_TXQ_LOCK_ASSERT(txq);
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0 ||
sc->vtnet_link_active == 0) {
if (m != NULL)
error = drbr_enqueue(ifp, br, m);
return (error);
}
if (m != NULL) {
error = drbr_enqueue(ifp, br, m);
if (error)
return (error);
}
Complete any pending Tx frames before attempting the next transmit Also complete pending frames in the watchdog function when the EVENT_IDX feature was negotiated just in case the completion interrupt was postponed.
2013-09-02 22:28:31 -04:00
vtnet_txq_eof(txq);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
again:
enq = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
while ((m = drbr_peek(ifp, br)) != NULL) {
Check for a full virtqueue in the multiqueue transmit path With most hosts, we'll negotiate indirect descriptors, so all we need is one available descriptor to transmit a frame.
2014-01-25 14:58:53 -05:00
if (virtqueue_full(vq)) {
drbr_putback(ifp, br, m);
break;
}
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
if (vtnet_txq_encap(txq, &m, M_NOWAIT) != 0) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (m != NULL)
drbr_putback(ifp, br, m);
else
drbr_advance(ifp, br);
break;
}
drbr_advance(ifp, br);
enq++;
ETHER_BPF_MTAP(ifp, m);
}
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
if (enq > 0 && vtnet_txq_notify(txq) != 0) {
if (tries++ < VTNET_NOTIFY_RETRIES)
goto again;
txq->vtntx_stats.vtxs_rescheduled++;
taskqueue_enqueue(txq->vtntx_tq, &txq->vtntx_intrtask);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
make sure ifp->if_transmit returns 0 if a buffer is enqueued. A similar fix should be applied to vmxnet, ixgbe, igb, i40e. (some of them previously reported by Michael Tuexen) Drivers using if_transmit are correct, and so are most of the other drivers that reassing if_transmit. Among other things, this bug causes panics when using netmap emulation on top of generic drivers. Approved by: bryanv MFC after: 3 days
2014-06-04 12:57:05 -04:00
return (0);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static int
vtnet_txq_mq_start(struct ifnet *ifp, struct mbuf *m)
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
int i, npairs, error;
sc = ifp->if_softc;
npairs = sc->vtnet_act_vq_pairs;
Start process of removing the use of the deprecated "M_FLOWID" flag from the FreeBSD network code. The flag is still kept around in the "sys/mbuf.h" header file, but does no longer have any users. Instead the "m_pkthdr.rsstype" field in the mbuf structure is now used to decide the meaning of the "m_pkthdr.flowid" field. To modify the "m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX" macros as defined in the "sys/mbuf.h" header file. This patch introduces new behaviour in the transmit direction. Previously network drivers checked if "M_FLOWID" was set in "m_flags" before using the "m_pkthdr.flowid" field. This check has now now been replaced by checking if "M_HASHTYPE_GET(m)" is different from "M_HASHTYPE_NONE". In the future more hashtypes will be added, for example hashtypes for hardware dedicated flows. "M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is valid and has no particular type. This change removes the need for an "if" statement in TCP transmit code checking for the presence of a valid flowid value. The "if" statement mentioned above is now a direct variable assignment which is then later checked by the respective network drivers like before. Additional notes: - The SCTP code changes will be committed as a separate patch. - Removal of the "M_FLOWID" flag will also be done separately. - The FreeBSD version has been bumped. MFC after: 1 month Sponsored by: Mellanox Technologies
2014-12-01 06:45:24 -05:00
if (M_HASHTYPE_GET(m) != M_HASHTYPE_NONE)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
i = m->m_pkthdr.flowid % npairs;
else
i = curcpu % npairs;
txq = &sc->vtnet_txqs[i];
if (VTNET_TXQ_TRYLOCK(txq) != 0) {
error = vtnet_txq_mq_start_locked(txq, m);
VTNET_TXQ_UNLOCK(txq);
} else {
error = drbr_enqueue(ifp, txq->vtntx_br, m);
taskqueue_enqueue(txq->vtntx_tq, &txq->vtntx_defrtask);
}
return (error);
}
static void
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_txq_tq_deferred(void *xtxq, int pending __unused)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
txq = xtxq;
sc = txq->vtntx_sc;
VTNET_TXQ_LOCK(txq);
if (!drbr_empty(sc->vtnet_ifp, txq->vtntx_br))
vtnet_txq_mq_start_locked(txq, NULL);
VTNET_TXQ_UNLOCK(txq);
}
#endif /* VTNET_LEGACY_TX */
Move duplicated transmit start code into a single function
2014-01-25 14:55:42 -05:00
static void
vtnet_txq_start(struct vtnet_txq *txq)
{
struct vtnet_softc *sc;
struct ifnet *ifp;
sc = txq->vtntx_sc;
ifp = sc->vtnet_ifp;
#ifdef VTNET_LEGACY_TX
if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd))
vtnet_start_locked(txq, ifp);
#else
if (!drbr_empty(ifp, txq->vtntx_br))
vtnet_txq_mq_start_locked(txq, NULL);
#endif
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_txq_tq_intr(void *xtxq, int pending __unused)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
struct ifnet *ifp;
txq = xtxq;
sc = txq->vtntx_sc;
ifp = sc->vtnet_ifp;
VTNET_TXQ_LOCK(txq);
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) {
VTNET_TXQ_UNLOCK(txq);
return;
}
vtnet_txq_eof(txq);
Move duplicated transmit start code into a single function
2014-01-25 14:55:42 -05:00
vtnet_txq_start(txq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_UNLOCK(txq);
}
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_txq_eof(struct vtnet_txq *txq)
{
struct virtqueue *vq;
struct vtnet_tx_header *txhdr;
struct mbuf *m;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
int deq;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vq = txq->vtntx_vq;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
deq = 0;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_LOCK_ASSERT(txq);
while ((txhdr = virtqueue_dequeue(vq, NULL)) != NULL) {
m = txhdr->vth_mbuf;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
deq++;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq->vtntx_stats.vtxs_opackets++;
txq->vtntx_stats.vtxs_obytes += m->m_pkthdr.len;
if (m->m_flags & M_MCAST)
txq->vtntx_stats.vtxs_omcasts++;
m_freem(m);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
uma_zfree(vtnet_tx_header_zone, txhdr);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
if (virtqueue_empty(vq))
txq->vtntx_watchdog = 0;
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
return (deq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_tx_vq_intr(void *xtxq)
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
struct ifnet *ifp;
txq = xtxq;
sc = txq->vtntx_sc;
ifp = sc->vtnet_ifp;
if (__predict_false(txq->vtntx_id >= sc->vtnet_act_vq_pairs)) {
/*
* Ignore this interrupt. Either this is a spurious interrupt
* or multiqueue without per-VQ MSIX so every queue needs to
* be polled (a brain dead configuration we could try harder
* to avoid).
*/
vtnet_txq_disable_intr(txq);
return;
}
vtnet: fix netmap support netmap(4) support for vtnet(4) was incomplete and had multiple bugs. This commit fixes those bugs to bring netmap on vtnet in a functional state. Changelist: - handle errors returned by virtqueue_enqueue() properly (they were previously ignored) - make sure netmap XOR rest of the kernel access each virtqueue. - compute the number of netmap slots for TX and RX separately, according to whether indirect descriptors are used or not for a given virtqueue. - make sure sglist are freed according to their type (mbufs or netmap buffers) - add support for mulitiqueue and netmap host (aka sw) rings. - intercept VQ interrupts directly instead of intercepting them in txq_eof and rxq_eof. This simplifies the code and makes it easier to make sure taskqueues are not running for a VQ while it is in netmap mode. - implement vntet_netmap_config() to cope with changes in the number of queues. Reviewed by: bryanv Approved by: gnn (mentor) MFC after: 3 days Sponsored by: Sunny Valley Networks Differential Revision: https://reviews.freebsd.org/D17916
2018-11-14 10:39:48 -05:00
#ifdef DEV_NETMAP
if (netmap_tx_irq(ifp, txq->vtntx_id) != NM_IRQ_PASS)
return;
#endif /* DEV_NETMAP */
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_LOCK(txq);
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) {
VTNET_TXQ_UNLOCK(txq);
return;
}
vtnet_txq_eof(txq);
Move duplicated transmit start code into a single function
2014-01-25 14:55:42 -05:00
vtnet_txq_start(txq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
VTNET_TXQ_UNLOCK(txq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_tx_start_all(struct vtnet_softc *sc)
{
struct vtnet_txq *txq;
int i;
VTNET_CORE_LOCK_ASSERT(sc);
for (i = 0; i < sc->vtnet_act_vq_pairs; i++) {
txq = &sc->vtnet_txqs[i];
VTNET_TXQ_LOCK(txq);
Move duplicated transmit start code into a single function
2014-01-25 14:55:42 -05:00
vtnet_txq_start(txq);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_UNLOCK(txq);
}
}
#ifndef VTNET_LEGACY_TX
static void
vtnet_qflush(struct ifnet *ifp)
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
struct mbuf *m;
int i;
sc = ifp->if_softc;
for (i = 0; i < sc->vtnet_act_vq_pairs; i++) {
txq = &sc->vtnet_txqs[i];
VTNET_TXQ_LOCK(txq);
while ((m = buf_ring_dequeue_sc(txq->vtntx_br)) != NULL)
m_freem(m);
VTNET_TXQ_UNLOCK(txq);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_qflush(ifp);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#endif
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_watchdog(struct vtnet_txq *txq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
struct ifnet *ifp;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
ifp = txq->vtntx_sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_TXQ_LOCK(txq);
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
if (txq->vtntx_watchdog == 1) {
/*
* Only drain completed frames if the watchdog is about to
* expire. If any frames were drained, there may be enough
* free descriptors now available to transmit queued frames.
* In that case, the timer will immediately be decremented
* below, but the timeout is generous enough that should not
* be a problem.
*/
if (vtnet_txq_eof(txq) != 0)
vtnet_txq_start(txq);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (txq->vtntx_watchdog == 0 || --txq->vtntx_watchdog) {
VTNET_TXQ_UNLOCK(txq);
return (0);
}
VTNET_TXQ_UNLOCK(txq);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
if_printf(ifp, "watchdog timeout on queue %d\n", txq->vtntx_id);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
vtnet_accum_stats(struct vtnet_softc *sc, struct vtnet_rxq_stats *rxacc,
struct vtnet_txq_stats *txacc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
bzero(rxacc, sizeof(struct vtnet_rxq_stats));
bzero(txacc, sizeof(struct vtnet_txq_stats));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
for (int i = 0; i < sc->vtnet_max_vq_pairs; i++) {
struct vtnet_rxq_stats *rxst;
struct vtnet_txq_stats *txst;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
rxst = &sc->vtnet_rxqs[i].vtnrx_stats;
rxacc->vrxs_ipackets += rxst->vrxs_ipackets;
rxacc->vrxs_ibytes += rxst->vrxs_ibytes;
rxacc->vrxs_iqdrops += rxst->vrxs_iqdrops;
rxacc->vrxs_csum += rxst->vrxs_csum;
rxacc->vrxs_csum_failed += rxst->vrxs_csum_failed;
rxacc->vrxs_rescheduled += rxst->vrxs_rescheduled;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
txst = &sc->vtnet_txqs[i].vtntx_stats;
txacc->vtxs_opackets += txst->vtxs_opackets;
txacc->vtxs_obytes += txst->vtxs_obytes;
txacc->vtxs_csum += txst->vtxs_csum;
txacc->vtxs_tso += txst->vtxs_tso;
txacc->vtxs_rescheduled += txst->vtxs_rescheduled;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
static uint64_t
vtnet_get_counter(if_t ifp, ift_counter cnt)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
struct vtnet_softc *sc;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_rxq_stats rxaccum;
struct vtnet_txq_stats txaccum;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
sc = if_getsoftc(ifp);
vtnet_accum_stats(sc, &rxaccum, &txaccum);
switch (cnt) {
case IFCOUNTER_IPACKETS:
return (rxaccum.vrxs_ipackets);
case IFCOUNTER_IQDROPS:
return (rxaccum.vrxs_iqdrops);
case IFCOUNTER_IERRORS:
return (rxaccum.vrxs_ierrors);
case IFCOUNTER_OPACKETS:
return (txaccum.vtxs_opackets);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#ifndef VTNET_LEGACY_TX
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
case IFCOUNTER_OBYTES:
return (txaccum.vtxs_obytes);
case IFCOUNTER_OMCASTS:
return (txaccum.vtxs_omcasts);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
#endif
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
default:
return (if_get_counter_default(ifp, cnt));
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_tick(void *xsc)
{
struct vtnet_softc *sc;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct ifnet *ifp;
int i, timedout;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
sc = xsc;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp = sc->vtnet_ifp;
timedout = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_ASSERT(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
for (i = 0; i < sc->vtnet_act_vq_pairs; i++)
timedout |= vtnet_watchdog(&sc->vtnet_txqs[i]);
if (timedout != 0) {
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
vtnet_init_locked(sc, 0);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
} else
callout_schedule(&sc->vtnet_tick_ch, hz);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_start_taskqueues(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
device_t dev;
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
int i, error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
dev = sc->vtnet_dev;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* Errors here are very difficult to recover from - we cannot
* easily fail because, if this is during boot, we will hang
* when freeing any successfully started taskqueues because
* the scheduler isn't up yet.
*
* Most drivers just ignore the return value - it only fails
* with ENOMEM so an error is not likely.
*/
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_req_vq_pairs; i++) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq = &sc->vtnet_rxqs[i];
error = taskqueue_start_threads(&rxq->vtnrx_tq, 1, PI_NET,
"%s rxq %d", device_get_nameunit(dev), rxq->vtnrx_id);
if (error) {
device_printf(dev, "failed to start rx taskq %d\n",
rxq->vtnrx_id);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
txq = &sc->vtnet_txqs[i];
error = taskqueue_start_threads(&txq->vtntx_tq, 1, PI_NET,
"%s txq %d", device_get_nameunit(dev), txq->vtntx_id);
if (error) {
device_printf(dev, "failed to start tx taskq %d\n",
txq->vtntx_id);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_free_taskqueues(struct vtnet_softc *sc)
{
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
int i;
for (i = 0; i < sc->vtnet_max_vq_pairs; i++) {
rxq = &sc->vtnet_rxqs[i];
if (rxq->vtnrx_tq != NULL) {
taskqueue_free(rxq->vtnrx_tq);
vtnet: fix typo in vtnet_free_taskqueues Because of a typo, the code was mistakenly resetting the vtnrx_vq pointer rather than vtntx_tq. Reviewed by: bryanv MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D19015
2019-01-29 09:31:41 -05:00
rxq->vtnrx_tq = NULL;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
txq = &sc->vtnet_txqs[i];
if (txq->vtntx_tq != NULL) {
taskqueue_free(txq->vtntx_tq);
txq->vtntx_tq = NULL;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_drain_taskqueues(struct vtnet_softc *sc)
{
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
int i;
for (i = 0; i < sc->vtnet_max_vq_pairs; i++) {
rxq = &sc->vtnet_rxqs[i];
if (rxq->vtnrx_tq != NULL)
taskqueue_drain(rxq->vtnrx_tq, &rxq->vtnrx_intrtask);
txq = &sc->vtnet_txqs[i];
if (txq->vtntx_tq != NULL) {
taskqueue_drain(txq->vtntx_tq, &txq->vtntx_intrtask);
#ifndef VTNET_LEGACY_TX
taskqueue_drain(txq->vtntx_tq, &txq->vtntx_defrtask);
#endif
}
}
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_drain_rxtx_queues(struct vtnet_softc *sc)
{
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
int i;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_max_vq_pairs; i++) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
rxq = &sc->vtnet_rxqs[i];
vtnet_rxq_free_mbufs(rxq);
txq = &sc->vtnet_txqs[i];
vtnet_txq_free_mbufs(txq);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_stop_rendezvous(struct vtnet_softc *sc)
{
struct vtnet_rxq *rxq;
struct vtnet_txq *txq;
int i;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Schedule Rx task if pending items when enabling interrupt Prior to V1, the driver would enable interrupts and then notify the host that DRIVER_OK. Since for V1, DRIVER_OK needs to be set before notifying the virtqueues, there may be items in the queues waiting to be processed by the time interrupts are enabled. This fixes a bug where the Rx queue would appear stuck, only being usable after an interface down/up cycle. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27922
2021-01-18 23:55:25 -05:00
VTNET_CORE_LOCK_ASSERT(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/*
* Lock and unlock the per-queue mutex so we known the stop
* state is visible. Doing only the active queues should be
* sufficient, but it does not cost much extra to do all the
if_vtnet: Schedule Rx task if pending items when enabling interrupt Prior to V1, the driver would enable interrupts and then notify the host that DRIVER_OK. Since for V1, DRIVER_OK needs to be set before notifying the virtqueues, there may be items in the queues waiting to be processed by the time interrupts are enabled. This fixes a bug where the Rx queue would appear stuck, only being usable after an interface down/up cycle. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27922
2021-01-18 23:55:25 -05:00
* queues.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
for (i = 0; i < sc->vtnet_max_vq_pairs; i++) {
rxq = &sc->vtnet_rxqs[i];
VTNET_RXQ_LOCK(rxq);
VTNET_RXQ_UNLOCK(rxq);
txq = &sc->vtnet_txqs[i];
VTNET_TXQ_LOCK(txq);
VTNET_TXQ_UNLOCK(txq);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_stop(struct vtnet_softc *sc)
{
device_t dev;
struct ifnet *ifp;
dev = sc->vtnet_dev;
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_ASSERT(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
sc->vtnet_link_active = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
callout_stop(&sc->vtnet_tick_ch);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/* Only advisory. */
vtnet_disable_interrupts(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
netmap: vtnet: enable/disable krings on any interface reinit See 3d65fd97e85ab807f3b for a detailed explanation. PR: 252453 MFC after: 1 week
2021-01-10 09:09:00 -05:00
#ifdef DEV_NETMAP
/* Stop any pending txsync/rxsync and disable them. */
netmap_disable_all_rings(ifp);
#endif /* DEV_NETMAP */
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
* Stop the host adapter. This resets it to the pre-initialized
* state. It will not generate any interrupts until after it is
* reinitialized.
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
*/
virtio_stop(dev);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_stop_rendezvous(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_drain_rxtx_queues(sc);
if_vtnet: Schedule Rx task if pending items when enabling interrupt Prior to V1, the driver would enable interrupts and then notify the host that DRIVER_OK. Since for V1, DRIVER_OK needs to be set before notifying the virtqueues, there may be items in the queues waiting to be processed by the time interrupts are enabled. This fixes a bug where the Rx queue would appear stuck, only being usable after an interface down/up cycle. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27922
2021-01-18 23:55:25 -05:00
sc->vtnet_act_vq_pairs = 1;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_virtio_reinit(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
device_t dev;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct ifnet *ifp;
uint64_t features;
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
dev = sc->vtnet_dev;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
ifp = sc->vtnet_ifp;
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
features = sc->vtnet_negotiated_features;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/*
* Re-negotiate with the host, removing any disabled receive
* features. Transmit features are disabled only on our side
* via if_capenable and if_hwassist.
*/
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if ((ifp->if_capenable & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)) == 0)
features &= ~(VIRTIO_NET_F_GUEST_CSUM | VTNET_LRO_FEATURES);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if ((ifp->if_capenable & IFCAP_LRO) == 0)
features &= ~VTNET_LRO_FEATURES;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if ((ifp->if_capenable & IFCAP_VLAN_HWFILTER) == 0)
features &= ~VIRTIO_NET_F_CTRL_VLAN;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = virtio_reinit(dev, features);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
if (error) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
device_printf(dev, "virtio reinit error %d\n", error);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
return (error);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
sc->vtnet_features = features;
virtio_reinit_complete(dev);
return (0);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_init_rx_filters(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct ifnet *ifp;
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX) {
vtnet_rx_filter(sc);
vtnet_rx_filter_mac(sc);
}
if (ifp->if_capenable & IFCAP_VLAN_HWFILTER)
vtnet_rx_filter_vlan(sc);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_init_rx_queues(struct vtnet_softc *sc)
{
device_t dev;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
struct ifnet *ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct vtnet_rxq *rxq;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
int i, clustersz, error;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
dev = sc->vtnet_dev;
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
ifp = sc->vtnet_ifp;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
clustersz = vtnet_rx_cluster_size(sc, ifp->if_mtu);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sc->vtnet_rx_clustersz = clustersz;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
if (sc->vtnet_flags & VTNET_FLAG_LRO_NOMRG) {
sc->vtnet_rx_nmbufs = howmany(sizeof(struct vtnet_rx_header) +
VTNET_MAX_RX_SIZE, clustersz);
KASSERT(sc->vtnet_rx_nmbufs < sc->vtnet_rx_nsegs,
("%s: too many rx mbufs %d for %d segments", __func__,
sc->vtnet_rx_nmbufs, sc->vtnet_rx_nsegs));
} else
sc->vtnet_rx_nmbufs = 1;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
for (i = 0; i < sc->vtnet_act_vq_pairs; i++) {
rxq = &sc->vtnet_rxqs[i];
/* Hold the lock to satisfy asserts. */
VTNET_RXQ_LOCK(rxq);
error = vtnet_rxq_populate(rxq);
VTNET_RXQ_UNLOCK(rxq);
if (error) {
if_vtnet: Rx path cleanup - Fix the NEEDS_CSUM and DATA_VALID checksum flags. The NEEDS_CSUM checksum is incomplete (partial) so offer a fallback for the driver to calculate the checksum. Simplify DATA_VALID because we know the host has validated the checksum. - Default 4K mbuf clusters for mergeable buffers. May need to scale this down to 2K clusters in certain configurations such many queue pairs, big queues (like 4096 in GCP), and low memory. - Use the MTU when calculated the receive mbuf cluster size when not doing TSO/LRO. This will need more adjustment once the MTU feature is supported. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27906
2021-01-18 23:55:24 -05:00
device_printf(dev, "cannot populate Rx queue %d\n", i);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (error);
}
}
return (0);
}
static int
vtnet_init_tx_queues(struct vtnet_softc *sc)
{
struct vtnet_txq *txq;
int i;
for (i = 0; i < sc->vtnet_act_vq_pairs; i++) {
txq = &sc->vtnet_txqs[i];
txq->vtntx_watchdog = 0;
if_vtnet: Move the Tx interrupt threshold into the Txq structure Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27911
2021-01-18 23:55:24 -05:00
txq->vtntx_intr_threshold = vtnet_txq_intr_threshold(txq);
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
#ifdef DEV_NETMAP
netmap_reset(NA(sc->vtnet_ifp), NR_TX, i, 0);
#endif /* DEV_NETMAP */
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
return (0);
}
static int
vtnet_init_rxtx_queues(struct vtnet_softc *sc)
{
int error;
error = vtnet_init_rx_queues(sc);
if (error)
return (error);
error = vtnet_init_tx_queues(sc);
if (error)
return (error);
return (0);
}
static void
vtnet_set_active_vq_pairs(struct vtnet_softc *sc)
{
device_t dev;
int npairs;
dev = sc->vtnet_dev;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if ((sc->vtnet_flags & VTNET_FLAG_MQ) == 0) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_act_vq_pairs = 1;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
return;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
npairs = sc->vtnet_req_vq_pairs;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (vtnet_ctrl_mq_cmd(sc, npairs) != 0) {
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
device_printf(dev, "cannot set active queue pairs to %d, "
"falling back to 1 queue pair\n", npairs);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
npairs = 1;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_act_vq_pairs = npairs;
}
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
static void
vtnet_update_rx_offloads(struct vtnet_softc *sc)
{
struct ifnet *ifp;
uint64_t features;
int error;
ifp = sc->vtnet_ifp;
features = sc->vtnet_features;
VTNET_CORE_LOCK_ASSERT(sc);
if (ifp->if_capabilities & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)) {
if (ifp->if_capenable & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6))
features |= VIRTIO_NET_F_GUEST_CSUM;
else
features &= ~VIRTIO_NET_F_GUEST_CSUM;
}
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
if (ifp->if_capabilities & IFCAP_LRO && !vtnet_software_lro(sc)) {
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
if (ifp->if_capenable & IFCAP_LRO)
features |= VTNET_LRO_FEATURES;
else
features &= ~VTNET_LRO_FEATURES;
}
error = vtnet_ctrl_guest_offloads(sc,
features & (VIRTIO_NET_F_GUEST_CSUM | VIRTIO_NET_F_GUEST_TSO4 |
VIRTIO_NET_F_GUEST_TSO6 | VIRTIO_NET_F_GUEST_ECN |
VIRTIO_NET_F_GUEST_UFO));
if (error) {
device_printf(sc->vtnet_dev,
"%s: cannot update Rx features\n", __func__);
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
vtnet_init_locked(sc, 0);
}
} else
sc->vtnet_features = features;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_reinit(struct vtnet_softc *sc)
{
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
device_t dev;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct ifnet *ifp;
int error;
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
dev = sc->vtnet_dev;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
ifp = sc->vtnet_ifp;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
bcopy(IF_LLADDR(ifp), sc->vtnet_hwaddr, ETHER_ADDR_LEN);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
error = vtnet_virtio_reinit(sc);
if (error)
return (error);
vtnet_set_macaddr(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_set_active_vq_pairs(sc);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
if (sc->vtnet_flags & VTNET_FLAG_CTRL_VQ)
vtnet_init_rx_filters(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
ifp->if_hwassist = 0;
if (ifp->if_capenable & IFCAP_TXCSUM)
ifp->if_hwassist |= VTNET_CSUM_OFFLOAD;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (ifp->if_capenable & IFCAP_TXCSUM_IPV6)
ifp->if_hwassist |= VTNET_CSUM_OFFLOAD_IPV6;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (ifp->if_capenable & IFCAP_TSO4)
Use the appropriate IPv4 or IPv6 TSO HW assist flag MFC after: 2 weeks
2014-12-31 21:03:09 -05:00
ifp->if_hwassist |= CSUM_IP_TSO;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (ifp->if_capenable & IFCAP_TSO6)
Use the appropriate IPv4 or IPv6 TSO HW assist flag MFC after: 2 weeks
2014-12-31 21:03:09 -05:00
ifp->if_hwassist |= CSUM_IP6_TSO;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error = vtnet_init_rxtx_queues(sc);
if (error)
return (error);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (0);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
vtnet_init_locked(struct vtnet_softc *sc, int init_mode)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
device_t dev;
struct ifnet *ifp;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
dev = sc->vtnet_dev;
ifp = sc->vtnet_ifp;
VTNET_CORE_LOCK_ASSERT(sc);
if (ifp->if_drv_flags & IFF_DRV_RUNNING)
return;
vtnet_stop(sc);
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
#ifdef DEV_NETMAP
/* Once stopped we can update the netmap flags, if necessary. */
switch (init_mode) {
case VTNET_INIT_NETMAP_ENTER:
nm_set_native_flags(NA(ifp));
break;
case VTNET_INIT_NETMAP_EXIT:
nm_clear_native_flags(NA(ifp));
break;
}
#endif /* DEV_NETMAP */
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
if (vtnet_reinit(sc) != 0) {
vtnet_stop(sc);
return;
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
ifp->if_drv_flags |= IFF_DRV_RUNNING;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
vtnet_update_link_status(sc);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
vtnet_enable_interrupts(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
callout_reset(&sc->vtnet_tick_ch, hz, vtnet_tick, sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
netmap: vtnet: enable/disable krings on any interface reinit See 3d65fd97e85ab807f3b for a detailed explanation. PR: 252453 MFC after: 1 week
2021-01-10 09:09:00 -05:00
#ifdef DEV_NETMAP
/* Re-enable txsync/rxsync. */
netmap_enable_all_rings(ifp);
#endif /* DEV_NETMAP */
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_init(void *xsc)
{
struct vtnet_softc *sc;
sc = xsc;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
netmap: vtnet: fix races in vtnet_netmap_reg() The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week
2020-06-14 16:47:31 -04:00
vtnet_init_locked(sc, 0);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
}
static void
vtnet_free_ctrl_vq(struct vtnet_softc *sc)
{
/*
* The control virtqueue is only polled and therefore it should
* already be empty.
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
KASSERT(virtqueue_empty(sc->vtnet_ctrl_vq),
("%s: ctrl vq %p not empty", __func__, sc->vtnet_ctrl_vq));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_exec_ctrl_cmd(struct vtnet_softc *sc, void *cookie,
struct sglist *sg, int readable, int writable)
{
struct virtqueue *vq;
vq = sc->vtnet_ctrl_vq;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(sc->vtnet_flags & VTNET_FLAG_CTRL_VQ);
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
VTNET_CORE_LOCK_ASSERT(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (!virtqueue_empty(vq))
return;
/*
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
* Poll for the response, but the command is likely completed before
* returning from the notify.
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
*/
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (virtqueue_enqueue(vq, cookie, sg, readable, writable) == 0) {
virtqueue_notify(vq);
virtqueue_poll(vq, NULL);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static int
vtnet_ctrl_mac_cmd(struct vtnet_softc *sc, uint8_t *hwaddr)
{
struct sglist_seg segs[3];
struct sglist sg;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct {
struct virtio_net_ctrl_hdr hdr __aligned(2);
uint8_t pad1;
uint8_t addr[ETHER_ADDR_LEN] __aligned(8);
uint8_t pad2;
uint8_t ack;
} s;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int error;
error = 0;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(sc->vtnet_flags & VTNET_FLAG_CTRL_MAC);
s.hdr.class = VIRTIO_NET_CTRL_MAC;
s.hdr.cmd = VIRTIO_NET_CTRL_MAC_ADDR_SET;
bcopy(hwaddr, &s.addr[0], ETHER_ADDR_LEN);
s.ack = VIRTIO_NET_ERR;
sglist_init(&sg, nitems(segs), segs);
error |= sglist_append(&sg, &s.hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &s.addr[0], ETHER_ADDR_LEN);
error |= sglist_append(&sg, &s.ack, sizeof(uint8_t));
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &s.ack, &sg, sg.sg_nseg - 1, 1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
return (s.ack == VIRTIO_NET_OK ? 0 : EIO);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
if_vtnet: Add support for CTRL_GUEST_OFFLOADS feature This allows the Rx checksum and LRO to be modified without a full reinit of the device. Remove IFCAP_RXCSUM_IPV6 from the interface capabilities since in VirtIO Rx checksums are just enabled or disabled for all protocols. Properly update IFCAP_LRO if LRO is becomes disabled when Rx checksums are disabled. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27916
2021-01-18 23:55:25 -05:00
static int
vtnet_ctrl_guest_offloads(struct vtnet_softc *sc, uint64_t offloads)
{
struct sglist_seg segs[3];
struct sglist sg;
struct {
struct virtio_net_ctrl_hdr hdr __aligned(2);
uint8_t pad1;
uint64_t offloads __aligned(8);
uint8_t pad2;
uint8_t ack;
} s;
int error;
error = 0;
MPASS(sc->vtnet_features & VIRTIO_NET_F_CTRL_GUEST_OFFLOADS);
s.hdr.class = VIRTIO_NET_CTRL_GUEST_OFFLOADS;
s.hdr.cmd = VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET;
s.offloads = vtnet_gtoh64(sc, offloads);
s.ack = VIRTIO_NET_ERR;
sglist_init(&sg, nitems(segs), segs);
error |= sglist_append(&sg, &s.hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &s.offloads, sizeof(uint64_t));
error |= sglist_append(&sg, &s.ack, sizeof(uint8_t));
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &s.ack, &sg, sg.sg_nseg - 1, 1);
return (s.ack == VIRTIO_NET_OK ? 0 : EIO);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_ctrl_mq_cmd(struct vtnet_softc *sc, uint16_t npairs)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct sglist_seg segs[3];
struct sglist sg;
struct {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct virtio_net_ctrl_hdr hdr __aligned(2);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint8_t pad1;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct virtio_net_ctrl_mq mq __aligned(2);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint8_t pad2;
uint8_t ack;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
} s;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = 0;
MPASS(sc->vtnet_flags & VTNET_FLAG_MQ);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.hdr.class = VIRTIO_NET_CTRL_MQ;
s.hdr.cmd = VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
s.mq.virtqueue_pairs = vtnet_gtoh16(sc, npairs);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.ack = VIRTIO_NET_ERR;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sglist_init(&sg, nitems(segs), segs);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error |= sglist_append(&sg, &s.hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &s.mq, sizeof(struct virtio_net_ctrl_mq));
error |= sglist_append(&sg, &s.ack, sizeof(uint8_t));
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &s.ack, &sg, sg.sg_nseg - 1, 1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (s.ack == VIRTIO_NET_OK ? 0 : EIO);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_ctrl_rx_cmd(struct vtnet_softc *sc, uint8_t cmd, bool on)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct sglist_seg segs[3];
struct sglist sg;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct virtio_net_ctrl_hdr hdr __aligned(2);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint8_t pad1;
uint8_t onoff;
uint8_t pad2;
uint8_t ack;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
} s;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
int error;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = 0;
MPASS(sc->vtnet_flags & VTNET_FLAG_CTRL_RX);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.hdr.class = VIRTIO_NET_CTRL_RX;
s.hdr.cmd = cmd;
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
s.onoff = on;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.ack = VIRTIO_NET_ERR;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sglist_init(&sg, nitems(segs), segs);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error |= sglist_append(&sg, &s.hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &s.onoff, sizeof(uint8_t));
error |= sglist_append(&sg, &s.ack, sizeof(uint8_t));
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &s.ack, &sg, sg.sg_nseg - 1, 1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (s.ack == VIRTIO_NET_OK ? 0 : EIO);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_set_promisc(struct vtnet_softc *sc, bool on)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
return (vtnet_ctrl_rx_cmd(sc, VIRTIO_NET_CTRL_RX_PROMISC, on));
}
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_set_allmulti(struct vtnet_softc *sc, bool on)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
return (vtnet_ctrl_rx_cmd(sc, VIRTIO_NET_CTRL_RX_ALLMULTI, on));
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_rx_filter(struct vtnet_softc *sc)
{
device_t dev;
struct ifnet *ifp;
dev = sc->vtnet_dev;
ifp = sc->vtnet_ifp;
VTNET_CORE_LOCK_ASSERT(sc);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (vtnet_set_promisc(sc, ifp->if_flags & IFF_PROMISC) != 0) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
device_printf(dev, "cannot %s promiscuous mode\n",
ifp->if_flags & IFF_PROMISC ? "enable" : "disable");
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (vtnet_set_allmulti(sc, ifp->if_flags & IFF_ALLMULTI) != 0) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
device_printf(dev, "cannot %s all-multicast mode\n",
ifp->if_flags & IFF_ALLMULTI ? "enable" : "disable");
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
static u_int
vtnet_copy_ifaddr(void *arg, struct sockaddr_dl *sdl, u_int ucnt)
{
struct vtnet_softc *sc = arg;
if (memcmp(LLADDR(sdl), sc->vtnet_hwaddr, ETHER_ADDR_LEN) == 0)
return (0);
if (ucnt < VTNET_MAX_MAC_ENTRIES)
bcopy(LLADDR(sdl),
&sc->vtnet_mac_filter->vmf_unicast.macs[ucnt],
ETHER_ADDR_LEN);
return (1);
}
static u_int
vtnet_copy_maddr(void *arg, struct sockaddr_dl *sdl, u_int mcnt)
{
struct vtnet_mac_filter *filter = arg;
if (mcnt < VTNET_MAX_MAC_ENTRIES)
bcopy(LLADDR(sdl), &filter->vmf_multicast.macs[mcnt],
ETHER_ADDR_LEN);
return (1);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static void
vtnet_rx_filter_mac(struct vtnet_softc *sc)
{
Force two byte alignment for all control message headers The header structure consists of two 1-byte elements, but it must always be describable by a single SG entry. Note for consistency, specify the alignment everywhere, even if the structure has the appropriate natural alignment since it contains a uint16_t. Obtained from: DragonFlyBSD MFC after: 1 week
2014-06-16 00:32:27 -04:00
struct virtio_net_ctrl_hdr hdr __aligned(2);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct vtnet_mac_filter *filter;
struct sglist_seg segs[4];
struct sglist sg;
struct ifnet *ifp;
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
bool promisc, allmulti;
u_int ucnt, mcnt;
int error;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
uint8_t ack;
ifp = sc->vtnet_ifp;
Catch up with Bryan Venteicher's virtio Hg repo: c162516 Remove vtblk_sector_size c162515 Wrap long license lines c162514 Remove vtblk_unit c162513 Wrap long lines in the license. c162512 Remove verbose messages when link goes up/down. A similar message is printed elsewhere as a result of if_link_state_change(). c162511 Explicity compare pointer to NULL c162510 Allocate the mac filter table at attach time. c162509 Add real BSD licenses to the header files copied from Linux. The chases upstream changes made in Linux awhile ago. c162508 Only notify if we actually dequeued something. c162507 Change a couple of if () { KASSERT(...) } to just KASSERTs. In non-debug kernels, the if() { } probably get optomized away, but I guess this is clearer. c162506 Remove VIRTIO_BLK_F_TOPOLOGY fields in the config. TOPOLOGY has since been removed from the spec, and the FreeBSD didn't really do anything with the fields anyways. c162505 Move vtblk_enqueue_request() outside the locks when getting the ident. c162504 Remove soon to be uneeded trylock during dump [1]. http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html c162503 Remove emtpy line c162502 Drop frame if cannot allocate a vtnet_tx_header. If we don't, we set OACTIVE, but if there are no other frames in flight, vtnet_txeof() will never be called to unset OACTIVE. The interface would have to be down/up'ed in order to become usable. We could be cuter here and only do this if the virtqueue is emtpy, but its probably not worth the complication. c162501 Start mbuf replacement loop at 1 for clarity Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org
2011-12-06 01:28:32 -05:00
filter = sc->vtnet_mac_filter;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(sc->vtnet_flags & VTNET_FLAG_CTRL_RX);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_ASSERT(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/* Unicast MAC addresses: */
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
ucnt = if_foreach_lladdr(ifp, vtnet_copy_ifaddr, sc);
promisc = (ucnt > VTNET_MAX_MAC_ENTRIES);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
if (promisc) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
ucnt = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_printf(ifp, "more than %d MAC addresses assigned, "
"falling back to promiscuous mode\n",
VTNET_MAX_MAC_ENTRIES);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
/* Multicast MAC addresses: */
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
mcnt = if_foreach_llmaddr(ifp, vtnet_copy_maddr, filter);
allmulti = (mcnt > VTNET_MAX_MAC_ENTRIES);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
if (allmulti) {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
mcnt = 0;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_printf(ifp, "more than %d multicast MAC addresses "
"assigned, falling back to all-multicast mode\n",
VTNET_MAX_MAC_ENTRIES);
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Convert to if_foreach_llmaddr() KPI.
2019-10-21 13:59:02 -04:00
if (promisc && allmulti)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
goto out;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
filter->vmf_unicast.nentries = vtnet_gtoh32(sc, ucnt);
filter->vmf_multicast.nentries = vtnet_gtoh32(sc, mcnt);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
hdr.class = VIRTIO_NET_CTRL_MAC;
hdr.cmd = VIRTIO_NET_CTRL_MAC_TABLE_SET;
ack = VIRTIO_NET_ERR;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sglist_init(&sg, nitems(segs), segs);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
error |= sglist_append(&sg, &hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &filter->vmf_unicast,
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sizeof(uint32_t) + ucnt * ETHER_ADDR_LEN);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
error |= sglist_append(&sg, &filter->vmf_multicast,
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sizeof(uint32_t) + mcnt * ETHER_ADDR_LEN);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
error |= sglist_append(&sg, &ack, sizeof(uint8_t));
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &ack, &sg, sg.sg_nseg - 1, 1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (ack != VIRTIO_NET_OK)
if_printf(ifp, "error setting host MAC filter table\n");
out:
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
if (promisc != 0 && vtnet_set_promisc(sc, true) != 0)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_printf(ifp, "cannot enable promiscuous mode\n");
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
if (allmulti != 0 && vtnet_set_allmulti(sc, true) != 0)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_printf(ifp, "cannot enable all-multicast mode\n");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
vtnet_exec_vlan_filter(struct vtnet_softc *sc, int add, uint16_t tag)
{
struct sglist_seg segs[3];
struct sglist sg;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
struct {
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
struct virtio_net_ctrl_hdr hdr __aligned(2);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint8_t pad1;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
uint16_t tag __aligned(2);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint8_t pad2;
uint8_t ack;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
} s;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
int error;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
error = 0;
MPASS(sc->vtnet_flags & VTNET_FLAG_VLAN_FILTER);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.hdr.class = VIRTIO_NET_CTRL_VLAN;
s.hdr.cmd = add ? VIRTIO_NET_CTRL_VLAN_ADD : VIRTIO_NET_CTRL_VLAN_DEL;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
s.tag = vtnet_gtoh16(sc, tag);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
s.ack = VIRTIO_NET_ERR;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
sglist_init(&sg, nitems(segs), segs);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
error |= sglist_append(&sg, &s.hdr, sizeof(struct virtio_net_ctrl_hdr));
error |= sglist_append(&sg, &s.tag, sizeof(uint16_t));
error |= sglist_append(&sg, &s.ack, sizeof(uint8_t));
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(error == 0 && sg.sg_nseg == nitems(segs));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (error == 0)
vtnet_exec_ctrl_cmd(sc, &s.ack, &sg, sg.sg_nseg - 1, 1);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (s.ack == VIRTIO_NET_OK ? 0 : EIO);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_rx_filter_vlan(struct vtnet_softc *sc)
{
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
int i, bit;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
uint32_t w;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
uint16_t tag;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
MPASS(sc->vtnet_flags & VTNET_FLAG_VLAN_FILTER);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK_ASSERT(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
/* Enable the filter for each configured VLAN. */
for (i = 0; i < VTNET_VLAN_FILTER_NWORDS; i++) {
w = sc->vtnet_vlan_filter[i];
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
while ((bit = ffs(w) - 1) != -1) {
w &= ~(1 << bit);
tag = sizeof(w) * CHAR_BIT * i + bit;
if (vtnet_exec_vlan_filter(sc, 1, tag) != 0) {
device_printf(sc->vtnet_dev,
"cannot enable VLAN %d filter\n", tag);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
}
}
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_update_vlan_filter(struct vtnet_softc *sc, int add, uint16_t tag)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
struct ifnet *ifp;
int idx, bit;
ifp = sc->vtnet_ifp;
idx = (tag >> 5) & 0x7F;
bit = tag & 0x1F;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (tag == 0 || tag > 4095)
return;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (add)
sc->vtnet_vlan_filter[idx] |= (1 << bit);
else
sc->vtnet_vlan_filter[idx] &= ~(1 << bit);
if (ifp->if_capenable & IFCAP_VLAN_HWFILTER &&
vtnet: don't update VLAN filter when parent is not running Submitted by: Gerrie Roos <groos -at- xiplink -dot- com> Reviewed by: gnn Sponsored by: XipLink, Inc. Differential Revision: https://reviews.freebsd.org/D9573
2017-02-13 16:44:29 -05:00
ifp->if_drv_flags & IFF_DRV_RUNNING &&
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_exec_vlan_filter(sc, add, tag) != 0) {
device_printf(sc->vtnet_dev,
"cannot %s VLAN %d %s the host filter table\n",
add ? "add" : "remove", tag, add ? "to" : "from");
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_register_vlan(void *arg, struct ifnet *ifp, uint16_t tag)
{
if (ifp->if_softc != arg)
return;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_update_vlan_filter(arg, 1, tag);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_unregister_vlan(void *arg, struct ifnet *ifp, uint16_t tag)
{
if (ifp->if_softc != arg)
return;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_update_vlan_filter(arg, 0, tag);
}
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
static void
vtnet_update_speed_duplex(struct vtnet_softc *sc)
{
struct ifnet *ifp;
uint32_t speed;
ifp = sc->vtnet_ifp;
if ((sc->vtnet_features & VIRTIO_NET_F_SPEED_DUPLEX) == 0)
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
return;
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
/* BMV: Ignore duplex. */
speed = virtio_read_dev_config_4(sc->vtnet_dev,
offsetof(struct virtio_net_config, speed));
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
if (speed != UINT32_MAX)
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
ifp->if_baudrate = IF_Mbps(speed);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static int
vtnet_is_link_up(struct vtnet_softc *sc)
{
uint16_t status;
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
if ((sc->vtnet_features & VIRTIO_NET_F_STATUS) == 0)
return (1);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Cleanup the interface setup methods Defer the ether_ifattach until the interface capabilities are configured Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27913
2021-01-18 23:55:25 -05:00
status = virtio_read_dev_config_2(sc->vtnet_dev,
offsetof(struct virtio_net_config, status));
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return ((status & VIRTIO_NET_S_LINK_UP) != 0);
}
static void
vtnet_update_link_status(struct vtnet_softc *sc)
{
struct ifnet *ifp;
int link;
ifp = sc->vtnet_ifp;
VTNET_CORE_LOCK_ASSERT(sc);
link = vtnet_is_link_up(sc);
/* Notify if the link status has changed. */
if (link != 0 && sc->vtnet_link_active == 0) {
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
vtnet_update_speed_duplex(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
sc->vtnet_link_active = 1;
if_link_state_change(ifp, LINK_STATE_UP);
} else if (link == 0 && sc->vtnet_link_active != 0) {
sc->vtnet_link_active = 0;
if_link_state_change(ifp, LINK_STATE_DOWN);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726
2021-02-22 12:26:21 -05:00
vtnet_ifmedia_upd(struct ifnet *ifp __unused)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
return (EOPNOTSUPP);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
vtnet_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr)
{
struct vtnet_softc *sc;
sc = ifp->if_softc;
ifmr->ifm_status = IFM_AVALID;
ifmr->ifm_active = IFM_ETHER;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_LOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
if (vtnet_is_link_up(sc) != 0) {
ifmr->ifm_status |= IFM_ACTIVE;
if_vtnet: Support VIRTIO_NET_F_SPEED_DUPLEX This features lets the guest driver know the speed and duplex of the "link". Instead of trying to support many media types based on the possible/likely speeds/duplexes, only use the speed to set the interface baudrate. Cleanup ifmedia code to match other drivers. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27908
2021-01-18 23:55:24 -05:00
ifmr->ifm_active |= IFM_10G_T | IFM_FDX;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
} else
ifmr->ifm_active |= IFM_NONE;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
VTNET_CORE_UNLOCK(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
vtnet_get_macaddr(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
if (sc->vtnet_flags & VTNET_FLAG_MAC) {
virtio_read_device_config_array(sc->vtnet_dev,
offsetof(struct virtio_net_config, mac),
&sc->vtnet_hwaddr[0], sizeof(uint8_t), ETHER_ADDR_LEN);
} else {
/* Generate a random locally administered unicast address. */
sc->vtnet_hwaddr[0] = 0xB2;
arc4rand(&sc->vtnet_hwaddr[1], ETHER_ADDR_LEN - 1, 0);
}
}
static void
vtnet_set_macaddr(struct vtnet_softc *sc)
{
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
device_t dev;
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
int error;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
dev = sc->vtnet_dev;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
if (sc->vtnet_flags & VTNET_FLAG_CTRL_MAC) {
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
error = vtnet_ctrl_mac_cmd(sc, sc->vtnet_hwaddr);
if (error)
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
device_printf(dev, "unable to set MAC address\n");
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
return;
}
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
/* MAC in config is read-only in modern VirtIO. */
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
if (!vtnet_modern(sc) && sc->vtnet_flags & VTNET_FLAG_MAC) {
for (int i = 0; i < ETHER_ADDR_LEN; i++) {
if_vtnet: Cleanup the reinit process In modern VirtIO, the virtqueues cannot be notified before setting DRIVER_OK status. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27932
2021-01-18 23:55:25 -05:00
virtio_write_dev_config_1(dev,
Read and write the MAC address in the config space byte by byte
2014-01-25 02:13:47 -05:00
offsetof(struct virtio_net_config, mac) + i,
sc->vtnet_hwaddr[i]);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
}
static void
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
vtnet_attached_set_macaddr(struct vtnet_softc *sc)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
{
if_vtnet: Defer updating generated MAC address until attached This improves spec compliance because the driver is not suppose to notify the device prior to setting the DRIVER_OK status, which could happen with the VIRTIO_NET_F_CTRL_MAC_ADDR. The VIRTIO_NET_F_MAC feature should always be negotiated so would be a rare situation. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27910
2021-01-18 23:55:24 -05:00
/* Assign MAC address if it was generated. */
if ((sc->vtnet_flags & VTNET_FLAG_MAC) == 0)
vtnet_set_macaddr(sc);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_vlan_tag_remove(struct mbuf *m)
{
struct ether_vlan_header *evh;
evh = mtod(m, struct ether_vlan_header *);
m->m_pkthdr.ether_vtag = ntohs(evh->evl_tag);
m->m_flags |= M_VLANTAG;
/* Strip the 802.1Q header. */
bcopy((char *) evh, (char *) evh + ETHER_VLAN_ENCAP_LEN,
ETHER_HDR_LEN - ETHER_TYPE_LEN);
m_adj(m, ETHER_VLAN_ENCAP_LEN);
}
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
static void
vtnet_set_rx_process_limit(struct vtnet_softc *sc)
{
int limit;
limit = vtnet_tunable_int(sc, "rx_process_limit",
vtnet_rx_process_limit);
if (limit < 0)
limit = INT_MAX;
sc->vtnet_rx_process_limit = limit;
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_setup_rxq_sysctl(struct sysctl_ctx_list *ctx,
struct sysctl_oid_list *child, struct vtnet_rxq *rxq)
{
struct sysctl_oid *node;
struct sysctl_oid_list *list;
struct vtnet_rxq_stats *stats;
char namebuf[16];
snprintf(namebuf, sizeof(namebuf), "rxq%d", rxq->vtnrx_id);
node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, namebuf,
Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718
2020-02-26 09:26:36 -05:00
CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "Receive Queue");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
list = SYSCTL_CHILDREN(node);
stats = &rxq->vtnrx_stats;
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "ipackets", CTLFLAG_RD,
&stats->vrxs_ipackets, "Receive packets");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "ibytes", CTLFLAG_RD,
&stats->vrxs_ibytes, "Receive bytes");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "iqdrops", CTLFLAG_RD,
&stats->vrxs_iqdrops, "Receive drops");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "ierrors", CTLFLAG_RD,
&stats->vrxs_ierrors, "Receive errors");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "csum", CTLFLAG_RD,
&stats->vrxs_csum, "Receive checksum offloaded");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "csum_failed", CTLFLAG_RD,
&stats->vrxs_csum_failed, "Receive checksum offload failed");
if_vtnet: Add counter for received host LRO Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27928
2021-01-18 23:55:26 -05:00
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "host_lro", CTLFLAG_RD,
&stats->vrxs_host_lro, "Receive host segmentation offloaded");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "rescheduled", CTLFLAG_RD,
&stats->vrxs_rescheduled,
"Receive interrupt handler rescheduled");
}
static void
vtnet_setup_txq_sysctl(struct sysctl_ctx_list *ctx,
struct sysctl_oid_list *child, struct vtnet_txq *txq)
{
struct sysctl_oid *node;
struct sysctl_oid_list *list;
struct vtnet_txq_stats *stats;
char namebuf[16];
snprintf(namebuf, sizeof(namebuf), "txq%d", txq->vtntx_id);
node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, namebuf,
Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718
2020-02-26 09:26:36 -05:00
CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "Transmit Queue");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
list = SYSCTL_CHILDREN(node);
stats = &txq->vtntx_stats;
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "opackets", CTLFLAG_RD,
&stats->vtxs_opackets, "Transmit packets");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "obytes", CTLFLAG_RD,
&stats->vtxs_obytes, "Transmit bytes");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "omcasts", CTLFLAG_RD,
&stats->vtxs_omcasts, "Transmit multicasts");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "csum", CTLFLAG_RD,
&stats->vtxs_csum, "Transmit checksum offloaded");
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "tso", CTLFLAG_RD,
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
&stats->vtxs_tso, "Transmit TCP segmentation offloaded");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, list, OID_AUTO, "rescheduled", CTLFLAG_RD,
&stats->vtxs_rescheduled,
"Transmit interrupt handler rescheduled");
}
static void
vtnet_setup_queue_sysctl(struct vtnet_softc *sc)
{
device_t dev;
struct sysctl_ctx_list *ctx;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
struct sysctl_oid *tree;
struct sysctl_oid_list *child;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int i;
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
dev = sc->vtnet_dev;
ctx = device_get_sysctl_ctx(dev);
tree = device_get_sysctl_tree(dev);
child = SYSCTL_CHILDREN(tree);
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_req_vq_pairs; i++) {
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_setup_rxq_sysctl(ctx, child, &sc->vtnet_rxqs[i]);
vtnet_setup_txq_sysctl(ctx, child, &sc->vtnet_txqs[i]);
}
}
static void
vtnet_setup_stat_sysctl(struct sysctl_ctx_list *ctx,
struct sysctl_oid_list *child, struct vtnet_softc *sc)
{
struct vtnet_statistics *stats;
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
struct vtnet_rxq_stats rxaccum;
struct vtnet_txq_stats txaccum;
vtnet_accum_stats(sc, &rxaccum, &txaccum);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
stats = &sc->vtnet_stats;
- Provide if_get_counter() method for vtnet(4). - Do not accumulate statistics on every tick. - Accumulate statistics in vtnet_setup_stat_sysctl() and in vtnet_get_counter(). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 15:15:40 -04:00
stats->rx_csum_offloaded = rxaccum.vrxs_csum;
stats->rx_csum_failed = rxaccum.vrxs_csum_failed;
stats->rx_task_rescheduled = rxaccum.vrxs_rescheduled;
stats->tx_csum_offloaded = txaccum.vtxs_csum;
stats->tx_tso_offloaded = txaccum.vtxs_tso;
stats->tx_task_rescheduled = txaccum.vtxs_rescheduled;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "mbuf_alloc_failed",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->mbuf_alloc_failed,
"Mbuf cluster allocation failures");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_frame_too_large",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_frame_too_large,
"Received frame larger than the mbuf chain");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_enq_replacement_failed",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_enq_replacement_failed,
"Enqueuing the replacement receive mbuf failed");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_mergeable_failed",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_mergeable_failed,
"Mergeable buffers receive failures");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_bad_ethtype",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_csum_bad_ethtype,
"Received checksum offloaded buffer with unsupported "
"Ethernet type");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_bad_ipproto",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_csum_bad_ipproto,
"Received checksum offloaded buffer with incorrect IP protocol");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_bad_offset",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_csum_bad_offset,
"Received checksum offloaded buffer with incorrect offset");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_bad_proto",
CTLFLAG_RD, &stats->rx_csum_bad_proto,
"Received checksum offloaded buffer with incorrect protocol");
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_failed",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_csum_failed,
"Received buffer checksum offload failed");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_csum_offloaded",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_csum_offloaded,
"Received buffer checksum offload succeeded");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "rx_task_rescheduled",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->rx_task_rescheduled,
"Times the receive interrupt task rescheduled itself");
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_csum_unknown_ethtype",
CTLFLAG_RD, &stats->tx_csum_unknown_ethtype,
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
"Aborted transmit of checksum offloaded buffer with unknown "
"Ethernet type");
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_csum_proto_mismatch",
CTLFLAG_RD, &stats->tx_csum_proto_mismatch,
"Aborted transmit of checksum offloaded buffer because mismatched "
"protocols");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_tso_not_tcp",
CTLFLAG_RD, &stats->tx_tso_not_tcp,
"Aborted transmit of TSO buffer with non TCP protocol");
if_vtnet: Misc Tx path cleanup - Add and fix a few error path counters - Improve sysctl descriptions - Use flags consistently to determine IPv4 vs IPv6 Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27926
2021-01-18 23:55:26 -05:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_tso_without_csum",
CTLFLAG_RD, &stats->tx_tso_without_csum,
"Aborted transmit of TSO buffer without TCP checksum offload");
Use m_defrag() instead of m_collapse() to compact a long mbuf chain This should be an infrequent occurrence, so remove the per-queue counters in favor of just global counters in the softc.
2014-02-02 00:20:46 -05:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_defragged",
CTLFLAG_RD, &stats->tx_defragged,
"Transmit mbufs defragged");
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_defrag_failed",
CTLFLAG_RD, &stats->tx_defrag_failed,
"Aborted transmit of buffer because defrag failed");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_csum_offloaded",
CTLFLAG_RD, &stats->tx_csum_offloaded,
"Offloaded checksum of transmitted buffer");
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_tso_offloaded",
CTLFLAG_RD, &stats->tx_tso_offloaded,
"Segmentation offload of transmitted buffer");
SYSCTL_ADD_UQUAD(ctx, child, OID_AUTO, "tx_task_rescheduled",
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
CTLFLAG_RD, &stats->tx_task_rescheduled,
"Times the transmit interrupt task rescheduled itself");
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
static void
vtnet_setup_sysctl(struct vtnet_softc *sc)
{
device_t dev;
struct sysctl_ctx_list *ctx;
struct sysctl_oid *tree;
struct sysctl_oid_list *child;
dev = sc->vtnet_dev;
ctx = device_get_sysctl_ctx(dev);
tree = device_get_sysctl_tree(dev);
child = SYSCTL_CHILDREN(tree);
SYSCTL_ADD_INT(ctx, child, OID_AUTO, "max_vq_pairs",
CTLFLAG_RD, &sc->vtnet_max_vq_pairs, 0,
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
"Number of maximum supported virtqueue pairs");
if_vtnet: Rework 4be723f63 max multiqueue pairs check Verify the max_virtqueue_pairs is within the range allowed by the spec. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27920
2021-01-18 23:55:25 -05:00
SYSCTL_ADD_INT(ctx, child, OID_AUTO, "req_vq_pairs",
CTLFLAG_RD, &sc->vtnet_req_vq_pairs, 0,
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
"Number of requested virtqueue pairs");
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
SYSCTL_ADD_INT(ctx, child, OID_AUTO, "act_vq_pairs",
CTLFLAG_RD, &sc->vtnet_act_vq_pairs, 0,
"Number of active virtqueue pairs");
vtnet_setup_stat_sysctl(ctx, child, sc);
}
if_vtnet: Add support for software LRO This useful when running on hosts that support checksum offloading but not the GUEST_TSO (LRO) feature. Or potentially, some GRO-like support when doing forwarding. Only enable SW LRO when the host LRO is not available since both tends to be harmful, and difficult to enable/disable selectively with only a single IFCAP_LRO flag. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27919
2021-01-18 23:55:25 -05:00
static void
vtnet_load_tunables(struct vtnet_softc *sc)
{
sc->vtnet_lro_entry_count = vtnet_tunable_int(sc,
"lro_entry_count", vtnet_lro_entry_count);
if (sc->vtnet_lro_entry_count < TCP_LRO_ENTRIES)
sc->vtnet_lro_entry_count = TCP_LRO_ENTRIES;
sc->vtnet_lro_mbufq_depth = vtnet_tunable_int(sc,
"lro_mbufq_depth", vtnet_lro_mbufq_depth);
}
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_enable_intr(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (virtqueue_enable_intr(rxq->vtnrx_vq));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_disable_intr(struct vtnet_rxq *rxq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
virtqueue_disable_intr(rxq->vtnrx_vq);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static int
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_txq_enable_intr(struct vtnet_txq *txq)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
struct virtqueue *vq;
vq = txq->vtntx_vq;
if (vtnet_txq_below_threshold(txq) != 0)
return (virtqueue_postpone_intr(vq, VQ_POSTPONE_LONG));
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Rework when the Tx queue completion interrupt is enabled The Tx interrupt is now kept disabled in the common case, only enabled when the number of free descriptors in the queue falls below a threshold. Transmitted frames are cleared from the VQ before subsequent transmit, or in the watchdog timer. This was a very big performance improvement for an experimental Netmap bhyve backend. MFC after: 1 month
2014-07-10 01:36:04 -04:00
/*
* The free count is above our threshold. Keep the Tx interrupt
* disabled until the queue is fuller.
*/
return (0);
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_txq_disable_intr(struct vtnet_txq *txq)
{
virtqueue_disable_intr(txq->vtntx_vq);
}
static void
vtnet_enable_rx_interrupts(struct vtnet_softc *sc)
{
if_vtnet: Schedule Rx task if pending items when enabling interrupt Prior to V1, the driver would enable interrupts and then notify the host that DRIVER_OK. Since for V1, DRIVER_OK needs to be set before notifying the virtqueues, there may be items in the queues waiting to be processed by the time interrupts are enabled. This fixes a bug where the Rx queue would appear stuck, only being usable after an interface down/up cycle. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27922
2021-01-18 23:55:25 -05:00
struct vtnet_rxq *rxq;
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int i;
if_vtnet: Schedule Rx task if pending items when enabling interrupt Prior to V1, the driver would enable interrupts and then notify the host that DRIVER_OK. Since for V1, DRIVER_OK needs to be set before notifying the virtqueues, there may be items in the queues waiting to be processed by the time interrupts are enabled. This fixes a bug where the Rx queue would appear stuck, only being usable after an interface down/up cycle. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27922
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_act_vq_pairs; i++) {
rxq = &sc->vtnet_rxqs[i];
if (vtnet_rxq_enable_intr(rxq) != 0)
taskqueue_enqueue(rxq->vtnrx_tq, &rxq->vtnrx_intrtask);
}
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
}
static void
vtnet_enable_tx_interrupts(struct vtnet_softc *sc)
{
int i;
for (i = 0; i < sc->vtnet_act_vq_pairs; i++)
vtnet_txq_enable_intr(&sc->vtnet_txqs[i]);
}
static void
vtnet_enable_interrupts(struct vtnet_softc *sc)
{
vtnet_enable_rx_interrupts(sc);
vtnet_enable_tx_interrupts(sc);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
static void
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_disable_rx_interrupts(struct vtnet_softc *sc)
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
{
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
int i;
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_max_vq_pairs; i++)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_rxq_disable_intr(&sc->vtnet_rxqs[i]);
}
static void
vtnet_disable_tx_interrupts(struct vtnet_softc *sc)
{
int i;
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_max_vq_pairs; i++)
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
vtnet_txq_disable_intr(&sc->vtnet_txqs[i]);
}
static void
vtnet_disable_interrupts(struct vtnet_softc *sc)
{
vtnet_disable_rx_interrupts(sc);
vtnet_disable_tx_interrupts(sc);
}
static int
vtnet_tunable_int(struct vtnet_softc *sc, const char *knob, int def)
{
char path[64];
snprintf(path, sizeof(path),
"hw.vtnet.%d.%s", device_get_unit(sc->vtnet_dev), knob);
TUNABLE_INT_FETCH(path, &def);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
Import multiqueue VirtIO net driver from my user/bryanv/vtnetmq branch This is a significant rewrite of much of the previous driver; lots of misc. cleanup was also performed, and support for a few other minor features was also added.
2013-09-01 00:33:47 -04:00
return (def);
Import virtio base, PCI front-end, and net/block/balloon drivers. Tested on Qemu/KVM, VirtualBox, and BHyVe. Currently built as modules-only on i386/amd64. Man pages not yet hooked up, pending review. Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org Reviewed by: bz MFC after: 4 weeks or so
2011-11-18 00:43:43 -05:00
}
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
#ifdef DEBUGNET
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
static void
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
vtnet_debugnet_init(struct ifnet *ifp, int *nrxr, int *ncl, int *clsize)
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
{
struct vtnet_softc *sc;
sc = if_getsoftc(ifp);
VTNET_CORE_LOCK(sc);
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
*nrxr = sc->vtnet_req_vq_pairs;
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
*ncl = DEBUGNET_MAX_IN_FLIGHT;
if_vtnet: Add initial modern (V1) support Very basic support to get packets flowing on modern QEMU but still several conformance issues remain that will be addressed in later commits. First of many passes at cleaning up various accumulated cruft Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27904
2021-01-18 23:55:24 -05:00
*clsize = sc->vtnet_rx_clustersz;
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
VTNET_CORE_UNLOCK(sc);
}
static void
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
vtnet_debugnet_event(struct ifnet *ifp __unused, enum debugnet_ev event __unused)
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
{
}
static int
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
vtnet_debugnet_transmit(struct ifnet *ifp, struct mbuf *m)
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
{
struct vtnet_softc *sc;
struct vtnet_txq *txq;
int error;
sc = if_getsoftc(ifp);
if ((if_getdrvflags(ifp) & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) !=
IFF_DRV_RUNNING)
return (EBUSY);
txq = &sc->vtnet_txqs[0];
error = vtnet_txq_encap(txq, &m, M_NOWAIT | M_USE_RESERVE);
if (error == 0)
(void)vtnet_txq_notify(txq);
return (error);
}
static int
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
vtnet_debugnet_poll(struct ifnet *ifp, int count)
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
{
struct vtnet_softc *sc;
int i;
sc = if_getsoftc(ifp);
if ((if_getdrvflags(ifp) & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) !=
IFF_DRV_RUNNING)
return (EBUSY);
(void)vtnet_txq_eof(&sc->vtnet_txqs[0]);
if_vtnet: Limit allocations of unused virtqueues For multiqueue, we may use fewer than the provided maximum number of queues. Try to limit allocations of the unused queues: no interrupts, no indirect descriptors, and no taskqueues. Reviewed by: grehan (mentor) Differential Revision: https://reviews.freebsd.org/D27921
2021-01-18 23:55:25 -05:00
for (i = 0; i < sc->vtnet_act_vq_pairs; i++)
Add netdump support to vtnet(4). Tested with bhyve. Reviewed by: bryanv, julian MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15261
2018-05-05 20:53:52 -04:00
(void)vtnet_rxq_eof(&sc->vtnet_rxqs[i]);
return (0);
}
Split out a more generic debugnet(4) from netdump(4) Debugnet is a simplistic and specialized panic- or debug-time reliable datagram transport. It can drive a single connection at a time and is currently unidirectional (debug/panic machine transmit to remote server only). It is mostly a verbatim code lift from netdump(4). Netdump(4) remains the only consumer (until the rest of this patch series lands). The INET-specific logic has been extracted somewhat more thoroughly than previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as much as possible as is protocol-independent, remains in debugnet.c. The separation is not perfect and future improvement is welcome. Supporting INET6 is a long-term goal. Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to 'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the generic module would be more confusing than the refactoring. The only functional change here is the mbuf allocation / tracking. Instead of initiating solely on netdump-configured interface(s) at dumpon(8) configuration time, we watch for any debugnet-enabled NIC for link activation and query it for mbuf parameters at that time. If they exceed the existing high-water mark allocation, we re-allocate and track the new high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone. In a future patch in this series, this will allow initiating netdump from panic ddb(4) without pre-panic configuration. No other functional change intended. Reviewed by: markj (earlier version) Some discussion with: emaste, jhb Objection from: marius Differential Revision: https://reviews.freebsd.org/D21421
2019-10-17 12:23:03 -04:00
#endif /* DEBUGNET */
Reference in a new issue Copy permalink