Commit graph

321 commits

Author SHA1 Message Date
Vladimir Kondratyev
062f15004f LinuxKPI: Remove vma argument from fault method of vm_operations_struct
It is removed from Linux since 4.11.
In FreeBSD it results in several #ifdefs in drm-kmod.

Reviewed by:	emaste, hselasky, manu
Differential revision:	https://reviews.freebsd.org/D32169
2021-09-29 23:26:32 +03:00
Vladimir Kondratyev
5ca1f3f5e3 LinuxKPI: Hide some internal symbols in linux_interrupt.c
Reviewed by:	hselasky, manu
Differential revision:	https://reviews.freebsd.org/D32168
2021-09-29 23:26:14 +03:00
Vladimir Kondratyev
c072f6e856 LinuxKPI: Import linux_page.c and some dependent code from drm-kmod
No functional changes intended

Reviewed by:	hselasky, manu, markj
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32167
2021-09-29 23:15:37 +03:00
Vladimir Kondratyev
b52e363840 LinuxKPI: Implement backlight_enable and backlight_disable functions
For now, disable backlight if brightness level is set to 0.
In the future we may implement separate knob in backlight(8).

Required by drm-kmod v5.6

Reviewed by:	hselasky, manu
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32165
2021-09-29 23:15:12 +03:00
Vladimir Kondratyev
62ff0566c9 LinuxKPI: Allow cdev_pager prefault handler to steal pages
from other vm_objects. This workarounds "Page already inserted" panic
in vm_page_insert routine triggered on attempt to mmap file created
with shmem_file_setup call. After introduction of "GTT mmap
interface v4" a.k.a. MMAP_OFFSET, vm_objects allocated by these calls
may try to own intersected sets of pages that leads to the assertion.

Reviewed by:	kib
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32090
2021-09-29 23:14:05 +03:00
Vladimir Kondratyev
f6823dac8f LinuxKPI: Factor out vmf_insert_pfn_prot() routine
from GEM and TTM page fault handlers and move it in to base system. This
code is tightly integrated with LKPI mmap support to belong to drm-kmod.

As this routine requires associated vm_object to be locked, it got
additional _locked suffix.

Reviewed by:	hselasky, markj
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32068
2021-09-29 23:13:41 +03:00
Vladimir Kondratyev
7d92d48358 LinuxKPI: Invoke release handler when file is destroyed by fput()
Required by drm_kmod 5.6

Reviewed by:	hselasky, manu
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32067
2021-09-29 23:13:27 +03:00
Vladimir Kondratyev
2fe9ea5d3a LinuxKPI: allocate current before taking shrinkers lock
This fixes following warnings when shrinkers are invoked first time:

uma_zalloc_debug: zone "lkpicurr" with the following non-sleepable
locks held: exclusive sleep mutex lkpi-shrinker (lkpi-shrinker)

uma_zalloc_debug: zone "lkpimm" with the following non-sleepable locks
held: exclusive sleep mutex lkpi-shrinker (lkpi-shrinker)

Reviewed by:	hselasky, manu
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D32066
2021-09-29 23:12:58 +03:00
Vladimir Kondratyev
66ea390652 LinuxKPI: Remove FreeBSD struct resource from all LKPI headers
except linux/pci.h to avoid conflicts with Linux version.
This allows to #define resource in drm-kmod globally and strip some #ifdef-s

Reviewed by:	hselasky, manu
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D31673
2021-09-29 23:12:36 +03:00
Vladimir Kondratyev
a81b36c6d3 LinuxKPI: Implement get_file_rcu()
get_file_rcu() grabs a file if the file->f_count is not zero.

Required by drm-kmod 5.6

Reviewed by:	hselasky, manu (previous version)
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D31672
2021-09-29 23:12:25 +03:00
Bjoern A. Zeeb
1269873159 LinuxKPI: fix build
Add a missing "static" for non-{i386,amd64,arm64} which was missed in
c39eefe715.   This should ifx the builds.

Sponsored by:	The FreeBSD Foundation
MFC after:	7 days
X-MFC with:	c39eefe715
2021-09-29 13:50:12 +00:00
Bjoern A. Zeeb
c39eefe715 LinuxKPI: implement dma_set_coherent_mask()
Coherent is lower 32bit only by default in Linux and our only default
dma mask is 64bit currently which violates expectations unless
dma_set_coherent_mask() was called explicitly with a different mask.

Implement coherent by creating a second tag, and storing the tags in the
objects and use the tag from the object wherever possible.
This currently does not update the scatterlist or pool (both could be
converted but S/G cannot be MFCed as easily).

There is a 2nd change embedded in the updated logic of
linux_dma_alloc_coherent() to always zero the allocation as
otherwise some drivers get cranky on uninialised garbage.

Sponsored by:	The FreeBSD Foundation
MFC after:	7 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D32164
2021-09-29 12:41:28 +00:00
Hans Petter Selasky
469884cf04 LinuxKPI: Make FPU sections thread-safe and use the NOCTX flag.
Reviewed by:	kib
Submitted by:	greg@unrelenting.technology
Differential Revision:	https://reviews.freebsd.org/D29921
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2021-07-31 15:36:48 +02:00
Konstantin Belousov
d96f55bc71 linuxkpi: remove global atomic counter of the task allocations
Use thread_reap_barrier() to ensure that no threads are kept in the
zombies list which could have the linuxkpi task allocated.

Also fix order of initialization and teardown for current task
allocation hooks and resources. Register current task allocator after
zones are initialized. Deregister allocator before cycling over threads
and zeroing task pointer.

Reviewed by:	hselasky, markj
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D30468
2021-07-27 20:01:19 +03:00
Vladimir Kondratyev
98a6984a9e LinuxKPI: Use macro for implementation of some dma_map_* functions
This allows to remove unimplemented attrs parameter which type differs
between Linux kernel versions and to compile both drm-kmod and ofed
callers unmodified.
Also convert it to 'unsigned long' type to match modern Linuxes.

Reviewed by:	hselasky
Differential revision:	https://reviews.freebsd.org/D30932
2021-07-05 03:20:23 +03:00
Vladimir Kondratyev
1ab61a1932 LinuxKPI: Do not wait for a grace period in rcu_barrier()
Linux docs explicitly state that this is not required [1]:

"Important note: The rcu_barrier() function is not, repeat, not,
obligated to wait for a grace period.  It is instead only required to
wait for RCU callbacks that have already been posted.  Therefore, if
there are no RCU callbacks posted anywhere in the system, rcu_barrier()
is within its rights to return immediately.  Even if there are
callbacks posted, rcu_barrier() does not necessarily need to wait for
a grace period."

[1] https://www.kernel.org/doc/Documentation/RCU/Design/Requirements/Requirements.html

Reviewed by:	emaste, hselasky, manu
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D30809
2021-07-05 03:19:50 +03:00
Vladimir Kondratyev
a2b83b59db LinuxKPI: Allow kmem_cache_free() to be called from critical sections
as it is required by i915kms driver from Linux kernel v 5.5.
This is done with asynchronous freeing of requested memory areas from
taskqueue thread. As memory to be freed is reused to store linked list
entry, backing UMA zone item size is rounded up to pointer size.

While here, make struct linux_kmem_cache private to LKPI to reduce amount
of BSD headers included by linux/slab.h and switch RCU code to usage of
LKPI's linux_irq_work_tq taskqueue to avoid injection of current into
system-wide taskqueue_fast thread context.

Submitted by:	nc (initial version for drm-kmod)
Reviewed by:	manu, nc
Differential revision:	https://reviews.freebsd.org/D30760
2021-07-05 03:18:14 +03:00
Dmitry Chagin
5ca9d41700 LinuxKPI: Rename a short description of the kmalloc type.
To avoid duplication in the vmstat -m output rename the kmalloc type short
description to 'lkpikmalloc' as the Linux emulation layer historically names
its linux malloc type as 'linux'.

Reviewed by:		hselasky, kib, emaste
Differential Revision:	https://reviews.freebsd.org/D30928
MFC after:		2 weeks
2021-06-29 20:20:01 +03:00
Dmitry Chagin
1fd26da926 LinuxKPI: Put compat code under appropriate condition.
Reviewed by:		hselasky, emaste, kib
Differential Revision:	https://reviews.freebsd.org/D30927
MFC after:		2 weeks
2021-06-29 20:19:17 +03:00
Dmitry Chagin
945accf502 LinuxKPI: Use the proper API to determine the ABI of the running process.
Reviewed by:		markj, hselasky, kib
Differential Revision:	https://reviews.freebsd.org/D30924
MFC after:		2 weeks
2021-06-29 20:17:16 +03:00
Bjoern A. Zeeb
399da52fff LinuxKPI: firmware, implement deferred loading for "nowait"
Change linuxkpi_request_firmware_nowait() to deferred firmware loading
scheduling a task.  This changes behaviour in some cases that we
return from loading the driver before the driver is finished
initialising if the driver does not deal with it (wait).
This brings the behaviour one would expect from when this function is
called and I implemented it to see if it would help a specific case.

Sponsored by:	The FreeBSD Foundation
MFC after:	12 days
Reviewed by:	hselasky, imp (earlier version)
Differential Revision: https://reviews.freebsd.org/D30830
2021-06-28 12:13:43 +00:00
Bjoern A. Zeeb
5f88df77a6 LinuxKPI: fix build after d4a4960c65 pci: "pcim" (managed) support
Fix a last minute change from d4a4960c65
based on review feedback in where a function now gets called before
it is declared which did not fully get merged back to my commit branch.

Noticed by:	CI, jkim
MFC after:	10 days
X-MFC with:	d4a4960c65
Sponsored-by:	The FreeBSD Foundation
2021-06-18 22:49:12 +00:00
Bjoern A. Zeeb
d4a4960c65 LinuxKPI: pci: cleanup some code and add support for "pcim" (managed)
Restructure some code and add support for various "managed" versions
for PCI resource management.
This is beyond of what iwlwifi needs but some was found with other
wireless drivers and it mostly all goes together.
Add one FreeBSD sepcific feature returning the resource rather than
the handle to allow us to use bus_*() functions in drivers directly.

Sponsored by:	The FreeBSD Foundation
MFC after:	10 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30558
2021-06-18 21:20:09 +00:00
Bjoern A. Zeeb
c3518147ce LinuxKPI: fix pci device devres initialisation
Given we are manually setting up the "device" in PCI in some cases,
we need to initialise the list and lock for device devres here as well
as otherwise we will panic on the uninitialised lock.

Sponsored by:	The FreeBSD Foundation
MFC after:	5 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30681
2021-06-18 21:20:09 +00:00
Bjoern A. Zeeb
d16b6cb178 LinuxKPI: enhance the irq KPI for managed and threaded operations.
Move request_irq() to an internal function which serves request_irq()
and the newly added request_threaded_irq() and devm_request_threaded_irq().
Likewise factor out parts of free_irq() to also be used with
devm_free_irq().  Add the storage and call to a thread_handler in case
of IRQ_WAKE_THREAD.
This is needed for the iwlwifi driver.

Sponsored by:	The FreeBSD Foundation
MFC after:	10 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30549
2021-06-18 21:20:09 +00:00
Bjoern A. Zeeb
8e106c5230 LinuxKPI: extend pci.h by various functions for wireless driver
Add dummy functions for dealing with "HotPlug" events which we currently
do not support.

Add pci_dev_get(), pci_find_ext_capability() and pci_pme_capable().

The added pcie_find_root_port() is a bit special as we need to create
another linux pci device;  for that make lkpinew_pci_dev() public
which is also helpful for other cases when we want to use the Linux
routines to check for device identifiers only and need a container
for the "bsddev" to use natively.  This has proven to avoid basic
checking code for the sake of rewriting it to native field names
elsewhere.  Given we cache the newly created "root" we also need to
make sure we clean it up.

Sponsored by:	The FreeBSD Foundation
MFC after:	10 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30521
2021-06-18 21:20:09 +00:00
Bjoern A. Zeeb
2afeed13b5 LinuxKPI: add dmam_pool_create() support
dmam_pool_create() is a "managed" version of dma_pool_create() which
will cleanup everything left when the device goes away using the
devres framework.  For that add an internal cleanup function to be
called from devres release.
This is used by at least one wireless driver.

Sponsored by:	The FreeBSD Foundation
MFC after:	10 days
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30520
2021-06-18 21:20:09 +00:00
Bjoern A. Zeeb
801cf532e7 LinuxKPI: add KPI for netdev_notifier_info returning ifp
While currently the ifp gets cast to a net_device and then returned
and consumers are expecting an ifp again, allow parallel usage now and
in the future by extending and also passing the ifp directly back in
the netdev_notifier_info.  Add a function to return the ifp instead of
the net_device.

Sponsored by:	The FreeBSD Foundation
MFC after:	10 days
Suggested by:	hselasky
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30522
2021-06-18 17:55:30 +00:00
Bjoern A. Zeeb
5fce802722 LinuxKPI: add cpu.h for cpumask_*()
Add linux/cpu.h for cpumask_*() functions found in wireless drivers
and make sure cpu_online_mask is always initialised.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D30421
2021-05-25 18:01:48 +00:00
Hans Petter Selasky
b764a42653 There is a window where threads are removed from the process list and where
the thread destructor is invoked. Catch that window by waiting for all
task_struct allocations to be returned before freeing the UMA zone in the
LinuxKPI. Else UMA may fail to release the zone due to concurrent access
and panic:

panic() - Bad link element prev->next != elm
zone_release()
bucket_drain()
bucket_free()
zone_dtor()
zone_free_item()
uma_zdestroy()
linux_current_uninit()

This failure can be triggered by loading and unloading the LinuxKPI module
in a loop:

while true
do
kldload linuxkpi
kldunload linuxkpi
done

Discussed with:	kib@
MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-05-21 13:18:41 +02:00
Hans Petter Selasky
209d4919c5 Make sure all tasklets are drained before unloading the LinuxKPI.
Else use-after-free may happen.

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-05-21 11:21:32 +02:00
Hans Petter Selasky
b8f113cab9 Implement cdev_device_add() and cdev_device_del() in the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-05-11 21:00:23 +02:00
Hans Petter Selasky
67807f5066 cdev_del() should only put it's kernel object in the LinuxKPI.
The destructor takes care of the rest.

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-05-11 21:00:23 +02:00
Hans Petter Selasky
904390b478 Implement read-only VM_SHARED flag in the LinuxKPI.
For use by mmap(2) callbacks.

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-05-11 21:00:14 +02:00
Neel Chauhan
9781105bea linuxkpi: Introduce tasklet_disable_nosync()
This is needed for the drm-kmod 5.5 update.

Reviewed by:		hselasky (src)
MFC after:		3 days
Differential Revision:	https://reviews.freebsd.org/D30024
2021-04-28 08:05:57 -07:00
Konstantin Belousov
fad437ba61 linuxkpi: reduce number of stray mm_struct allocations
Only allocate struct_mm after we checked that other threads do not carry
useful mm_struct.  If they don't, drop process lock, allocate, and recheck.

Note that for M_NOWAIT allocations we could avoid dropping process lock,
but I do not think that this increased complexity is useful.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-19 21:34:08 +03:00
Konstantin Belousov
165ba13fb8 linuxkpi: guarantee allocations of task and mm for interrupt threads
Create and use zones for task and mm.  Reserve items in zones based on the
estimation of the max number of interrupts in the system.  Use M_USE_RESERVE
to allow to take reserved items when allocation occurs from the interrupt
thread context.

Of course, this would only work first time we allocate the task for
interrupt thread. If interrupt is deallocated and allocated anew,
creating a new thread, it might be that zone is depleted. It still
should be good enough for practical uses.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-19 21:34:08 +03:00
Konstantin Belousov
4ce1f6162e linuxkpi: some style, wrap too long lines
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-19 21:34:07 +03:00
Konstantin Belousov
5b3b19db73 linuxkpi: remove erronously committed diff save file
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:42:13 +03:00
Konstantin Belousov
8011fb795b linuxkpi: drop single-use variable
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:38:29 +03:00
Konstantin Belousov
f6b108837e linuxkpi: avoid counting per-thread use for the embedded linux cdevs
The counter is not used to control destroy.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:38:29 +03:00
Konstantin Belousov
7f9867f8c6 linuxkpi: do not destroy/free embedded linux cdevs
They have their own lifetime managed by the containing objects.
Premature and unexpected free causes corruption.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:38:29 +03:00
Konstantin Belousov
28b482e2ba linuxkpi: rename cdev to ldev
the variables hold pointers to a linux_cdev, not to a FreeBSD cdev.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:38:28 +03:00
Konstantin Belousov
7b0125cbec linuxkpi: copy ldev into local to test and free the same pointer
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
2021-04-06 03:38:28 +03:00
Bjoern A. Zeeb
37c3241a43 LinuxKPI: treat firmware file names more lenient
A lot of firmware files have a "-" in the name.  That "-" is a problem
when dealing with shell variables or loader (e.g., auto-loading .ko).
It may thus often be convenient to generate firmware kernel object files
with s/-/_/g in the name.  In order to automatically find them from
drivers using LinuxKPI also substitue the '-' for a '_' like we do
for '/' and '.' already.

Reviewed-by:	hselasky, manu (ok)
MFC-after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D29514
2021-04-02 10:03:39 +00:00
Hans Petter Selasky
1777720880 Reduce chance of RCU deadlock in the LinuxKPI by implementing the section
feature of the concurrency kit, CK.

Differential Revision:	https://reviews.freebsd.org/D29467
Reviewed by:	kib@ and markj@
MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-03-29 10:55:14 +02:00
Bjoern A. Zeeb
fdcfe8a298 LinuxKPI: netdevice notifier callback argument
Introduce struct netdev_notifier_info as a container to pass
net_device to the callback functions.
Adjust netdev_notifier_info_to_dev() to return the net_device field.

Add explicit casts from ifp to ni->dev even though currently
struct net_device is defined to struct ifnet.  This is needed in
preparation for untangling this and improving the net_device compat
code.

Obtained-from:	bz_iwlwifi
Sponsored-by:	The FreeBSD Foundation
MFC-after:	2 weeks
Reviewed-by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D29365
2021-03-26 13:00:23 +00:00
Bjoern A. Zeeb
bc042266b2 LinuxKPI: add net_ratelimit()
Add a net_ratelimit() compat implementation based on ppsratecheck().
Add a sysctl to allow tuning of the number of messages.

Sponsored-by:	The FreeBSD Foundation
MFC-after:	2 weeks
Reviewed-by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D29399
2021-03-26 12:05:48 +00:00
Hans Petter Selasky
dfb33cb0ef Allocating the LinuxKPI current structure from a software interrupt thread
must be done using the M_NOWAIT flag after 1ae20f7c70 .

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-03-10 13:27:40 +01:00
Hans Petter Selasky
d1cbe79089 Allocating the LinuxKPI current structure from an interrupt thread must be
done using the M_NOWAIT flag after 1ae20f7c70 .

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2021-03-10 10:51:04 +01:00