Commit graph

394 commits

Author SHA1 Message Date
Chuck Silvers
8d280d9144 x86/ucode: add support for early loading of CPU ucode on AMD.
Sponsored by:	Netflix
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D43318
2024-12-11 13:34:58 +01:00
Chuck Silvers
ec1c35debb x86/ucode: add const where appropriate
Sponsored by:   Netflix
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D43865
2024-12-11 13:34:57 +01:00
Konstantin Belousov
90d8c39d22 x86 specialreg: add bit masks definitions for LASS and LAM features
(cherry picked from commit 9f718b57b846b8e1fa562584747fdae42ffed8f4)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
a00d2dd061 x86 specialreg: add bit masks definitions for EFER features
(cherry picked from commit 6308db659f2ad45b30bbf1d9c47abdc97d14ebb0)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
6f26bce468 x86 specialreg: add bit masks definitions for LAM in %cr3
(cherry picked from commit 5999b74e963769cbd12eadbe1cebcd34fb312f01)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
5a0da3fb38 x86 specialreg: reorder %cr3 bits masks definitions by value
(cherry picked from commit cc4b25f10211fdd5a347a5a9fc26b2c86476db7f)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
03005c9195 x86 specialreg.h: add all defined bits for %cr4
(cherry picked from commit cc11bc1150d59c3da5c7319c9a23b3e48b069f1f)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
0066c70cd8 x86 specialreg.h: visually align %cr4 and MSR_EFER bit mask definitions
(cherry picked from commit 2ac21f2c98edf50e22f06a63f3528a99f4f963a0)
2024-10-30 04:40:30 +02:00
Konstantin Belousov
858397f868 apic: add ioapic_get_dev() method
(cherry picked from commit ea4e4449b85593025c2eb31bb642732971d96225)
2024-10-24 05:44:40 +03:00
Konstantin Belousov
339b47f019 x86/iommu: extract useful utilities into x86_iommu.c
(cherry picked from commit 40d951bc5932deb87635f5c1780a6706d0c7c012)
2024-06-01 12:24:24 +03:00
Konstantin Belousov
f03a01194c AMD CPUs: update bits and data from CPUID 0x8000_0008
(cherry picked from commit c6113ac5a2c2bfee875979ebd40a007ef4d60069)
2024-05-21 05:45:59 +03:00
Koine Yuusuke
983aec544b x86: Add Intel TD/HFI related MSR/CPUID defines to specialregs.h
(cherry picked from commit 338d53965d9f4bfb1a83c1bcff3ff78944ce3a9f)
2024-05-21 05:45:59 +03:00
Ed Maste
18119e711f Increase IOAPIC_MAX_ID to 255 (from 254)
A test system provided by AMD panicked with "madt_parse_apics:
I/O APIC ID 255 too high".  I/O APIC ID 255 is acceptable, so increase
the limit.

Reviewed by:	jhb, kib
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45157

(cherry picked from commit 4e3286818327453efd8e4c133928ba26a169567a)
2024-05-15 13:21:14 -04:00
John Baldwin
4b5f644088 x86: Support multiple PCI MCFG regions
In particular, this enables support for PCI config access for domains
(segments) other than 0.

Reported by:	cperciva
Tested by:	cperciva (m7i.metal-48xl AWS instance)
Reviewed by:	imp
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D42828

(cherry picked from commit f54a3890b1f419d4a9edc99000efef52b9071b8f)
2024-01-18 15:23:44 -08:00
John Baldwin
b5d2a06b2c pci_cfgreg: Add shims to preserve ABI of pci_cfgreg(read|write)
This is a direct commit to stable/14 to preserve the ABI of the
the pci_cfgregread and pci_cfgregwrite functions.   The new routines
are renamed to add a _domain suffix and macros map the new API to
the new functions.

Note: No API compatibility has been provided as modules in ports
should not be using this internal API (normal PCI drivers use
pci_read_config and pci_write_config with a device_t).
2024-01-18 15:19:11 -08:00
John Baldwin
98b822fccd pci_cfgreg: Add a PCI domain argument to the low-level register API
This commit changes the API of pci_cfgreg(read|write) to add a domain
argument (referred to as a segment in ACPI parlance) (note that this
is not the same as a NUMA domain, but something PCI-specific).  This
does not yet enable access to domains other than 0, but updates the
API to support domains.

Places that use hard-coded bus/slot/function addresses have been
updated to hardcode a domain of 0.  A few places that have the PCI
domain (segment) available such as the acpi_pcib_acpi.c Host-PCI
bridge driver pass the PCI domain.

The hpt27xx(4) and hptnr(4) drivers fail to attach to a device not on
domain 0 since they provide APIs to their binary blobs that only
permit bus/slot/function addressing.

The x86 non-ACPI PCI bus drivers all hardcode a domain of 0 as they do
not support multiple domains.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D42827

(cherry picked from commit 1587a9db92c03c738bb3f0fc5874b43c961e7c99)
2024-01-18 14:44:51 -08:00
Mark Johnston
675c24cd01 x86: Make cpu_model[] public
No functional change intended.

Reviewed by:	emaste, imp, kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D43281

(cherry picked from commit d63ea03674007451682334fa3897918c5c3711a4)
2024-01-11 09:19:05 -05:00
John Baldwin
f27c8ff817 x86: Rename {stop,start}_emulating to fpu_{enable,disable}
While here, centralize the macros in <x86/fpu.h>.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D42135

(cherry picked from commit cc1cb9ea0c5607931fa9b7ecf786468d38fb8491)
2023-12-29 13:49:42 -08:00
Olivier Certner
6c59ac8c79 x86: AMD Zen2: Zenbleed chicken bit mitigation
Applies only to bare-metal Zen2 processors.  The system currently
automatically applies it to all of them.

Tunable/sysctl 'machdep.mitigations.zenbleed.enable' can be used to
forcibly enable or disable the mitigation at boot or run-time.  Possible
values are:

    0: Mitigation disabled
    1: Mitigation enabled
    2: Run the automatic determination.

Currently, value 2 is the default and has identical effect as value 1.
This might change in the future if we choose to take into account
microcode revisions in the automatic determination process.

The tunable/sysctl value is simply ignored on non-applicable CPU models,
which is useful to apply the same configuration on a set of machines
that do not all have Zen2 processors.  Trying to set it to any integer
value not listed above is silently equivalent to setting it to value 2
(automatic determination).

The current mitigation state can be queried through sysctl
'machdep.mitigations.zenbleed.state', which returns "Not applicable",
"Mitigation enabled" or "Mitigation disabled".  Note that this state is
not guaranteed to be accurate in case of intervening modifications of
the corresponding chicken bit directly via cpuctl(4) (this includes the
cpucontrol(8) utility).  Resetting the desired policy through
'machdep.mitigations.zenbleed.enable' (possibly to its current value)
will reset the hardware state and ensure that the reported state is
again coherent with it.

Reviewed by:	kib
Sponsored by:   The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D41817

(cherry picked from commit ebaea1bcd2eb0aa90937637ed305184b6fedc69b)
2023-10-10 09:34:31 -04:00
Olivier Certner
d0c0dcf9db x86: Add defines for workaround bits in AMD's MSR "Decode Configuration"
They are a bit more informative than raw hexadecimal values.

While here, sort existing defines of bits for AMD MSRs to match the address
order.

Reviewed by:	kib, emaste
Sponsored by:   The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D41816

(cherry picked from commit 125bbadf6084ac341673c9eb1979a740d3d5899a)
2023-09-25 09:45:06 -04:00
Warner Losh
71625ec9ad sys: Remove $FreeBSD$: one-line .c comment pattern
Remove /^/[*/]\s*\$FreeBSD\$.*\n/
2023-08-16 11:54:24 -06:00
Warner Losh
2ff63af9b8 sys: Remove $FreeBSD$: one-line .h pattern
Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
2023-08-16 11:54:18 -06:00
Warner Losh
95ee2897e9 sys: Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
2023-08-16 11:54:11 -06:00
Konstantin Belousov
93626d5437 tc_fill_vdso_timehands32(): fix
On 64bit, there is a 4-byte hole in struct vdso_timekeep32 after
tk_current, if the structure is not packed.  This is due to the MD
th_x86_pvc_last_systime being 64bit.

Change amd64 VDSO_TIMEHANDS_MD32 to not use uint64_t, replace it with
pair of uint32_t, as it is done for all other members.

PR:	273085
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2023-08-13 01:34:08 +03:00
Elliott Mitchell
eee6537665 x86: remove intr_bind
`intr_bind(u_int vector, u_char cpu);` looked suspicious since
everywhere else "cpu" is a u_int and >256 processors isn't unreasonable
now.  `intr_bind()` is not used anywhere in FreeBSD (now, after commit
bf42f3738087).  Time to remove.

Relnotes:	Yes
Reviewed by:	mjg
Differential Revision: https://reviews.freebsd.org/D36901
2023-08-03 17:01:56 -04:00
Mark Johnston
e60316d1ea x86: Add defines for a couple of thermal and PM bits
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-06-19 13:32:22 -04:00
Mark Johnston
9fb6718d1b smp: Dynamically allocate the stoppcbs array
This avoids bloating the kernel image when MAXCPU is large.

A follow-up patch for kgdb and other kernel debuggers is needed since
the stoppcbs symbol is now a pointer.  Bump __FreeBSD_version so that
debuggers can use osreldate to figure out how to handle stoppcbs.

PR:		269572
MFC after:	never
Reviewed by:	mjg, emaste
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39806
2023-05-25 18:09:55 -04:00
Warner Losh
b61a573019 spdx: The BSD-2-Clause-NetBSD identifier is obsolete, drop -NetBSD
The SPDX folks have obsoleted the BSD-2-Clause-NetBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix
2023-05-12 10:44:04 -06:00
Warner Losh
4d846d260e spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix
2023-05-12 10:44:03 -06:00
Dimitry Andric
bab8274c09 Use bool for one-bit wide bit-fields
A signed one-bit wide bit-field can take only the values 0 and -1. Clang
16 introduced a warning that "implicit truncation from 'int' to a
one-bit wide bit-field changes value from 1 to -1". Fix the warnings by
using C99 bool.

Reported by:	Clang 16
Reviewed by:	emaste, jhb
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D39705
2023-04-25 19:26:03 +02:00
Dmitry Chagin
de4da6cd04 x86: Move i386 timerreg.h to x86
Reviewed by:		emaste, jhb
Differential Revision:	https://reviews.freebsd.org/D39656
MFC after:		1 month
2023-04-20 19:42:59 +03:00
Dmitry Chagin
d1f4c44aa8 x86: Move i386 ppireg.h to x86
Differential Revision:	https://reviews.freebsd.org/D39655
MFC after:		1 month
2023-04-20 19:42:59 +03:00
Elliott Mitchell
6699c22c1c xen/intr: move interrupt allocation/release to architecture
Simply moving the interrupt allocation and release functions into files
which belong to the architecture.  Since x86 interrupt handling is quite
distinct from other architectures, this is a crucial necessary step.

Identifying the border between x86 and architecture-independent is
actually quite tricky.  Similarly, getting the prototypes for the
border right is also quite tricky.

Inspired by the work of Julien Grall <julien@xen.org>,
2015-10-20 09:14:56, but heavily adjusted.

Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D30936
2023-04-14 15:58:56 +02:00
Julien Grall
2d795ab1ea xen/intr: move x86 PIC interface to xen_arch_intr.c, introduce wrappers
The x86 PIC interface is very much x86-specific and not used by other
architectures.  Since most of xen_intr.c can be shared with other
architectures, the PIC interface needs to be broken off.

Introduce wrappers for calls into the architecture-dependent interrupt
layer.  All architectures need roughly the same functionality, but the
interface is slightly different between architectures.  Due to the
wrappers being so thin, all of them are implemented as inline in
arch-intr.h.

The original implementation was done by Julien Grall in 2015, but this
has required major updating.

Removal of PVHv1 meant substantial portions disappeared.  The original
implementation took care of moving interrupt allocation to
xen_arch_intr.c, but this has required massive rework and was broken
off.

In the original implementation the wrappers were normal functions.  Some
had empty stubs in xen_intr.c and were removed.

Reviewed by: royger
Submitted by: Elliott Mitchell <ehem+freebsd@m5p.com>
Original implementation: Julien Grall <julien@xen.org>, 2015-10-20 09:14:56
Differential Revision: https://reviews.freebsd.org/D30909
2023-04-14 15:58:56 +02:00
Julien Grall
ab7ce14b1d xen/intr: introduce dev/xen/bus/intr-internal.h
Move the xenisrc structure which needs to be shared between the core Xen
interrupt code and architecture-dependent code into a separate header.  A
similar situation exists for the NR_EVENT_CHANNELS constant.

Turn xi_intsrc into a type definition named xi_arch to reflect the new
purpose of being an architectural variable for the interrupt source.

This was originally implemented by Julien Grall, but has been heavily
modified.  The core side was renamed "intr-internal.h" and is #include'd
by "arch-intr.h" instead of the other way around.  This allows the
architecture to add function definitions which use struct xenisrc.

The original version only moved xi_intsrc into xen_arch_isrc_t.  Moving
xi_vector was done by the submitter.

The submitter had also moved xi_activehi and xi_edgetrigger into
xen_arch_isrc_t.  Those disappeared with the removal of PVHv1 support.

Copyright note.  The current xenisrc structure was introduced at
76acc41fb7 by Justin T. Gibbs.  Traces remain, but the strength of
Copyright claims from before 2013 seem pretty weak.

Reviewed by: royger
Submitted by: Elliott Mitchell <ehem+freebsd@m5p.com>, 2021-03-17 19:09:01
Original implementation: Julien Grall <julien@xen.org>, 2015-10-20 09:14:56
Differential Revision: https://reviews.freebsd.org/D30648
[royger]
 - Adjust some line lengths
 - Fix comment about NR_EVENT_CHANNELS after movement.
 - Use #include instead of symlinks.
2023-04-14 15:58:53 +02:00
Julien Grall
28a78d860e xen: introduce XEN_CPUID_TO_VCPUID()/XEN_VCPUID()
Part of the series for allowing FreeBSD/ARM to run on Xen.  On ARM the
function is a trivial pass-through, other architectures need distinct
implementations.

While implementing XEN_VCPUID() as a call to XEN_CPUID_TO_VCPUID()
works, that involves multiple accesses to the PCPU region.  As such make
this a distinct macro.  Only callers in machine independent code have
been switched.

Add a wrapper for the x86 PIC interface to use matching the old
prototype.

Partially inspired by the work of Julien Grall <julien@xen.org>,
2015-08-01 09:45:06, but XEN_VCPUID() was redone by Elliott Mitchell on
2022-06-13 12:51:57.

Reviewed by: royger
Submitted by: Elliott Mitchell <ehem+freebsd@m5p.com>
Original implementation: Julien Grall <julien@xen.org>, 2014-04-19 08:57:40
Original implementation: Julien Grall <julien@xen.org>, 2014-04-19 14:32:01
Differential Revision: https://reviews.freebsd.org/D29404
2023-04-14 15:58:46 +02:00
Brooks Davis
eb232cffc9 amd64: reduce header pollution in _stdint.h
In 38d1ac34ff SIGATOMIC_{MIN,MAX} were
defined in terms of LONG_{MIN,MAX}.  Later, they were switched to
__LONG_{MIN,MAX} in 78fe75bc28 where an
include of machine/_limits.h was added.  Switch to using fixed width
INT64_{MIN,MAX} and remove the header pollution.

No functional change.

Reviewed by:	theraven, emaste
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D39196
2023-03-22 16:23:57 +00:00
Mark Johnston
2bed14192c pvclock: Export a vDSO page even without rdtscp available
When the cycle counter is "stable", i.e., synchronized across vCPUs by
the hypervisor, userspace can use a serialized rdtsc instead of relying
on rdtscp, just like the kernel timecounter does.  This can be useful
for performance in guests where the hypervisor hides rdtscp for some
reason.

To avoid breaking compatibility with older userspace which expects
rdtscp to be usable when pvclock exports timekeeping info, hide this
feature behind a sysctl.

Reviewed by:	kib
Tested by:	Shrikanth R Kamath <kshrikanth@juniper.net>
MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D38342
2023-02-03 11:48:25 -05:00
Dmitry Chagin
5c32146723 amd64: Eliminate write only cpu_fxsr.
Reviewed by:		kib
Differential Revision:	https://reviews.freebsd.org/D38289
MFC after:		1 week
2023-02-01 18:17:06 +03:00
Konstantin Belousov
11989314dc x86: add more definitions for XCR0 bits
This covers all currently defined bits, adding PKRU and TILE.

Reviewed by:	jhb, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38219
2023-01-27 19:44:49 +02:00
Konstantin Belousov
45ac7755a7 amd64: identify small cores
Reviewed by:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37770
2023-01-01 00:09:45 +02:00
Warner Losh
1d21f64149 bhyve: Implement MSR_MISC_FEATURES_ENABLES
Linux reads MISC_FEATURES_ENABLES to manage the CPUID faulting feature
(undocumented in the Intel SDM, but documented in 323850-004 (Intel
Virtualization Technology FlexMigration Application Note). Since bhyve
doesn't emulate this feature, we always return 0. Neither does bhyve
support the MONITOR/MWAIT fault bit also in this MSR (which is
documented in the sdm), so always return 0.

Sponsored by:		Netflix
Reviewed by:		jhb
Differential Revision:	https://reviews.freebsd.org/D36602
2022-10-27 11:34:41 -06:00
Ed Maste
c8113dad7e Increase MAX_APIC_ID safeguard to 0x800
MAX_APIC_ID must be at least twice MAXCPU.  Increase it to 0x800 so that
it is possible to set MAXCPU to 512 or 1024 in a custom kernel config
file.

Note that increasing this limit does not itself cause any allocations
to be larger; it just allows madt_parse_cpu() to process higher APIC
IDs.

APIC IDs may be sparse and so we can waste memory.  This is independent
of this change, but becomes more of an issue as the maximum APIC ID
grows.  This should be addressed with future work.

Reviewed by:	royger
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D37067
2022-10-27 12:33:34 -04:00
Konstantin Belousov
829145388b x86/include/elf.h: make inclusion blocks for elf32.h and elf64.h similar
They were copy-pasted when x86/include/elf.h file was merged from its
i386 and amd64 counterparts.  Having the text around inclusions
significantly different is somewhat confusing.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37085
2022-10-25 19:00:44 +03:00
Konstantin Belousov
5f00525dfc i386: move hard-coded load address for PIE below default linker base
both for i386 native and compat32 amd64.  We know the ld-elf.so.1 size
in advance, it fits there.  Trying to push it up after the end of a
binary cannot work reliably and eventually fail for large binaries.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37085
2022-10-25 19:00:44 +03:00
Konstantin Belousov
fd25c62278 i386: check that trap() and syscall() run on the thread kstack
and not on the trampoline stack.  This is a useful way to ensure that
we did not enabled interrupts while on user %cr3 or trampoline stack.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2022-09-14 18:46:32 +03:00
Dmitry Chagin
050f5a8405 amd64: Reload CPU ext features after resume or cr4 changes
Reviewed by:		kib
Differential revision:	https://reviews.freebsd.org/D35555
MFC after:		2 weeks
2022-06-29 10:34:43 +03:00
Roger Pau Monné
091febc04a xen/blkback: do not use x86 CPUID in generic code
Move checker for whether Xen creates IOMMU mappings for foreign pages
into a helper that's defined in arch-specific code.

Reported by: Elliott Mitchell <ehem+freebsd@m5p.com>
Fixes: 1d528f95e8 ('xen/blkback: remove bounce buffering mode')
Sponsored by: Citrix Systems R&D
2022-06-28 09:51:57 +02:00
Dmitry Chagin
fe2c9f83a6 Remove dead code.
is_physical_memory() dead since 235a54de.

Reviewed by:		markj
Differential revision:	https://reviews.freebsd.org/D35056
MFC after:		2 weeks
2022-04-26 19:40:59 +03:00
John Baldwin
3d6f4411e4 Remove checks for <sys/cdefs.h> being included.
These files no longer depend on the macros required when these checks
were added.

PR:		263102 (exp-run)
Reviewed by:	brooks, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D34804
2022-04-12 10:06:18 -07:00