A bug in release/packages/generate-ucl.sh causes package dependencies
(other than shlib depends) to not be generated correctly, meaning
packages are missing their dependencies.
generate-ucl.sh creates the UCL file by:
1. copying ${uclsource} (template.ucl) to ${uclfile}
2. appending dependencies to ${uclfile}
3. calling generate-ucl.lua on ${uclsource} to create ${uclfile}
This breaks because the dependencies added in step 2 are overwritten in
step 3.
Fix this by calling generate-ucl.lua with ${uclfile} as both the input
and output file, so anything we added to ${uclfile} is preserved.
PR: 286551
Reviewed by: des, imp
Approved by: re (cperciva)
Approved by: kevans (mentor, stable/14 mfc)
Pull Request: https://github.com/freebsd/freebsd-src/pull/1203
(cherry picked from commit e05b6502f9ca36860236644884f12eff8e3243c4)
(cherry picked from commit fb6b1a797349992d8f166a2724eab2caf59abe28)
Early versions of Xen, including those used in the early days of EC2,
had a bug in their UART emulation whereby the TX FIFO wouldn't send the
expected interrupt when emptying; as a result, FreeBSD would write 16
characters to the serial console and then stop because we thought the
FIFO was forever full.
In 2013 (1c60b24baa) I added a loader tunable "hw.broken_txfifo"
which spinwaits for the FIFO TX rather than relying on the interrupt,
and enabled this in loader.conf in EC2 images.
A decade later, this workaround is almost certainly no longer needed in
EC2 -- most instances don't run Xen, and the bug was long since fixed
in Xen anyway -- but we've been holding on to the workaround "just in
case". Unfortunately, the spinwait behaviour is causing latency spikes
and triggering warnings from the ena(4) driver.
This commit removes the hw.broken_txfifo setting from loader.conf in
EC2 images, but leaves the loader tunable and associated code, since it
has been necessary in some other environments. (It seems that the TX
FIFO missing-interrupts bug has been independently written at least
three times!)
Approved by: re (cperciva)
MFC after: 1 minute
Sponsored by: Amazon
(cherry picked from commit 9a685c09f06a55b18589d75f9307563d84a17fa9)
(cherry picked from commit eadda156a50d3487ec1e6fc78f6cfe2df42448fa)
The X.Y-RELEASE DVDs use a quarterly package set which was built on
X.(Y-1)-RELEASE, and those kernel modules are never going to be
useful.
Approved by: re (cperciva)
MFC after: 30 seconds
Sponsored by: Amazon
(cherry picked from commit 94e44a074e5212cc3459e360a9de55500f7c41d0)
(cherry picked from commit 1ecfdc009172491a133d596332a9aaf6851a49c1)
The "kde5" package no longer exists; KDE goes to 6.
Note: Depending on the size of 13.5-BETA3 DVD images, KDE might end
up being removed from this list in the near future.
Approved by: re (cperciva)
With hat: re@
MFC after: 30 seconds
Sponsored by: Amazon
(cherry picked from commit 0d7b98c06c5ec9638020844ee460af075cfc6e54)
(cherry picked from commit 1c2e947d272ea42dbf7416d867101fe5164ffc6d)
Update from PRERELEASE to BETA1
Switch pkg(8) configuration to use the quarterly repository
Bump __FreeBSD_version
Approved by: re (implicit)
Sponsored by: Amazon
The runtime name is taken from the main pkg-base package that this
image is built off.
Sponsored by: SkunkWerks, GmbH
MFC after: 3 days
Reviewed by: dfr, emaste
Differential Revision: https://reviews.freebsd.org/D50043
(cherry picked from commit a0165254bfeb5e310f92d4e0c88fcb5c6ea802bf)
Having this enabled on Graviton systems prior to Graviton 4 results in
a resource leak and a kernel panic after repeated hotplug/unplug.
MFC after: 3 days
Sponsored by: Amazon
(cherry picked from commit ce9a34b1614e37dc3f8763586448063408c7bf16)
We need to pass this to cloudware builds, not just VM builds.
MFC after: 4 days
Fixes: 81ca663642ef ("release: Pass PKG_INSTALL_EPOCH to vmimage.subr")
Sponsored by: Amazon
(cherry picked from commit d14036ea424d5aa3eee20cc6f0b5d7117cc3931b)
This value, if not already set, comes from the timestamp of the most
recent git commit (which is now also available in src/release code as
GITEPOCH) or 0 if git is not installed.
This should allow /var/db/pkg/local.sqlite to be reproducible in VM
images which have packages installed (e.g. cloudware).
Reviewed by: emaste, bapt
MFC after: 5 days
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D49760
(cherry picked from commit 81ca663642ef1ed5111a88d2e9102f6788fab407)
Starting in 2015 I have published "AMI Builder AMIs" for FreeBSD/EC2:
These boot into a memory disk, extract a "clean" copy of FreeBSD onto
the root disk, mount it at /mnt, and allow the user to SSH in to make
customizations before creating a new AMI from the "running" instance
(in fact, from the FreeBSD installation which is not running but is
mounted on /mnt).
This provides a much cleaner mechanism for building customized FreeBSD
AMIs than the traditional Linux approach of "launch an EC2 instance,
SSH in and configure it, then try to wipe logs and credentials before
creating an AMI"; and it's easier than building a customized AMI ab
initio by modifying the FreeBSD release-building code.
This commit brings that functionality into the FreeBSD src tree and
into the collection of images built by the release engineering team:
The EC2 "BUILDER" flavour AMI is essentially a "SMALL" flavour AMI with
a compressed "BASE" flavour disk image, plus an init script which
juggles disks around (rerooting into a memory disk and extracting the
"BASE" image onto disk).
Polished by: bz, emaste
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D49930
(cherry picked from commit 58426589030308cd632477d328b9536b1634c54d)
Since PCIe device detaching is done via API, there is no opportunity
to "press the attention button a second time" and thus the 5 second
timeout mandated by PCIe serves no purpose.
MFC after: 2 weeks
Sponsored by: Amazon
(cherry picked from commit 80febaa1714e897d67a8217edc305a0e81498a6f)
On some EC2 instances, there is a race between removing a device from
the system and making the PCI bus stop reporting the presence of the
device. As a result, a PCI BUS_RESCAN performed immediately after
the _EJ0 method returns "sees" the device which is being ejected, which
then causes problems later (e.g. we won't recognize a new device being
plugged into that slot because we never knew it was vacant).
On other operating systems the bus is synchronously marked as needing
to be rescanned but the rescan does not occur until O(1) seconds later.
Create a new ACPI_Q_DELAY_BEFORE_EJECT_RESCAN quirk and set it in EC2
AMIs, and add a 10 ms DELAY between _EJ0 and BUS_RESCAN when tht quirk
is set.
Reviewed by: jhb
MFC after: 1 month
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D49252
(cherry picked from commit 55c3348ed78fb1d0891e8bb51a8948f95da3560b)
- requires base tar & flua, qemu-tools & curl from ports
- set ORACLE_PAR_URL to upload to local file:/// dir or cloud buckets
Reviewed by: emaste
Approved by: cperciva
Differential Revision: https://reviews.freebsd.org/D48382
Sponsored by: SkunkWerks, GmbH
(cherry picked from commit 0ce9a414adc33af29607adbd81e0760e014fcd76)
In order to signal to Graviton [123] systems that a device is ready
to be "ejected" (after a detach request is made via the EC2 API) we
need to set PCIM_PSTAT_PME to 1 and PCIM_PSTAT_PMEENABLE to 0. We are
not aware of any rationale for this requirement beyond "another OS
kernel happens to do this", i.e. this is effectively bug-for-bug
compatibility.
Arguably this should be done by the ACPI _EJ0 method on these systems,
but it is not.
Create a new ACPI_Q_CLEAR_PME_ON_DETACH quirk and set it in EC2 AMIs,
and add the PCI register write to acpi_pci_device_notify_handler when
that quirk is set.
Reviewed by: jhb
MFC after: 1 month
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D49146
(cherry picked from commit d70bac252d30adec4feba0c866dabe2c16a756d9)
This handles copying in install-boot.sh and bsdinstall's bootconfig.
install-boot.sh:
make_esp_file now optionally takes extra arguments so it can copy
multiple files. This is used by the amd64 release scripts.
make_esp_device also takes an extra optional argument for efibootname.
This is currently unused, but it can be used in the future to do
something like:
make_esp_device loader.efi bootx64
make_esp_device loader_ia32.efi bootia32
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1098
(cherry picked from commit 599273f942b8dc6f957487bb28f36694dab9dad2)
This avoids the need for buildah and skopeo for building releases.
Reviewed by: cpersiva
MFC after: 1 day
Differential Revision: https://reviews.freebsd.org/D48574
(cherry picked from commit e8a5b9fd73f4f437a03c85e7644daa55652e224b)
A new filesystems category was created in the ports tree, with 142
filesystem related ports moved to there, some of them renamed.
Update all references in the src tree to the new locations.
PR: 283881
Fixes: ports:6e2da9672f79f44 (filesystems: add new category)
MFC after: 1 month
Reviewed by: fuz, mhorne, bapt
Accepted by: mhorne (mentor)
Differential Revision: https://reviews.freebsd.org/D48406
(cherry picked from commit 066ef2aec187ae93a9df01d25fa8e47d67ff972b)
pkg used to store copies of upstream repository databases in
/var/db/pkg/repo-*.sqlite. About a year ago this was moved to
/var/db/pkg/repos/*/, resulting in FreeBSD cloud images no longer
having those (unhelpful since they'll be long out of date before
the cloud images are launched) databases removed.
Remove the correct location, and hope that future pkg updates don't
break the base system again.
Sponsored by: Amazon
(cherry picked from commit 078e8b34b13d6d0663661542eeac9007806fccdc)
While we're here, send the etcupdate log from generating base.txz
to stdout instead of /dev/null (see e972e408d19a) as well.
Reviewed by: emaste, gordon, jrtc27
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D48719
(cherry picked from commit ace4637ee0c666c6f5a1c664f8956f7302234edd)
All of the BE datasets need to be set canmount=noauto so that creating
a new BE and switching to it can actually work. With the current setup,
the zfs rc script will mount the `default` BE over whichever new BE is
activated once it runs.
Reported by: andrew
Reviewed by: andrew, re (cperciva), imp, markj
(cherry picked from commit 636d377264f51e3dd33bd7f33ebf03e2e148d40d)
We publish three sets of VM images on download.freebsd.org:
* Generic VM images
* BASIC-CLOUDINIT VM images
* BASIC-CI VM images
Of these, the first had names like
FreeBSD-14.2-RELEASE-amd64-ufs.raw.xz,
the second had names like
FreeBSD-14.2-RELEASE-amd64-BASIC-CLOUDINIT.ufs.qcow2.xz,
and the third had names like
FreeBSD-14.2-RELEASE-amd64-BASIC-CI.ufs.raw.xz
but were listed in the CHECKSUM files with names like
FreeBSD-14.2-RELEASE-amd64-BASIC-CI-ufs.raw.xz
Standardize these to consistently use a hyphen before the filesystem
type rather than a period.
Note: On FreeBSD 14 we ship images without the -${FS} component in
their names; these are hardlinks to the -ufs images for compatibility
purposes. On FreeBSD 13 we only have UFS images and don't include the
filesystem in the name.
MFC after: 1 week
Reported by: jmg
Sponsored by: Amazon
(cherry picked from commit 35623ccc375846648f4dcdfc47c7e84fc9db4852)
On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.
Reported by: Weekly snapshot builds
Fixes: 7e2996c1f5b4 ("release: install wireless firmware onto disc1 and dvd")
MFC after: 1 minute
(cherry picked from commit e8263ace39c8ecf11233c0a10d0b1839e6813046)
For "release" builds (as opposed to "snapshot" builds -- in this
context BETAs and RCs are "releases") ${SNAP_SUFFIX} is empty; but it
stuck into some ociimages filenames via a copy-and-paste error.
The final filenames on the download mirrors were not affected, so
this does not need to be merged to releng/14.2.
MFC after: 3 days
Sponsored by: Amazon
(cherry picked from commit d54fdd16636bbe6a38a5b0a636dd83cf64aa6e06)
Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install the firmware packages onto disc1 (and memstick) and dvd
if built on FreeBSD and NOPKG is not defined (to not break cross-builds
from Linux or OSX and to allow people to opt-out).
Sponsored by: The FreeBSD Foundation
Submitted by: cperciva (the orig. commands and where to place them)
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D47407
(cherry picked from commit 7e2996c1f5b4e684cae40c2418b68061df9997d9)
These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .txz files.
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon
(cherry picked from commit 976e1d36be688166e140cbaded01d433c61e6d47)
We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .txz files. Grep those out from the distribution
lists.
A better long-term fix would probably be to generate an explicit list
of the .txz files we want rather than using an overbroad glob and
filtering out the files we *don't* want.
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon
(cherry picked from commit 47866cdcc7278faa27250dcea16374231c676050)
Add the flavored port for all the
supported wireless drivers we recently added support for.
Sponsored by: The FreeBSD Foundation
Pointed out by: cperciva (as part of the set of changes)
Reviewed by: cperciva (#releng)
Differential Revision: https://reviews.freebsd.org/D47406
(cherry picked from commit 2483a2d36803896d27861c03e1f53dd9546b1e73)
If WITH_OCIIMAGES is set, "make ftp-stage" will now publish those bits
into a directory under /OCI-IMAGES/, similar to how we publish CI and
VM images.
MFC after: 1 minute
(cherry picked from commit e95e3e6eb7ef723aa5dd08de48a9ad3fec647c7d)
This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell image also contains pkg-bootstrap and can be easily
extended by installing packages (including pkgbase packages).
Reviewed by: dch, cpersiva, jlduran, zlei
Differential Revision: https://reviews.freebsd.org/D46759
MFC after: 2 days
(cherry picked from commit d03c82c28da86e0812b98b051d24ae5980804ad7)
In 2016, commit 8834318685 reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v' should have been 'grep -vE').
FreeBSD Update builds later began to depend on this, and in any case
since DVD images are intended as "include everything" images there's
no point excluding those; so remove the (broken) filtering from DVD
image builds.
Sponsored by: Amazon
MFC after: 2 days
(cherry picked from commit cdfd0600dc8882f0a0d0e6d9a1cdcf926edba6d6)
AWS Graviton [1234] systems have a bug in their ACPI where they mark
the PL061's GPIO pins as needing to be configured in PullUp mode (in
fact the PL061 has no pullup/pulldown resistors); this flag needs to
be removed in order for _AEI objects to be handled on these systems.
Reviewed by: Ali Saidi
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D47239
(cherry picked from commit 2f3f867ac6dd7ff3769366b828b79c44b38828e1)
While OK for 14.x, both 13.x and 15.0 are failing in snapshot
builds.
Reported by: cperciva
Sponsored by: SkunkWerks, GmbH
Reviewed by: emaste
Approved by: cperciva
Differential Revision: https://reviews.freebsd.org/D47285
(cherry picked from commit 89311e6f987ebb1a968eee6fe594b11bfb91977a)
- use raw image disk type and enable zfs, this yields smaller
images for upload after using native qcow2 + zstd compression
Reviewed by: lwhsu, emaste
Differential Revision: https://reviews.freebsd.org/D47055
MFC after: 3 days
Approved by: emaste
(cherry picked from commit c611041660845e8ee8bf3009b32327d1f783a47b)
This allows future releng tooling to use OCI for the industry
standard Open Container Initiative tooling, reducing potential
for confusion.
Approved by: cperciva
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D46975
MFC after: 3 days
Sponsored by: SkunkWerks, GmbH
(cherry picked from commit 1be84d745bcc11b766ac1fa03afd2c8ea8771737)
- Add the firstboot-freebsd-update package, as long as we do not have
pkgbase, this is needed
- Support SLAAC by default to complement DHCPv4 (use SYNCDHP instead)
Signed-off-by: Jose Luis Duran <jlduran@gmail.com>
(cherry picked from commit 120740221fd4a4577e63e6c279f9873cabe449d0)
These are the same as the standard "base" images except:
* They don't have kernel or world debug symbols,
* They don't have FreeBSD tests,
* They don't have 32-bit libraries,
* They don't have LLDB,
* They don't have the Amazon SSM Agent pre-installed,
* They don't default to installing the awscli at first boot.
This reduces the amount of disk space in use when the EC2 instance
finishes booting from ~5 GB to ~1 GB.
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D46509
(cherry picked from commit 647299caa06e38622dc05a4358f9407b62a9bdda)
Move it from VM_EXTRA_PACKAGES in ec2.conf to VM_EXTRA_PACKAGES in
ec2-{base,cloud-init}.conf
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D46508
(cherry picked from commit 40ff0753abb9c00b5f1e5df9ea00c9fdded55ac4)
Having the "base" FreeBSD network configuration (aka. what is used
when not using cloud-init) in ec2.conf will allow us to reuse it in
other AMIs.
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D46507
(cherry picked from commit f961ddb28d6909d4c67e3e0b6b60498bbcbf64cb)
This makes it possible for a VM build configuration file to pass
options to make installworld/installkernel/distribution, e.g.
WITHOUT_DEBUG_FILES=YES in order to produce smaller images.
Note that these options are only applied at install time, not at
build time (since the same build is installed into many different
VM images), so not all src.conf options are usable here.
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D46506
(cherry picked from commit 81d3df02bcf5ed6e41a91fd9fbc3b81cfe809ff3)
Allow the cloudware *_FLAVOURS and *_FSLIST values to be overridden
at the command line, to assist users who want to e.g. build only one
of the many EC2 AMIs available.
(cherry picked from commit 863975b6840b2833b0f772648ba2532806ffece8)
