While we're here, send the etcupdate log from generating base.txz
to stdout instead of /dev/null (see e972e408d19a) as well.
Reviewed by: emaste, gordon, jrtc27
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D48719
(cherry picked from commit ace4637ee0c666c6f5a1c664f8956f7302234edd)
On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.
Reported by: Weekly snapshot builds
Fixes: 7e2996c1f5b4 ("release: install wireless firmware onto disc1 and dvd")
MFC after: 1 minute
(cherry picked from commit e8263ace39c8ecf11233c0a10d0b1839e6813046)
Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install the firmware packages onto disc1 (and memstick) and dvd
if built on FreeBSD and NOPKG is not defined (to not break cross-builds
from Linux or OSX and to allow people to opt-out).
Sponsored by: The FreeBSD Foundation
Submitted by: cperciva (the orig. commands and where to place them)
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D47407
(cherry picked from commit 7e2996c1f5b4e684cae40c2418b68061df9997d9)
These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .txz files.
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon
(cherry picked from commit 976e1d36be688166e140cbaded01d433c61e6d47)
We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .txz files. Grep those out from the distribution
lists.
A better long-term fix would probably be to generate an explicit list
of the .txz files we want rather than using an overbroad glob and
filtering out the files we *don't* want.
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon
(cherry picked from commit 47866cdcc7278faa27250dcea16374231c676050)
This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell image also contains pkg-bootstrap and can be easily
extended by installing packages (including pkgbase packages).
Reviewed by: dch, cpersiva, jlduran, zlei
Differential Revision: https://reviews.freebsd.org/D46759
MFC after: 2 days
(cherry picked from commit d03c82c28da86e0812b98b051d24ae5980804ad7)
In 2016, commit 8834318685 reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v' should have been 'grep -vE').
FreeBSD Update builds later began to depend on this, and in any case
since DVD images are intended as "include everything" images there's
no point excluding those; so remove the (broken) filtering from DVD
image builds.
Sponsored by: Amazon
MFC after: 2 days
(cherry picked from commit cdfd0600dc8882f0a0d0e6d9a1cdcf926edba6d6)
Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each run.
Differential Revision: https://reviews.freebsd.org/D46317
(cherry picked from commit e972e408d19a58c4f2855eface487d06ef73b799)
While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- which are published via the FTP mirrors. We have
been handling these using a manual and error-prone process of copying
them from the internal build directories into the FTP staging tree,
but they should be handled using the "make install ftp-stage"
mechanism which is used for all the other published images.
This commit 1. Adds a new 'make cloud-install' target which is run as
part of 'make install' and installs BASIC-CI and BASIC-CLOUDINIT images,
and 2. Extends 'make ftp-stage' in Makefile.mirrors to copy those bits
into the FTP staging area.
As a side benefit, this will also make BASIC-CI and BASIC-CLOUDINIT
images available for arm64; until now they have only been available
for amd64.
MFC after: 1 week
(cherry picked from commit 4771c2e9d1c7db949a82dfe4f2b9878bb358a50e)
This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in weekly snapshots and thereby slipped past testing).
MFC after: 3 days
(cherry picked from commit 567cbd8ff1c4edc1f2015ec4f93608aaf5a3c546)
The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, and the latter is bootonly.iso and mini-memstick.img.
Unfortunately the scripts which package a directory tree into an ISO
or memory stick image also modify the directory, for example to
create an appropriate /etc/fstab file; so creating two images at the
same time breaks.
Resolve this by copying disc1 to disc1-disc1 and disc1-memstick,
and copying bootonly to bootonly-bootonly and bootonly-memstick,
before using those directories for constructing the ISO+memstick
images.
MFC after: 1 week
(cherry picked from commit a1bff97300abe4b46528357e39c83da764df1fd8)
With these as a single make command, `make -j` breaks when it tries to
package up a kernel which hasn't been distributed yet.
MFC after: 1 week
(cherry picked from commit 65c603ed65c700d6eacdf9e1e94dc42fd70fb0b7)
Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.
MFC after: 1 week
(cherry picked from commit 93e4813b9a157396ffbcf8456ca4f5c25599807d)
Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packages in the installer. Rename EXTRA_PACKAGES to
DISTRIBUTIONS, and move base.txz and kernel.txz to that list.
This introduces no functional change but is a small cleanup in advance
of some pkgbase experimentation.
Reviewed by: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43544
(cherry picked from commit 2e9b8c8c1e3938335e17e63049887f8e8c6e11bc)
This spits out errors but seemingly isn't actually fatal, so was missed.
Fixes: 6853d893c7 ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists")
This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise hard to distinguish.
It also provides a more user-friendly, and more accurate, prompt.
Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36419
This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/current.
Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35858
This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary to
use for its own builds.
Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35855
This is in preparation for non-FreeBSD and -DNO_ROOT builds. On
non-FreeBSD there is no host etcupdate to use, and -DNO_ROOT will
require additional flags that may not be supported by the host's
etcupdate when building on FreeBSD. Moreover, there's no guarantee
anyway that the host's etcupdate is quite right for the current tree;
upgrading from source only requires that the host's is good enough for
-p which just manually copies master.passwd and group, the rest of the
upgrade is done post-installworld. For example, should a new set of
autogenerated files be added that etcupdate is taught about, the host
won't know about them and so the bootstrapped current tree will
incorrectly contain them, leading to spurious diffs on the installed
system.
Reviewed by: gjb, delphij
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35853
Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
some other commands allows it to be overridden to use bsdtar in such
cases.
Reviewed by: brooks, delphij, gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35813
This requires a bunch of METALOG mangling to include the files we inject
into the tree. The mkisoimages.sh and make-memstick.sh scripts are now
called with the current directory inside the tree so that the relative
paths in the METALOG match up with the current directory. The scripts do
not require this when not using a METALOG, but for simplicity we always
do so. The Makefile mangles the real METALOG created from the install,
as those files are shared across all uses of the tree, but the shell
scripts create a temporary copy of the METALOG that they mangle as their
tree modifications are specific to that image. We also need to pass -D
to makefs to turn any duplicate METALOG entry errors into warnings, as
we have many (harmless) instances of those.
Whilst dvd1.iso should work, the !NOPKG code will need more work to
support this.
All media will also lack mergemaster and etcupdate trees, since more
work is needed to add -DNO_ROOT modes to them. Users of install media
built this way will have to manually bootstrap them.
Reviewed by: brooks, gjb
Differential Revision: https://reviews.freebsd.org/D33999
While here, fix all links to older en_US.ISO8859-1 documentation
in the src/ tree.
PR: 255026
Reported by: Michael Büker <freebsd@michael-bueker.de>
Reviewed by: dbaio
Approved by: blackend (mentor), re (gjb)
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D30265
To support -DNO_ROOT work. The top-level installworld target creates a
new METALOG starting with `#mtree 2.0` so it needs to be first, to avoid
overwriting installkernel METALOG entries.
Reviewed by: gjb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29582
Following 7b1d1a1658, the structure
for the reldoc target has significantly changed as result of the
ASCIIDoctor/Hugo migration. As the release notes related files
on the installation medium are inherently out of date, purge them
entirely.
Discussed within: re, doceng
No objection: re (silence), doceng (silence)
Timeout: 2 weeks
MFC after: 1 week
MFC to: stable/13, stable/12, and stable/11 only
Sponsored by: Rubicon Communications, LLC ("Netgate")
The 'reldoc' target includes release-related documentation on
installation medium. Since the switch from XML to ASCIIDoctor,
the file locations have moved, and it will take some time to sort
out how this target should work now.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
For platforms that don't have any of the memstick, cdrom, or dvdrom
release images (i.e. riscv64), the release-install target will trip up
when invoking md5(1) on the non-existent image files. Skipping this
allows the install to complete successfully.
- New file. Adds logic to search for the git binary, as well
as determining the branch and revision, used in various
places.
Makefile:
- Remove searching for the svn{,lite} binary.
Makefile.ec2:
- Reduce duplicated code, removing searching for the svn{,lite}
binary, in addition to EC2_SVN{BRANCH,REV}.
- Rename EC2_SVN* with GIT* for consistency.
Makefile.mirrors:
- Remove the SRCBRANCH declaration, replaced with the exported
GITBRANCH variable.
- Update _SNAP_SUFFIX from SVNREVISION to GITREV, and remove
the leading 'r' from it, since it will break git hashes.
- Remove yet another instance of duplicated code to search for
the svn{,version}lite binary.
Sponsored by: Rubicon Communications, LLC (netgate.com)
autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems. As of r296194 the amd man page claimed that it
is deprecated. Remove it from base now; the sysutils/am-utils port is
still available if necessary.
Discussed with: cy
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
src branch is used when generating the hardware.html page.
MFC after: 3 days
MFC before: 12.0-BETA3
MFC with: r339873
Sponsored by: The FreeBSD Foundation
This feature is disabled on install media as these generally won't have any
interesting kernels to be listed other than the default kernel, so the
potential performance penalty in these situations likely isn't worth it.
Approved by: re (kib)
target.
The doc/share/mk/doc.commands.mk sets SVN to /usr/local/bin/svn
by default, which is not necessarily installed by the documentation
project textproc/docproj port.
Ensure SVN can be evaluated properly to include the hardware pages
by iterating through /usr/local/bin and /usr/bin and looking for
both svn and svnlite binaries, and pass the SVN variable explicitly
through env(1) in the reldoc target to avoid failures if it does not
exist.
Approved by: re (rgrimes)
Sponsored by: The FreeBSD Foundation
There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.
Sponsored by: Citrix Systems R&D
PR: 230459
Reviewed by: gjb
Differential revision: https://reviews.freebsd.org/D16638
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.
Also remove checknr(1) and colcrt(1) which are only useful with groff.
Approved by: (no objections on the mailing lists)
Notably, this removes the toolchain from the CD-ROM image (it remains on DVD
images) and pushes the CD-ROM image well below the 700 MB mark.
MFC after: 10 days
