OPNsense - FreeBSD source
Find a file
Marcel Moolenaar 852a0932c1 Improve performance of mking(1) by keeping a list of "chunks" in memory,
that keeps track of a particular region of the image. In particular the
image_data() function needs to return to the caller whether a region
contains data or is all zeroes. This required reading the region from
the temporary file and comparing the bytes. When image_data() is used
multiple times for the same region, this will get painful fast.

With a chunk describing a region of the image, we now also have a way
to refer to the image provided on the command line. This means we don't
need to copy the image into a temporary file. We just keep track of the
file descriptor and offset within the source file on a per-chunk basis.

For streams (pipes, sockets, fifos, etc) we now use the temporary file
as a swap file. We read from the input file and create a chunk of type
"zeroes" for each sequence of zeroes that's a multiple of the sector
size. Otherwise, we allocte from the swap file, mmap(2) it, read into
the mmap(2)'d memory and create a chunk representing data.

For regular files, we use SEEK_HOLE and SEEK_DATA to handle sparse files
eficiently and create a chunk of type zeroes for holes and a chunk of
type data for data regions. For data regions, we still compare the bytes
we read to handle differences between a file system's block size and our
sector size.

After reading all files, image_write() is used by schemes to scribble in
the reserved sectors. Since this never amounts to much, keep this data
in memory in chunks of exactly 1 sector.

The output image is created by looking using the chunk list to find the
data and write it out to the output file. For chunks of type "zeroes"
we prefer to seek, but fall back to writing zeroes to handle pipes.
For chunks of type "file" and "memoty" we simply write.

The net effect of this is that for reasonably large images the execution
time drops from 1-2 minutes to 10-20 seconds. A typical speedup is about
5 to 8 times, depending on partition sizes, output format whether in
input files are sparse or not.

Bump version to 20141001.
2014-10-01 21:03:17 +00:00
bin Fix pkill unit tests. 2014-09-30 15:27:49 +00:00
cddl Revert r272189, the committed version was an old version and breaks build. 2014-09-27 09:39:19 +00:00
contrib Use a char * as iterator over a bounded string. 2014-09-25 22:22:57 +00:00
crypto Vendor import apr-util-1.5.4 - a minor bug fix release. 2014-09-24 18:12:23 +00:00
etc Regenerate usb.conf 2014-09-28 12:41:48 +00:00
games Update factor for changes to types in primes, which is a dependency. 2014-09-27 10:57:34 +00:00
gnu Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support 2014-10-01 08:26:51 +00:00
include Vendor import apr-util-1.5.4 - a minor bug fix release. 2014-09-24 18:12:23 +00:00
kerberos5 Fix incremental builds involving non-root users with read-only source files. 2014-09-18 14:50:21 +00:00
lib Clean up detection of hard-float ABIs. As with big-endian in r272368 we 2014-10-01 16:08:19 +00:00
libexec Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support 2014-10-01 08:26:51 +00:00
release Add ipheth(4) to the hardware notes. 2014-09-26 13:12:00 +00:00
rescue Remove duplicate prog. 2014-09-28 08:59:38 +00:00
sbin Merged from r183296. 2014-09-28 08:23:26 +00:00
secure Fix typo (LIBLDNSADD -> LIBLDNS) to fix "make checkdpadd" 2014-08-19 18:27:43 +00:00
share Revise r272363 by collapsing the tests into a for loop. 2014-10-01 20:52:08 +00:00
sys Add a sysctl, net.inet.icmp.tstamprepl, which can be used to disable replies 2014-10-01 18:07:34 +00:00
tests Fix source address selection on unbound sockets in the presence of multiple 2014-09-16 15:28:19 +00:00
tools Explicitly return None for negative event indices. Prior to this, 2014-09-30 17:19:07 +00:00
usr.bin Improve performance of mking(1) by keeping a list of "chunks" in memory, 2014-10-01 21:03:17 +00:00
usr.sbin Optimize program flow for execution speed. Also fix some more style(9) nits 2014-10-01 18:59:57 +00:00
.arcconfig Update the URL to the phabricator instance. 2014-08-14 16:25:43 +00:00
.arclint arc: add linting for python files 2014-06-02 00:21:42 +00:00
COPYRIGHT Bump copyright year. 2013-12-31 12:18:10 +00:00
LOCKS Explicitly require Security Officer's approval for kernel PRNG bits. 2013-09-17 14:19:05 +00:00
MAINTAINERS add myself as opencrypto maintainer as I'm doing some work in the 2014-08-11 19:03:20 +00:00
Makefile Revert r271159, Mis-patched the tree. 2014-09-05 14:35:34 +00:00
Makefile.inc1 Fix the TARGET_ABI value clang uses. It shpuld be gnueabi on all ARM 2014-10-01 12:47:25 +00:00
ObsoleteFiles.inc Add ObsoleteFiles.inc OLD_FILES entries for libnv 2014-10-01 01:56:51 +00:00
README Add the cddl/ directory. 2010-11-14 11:32:56 +00:00
UPDATING Import pjdfstest from ^/vendor/pjdfstest/abf03c3a47745d4521b0e4aa141317553ca48f91 2014-09-24 07:00:35 +00:00

This is the top level of the FreeBSD source directory.  This file
was last revised on:
$FreeBSD$

For copyright information, please see the file COPYRIGHT in this
directory (additional copyright information also exists for some
sources in this tree - please see the specific source directories for
more information).

The Makefile in this directory supports a number of targets for
building components (or all) of the FreeBSD source tree, the most
commonly used one being ``world'', which rebuilds and installs
everything in the FreeBSD system from the source tree except the
kernel, the kernel-modules and the contents of /etc.  The ``world''
target should only be used in cases where the source tree has not
changed from the currently running version.  See:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
for more information, including setting make(1) variables.

The ``buildkernel'' and ``installkernel'' targets build and install
the kernel and the modules (see below).  Please see the top of
the Makefile in this directory for more information on the
standard build targets and compile-time flags.

Building a kernel is a somewhat more involved process, documentation
for which can be found at:
   http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig.html
And in the config(8) man page.
Note: If you want to build and install the kernel with the
``buildkernel'' and ``installkernel'' targets, you might need to build
world before.  More information is available in the handbook.

The sample kernel configuration files reside in the sys/<arch>/conf
sub-directory (assuming that you've installed the kernel sources), the
file named GENERIC being the one used to build your initial installation
kernel.  The file NOTES contains entries and documentation for all possible
devices, not just those commonly used.  It is the successor of the ancient
LINT file, but in contrast to LINT, it is not buildable as a kernel but a
pure reference and documentation file.


Source Roadmap:
---------------
bin		System/user commands.

cddl		Various commands and libraries under the Common Development
		and Distribution License.

contrib		Packages contributed by 3rd parties.

crypto		Cryptography stuff (see crypto/README).

etc		Template files for /etc.

games		Amusements.

gnu		Various commands and libraries under the GNU Public License.
		Please see gnu/COPYING* for more information.

include		System include files.

kerberos5	Kerberos5 (Heimdal) package.

lib		System libraries.

libexec		System daemons.

release		Release building Makefile & associated tools.

rescue		Build system for statically linked /rescue utilities.

sbin		System commands.

secure		Cryptographic libraries and commands.

share		Shared resources.

sys		Kernel sources.

tools		Utilities for regression testing and miscellaneous tasks.

usr.bin		User commands.

usr.sbin	System administration commands.


For information on synchronizing your source tree with one or more of
the FreeBSD Project's development branches, please see:

  http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/synching.html