wrap msgpack to avoid future upstream api changes making troubles
or that we would have to globally spoil our code with extra params.
make sure the packing is always with use_bin_type=False,
thus generating "old" msgpack format (as borg always did) from
bytes objects.
make sure the unpacking is always with raw=True,
thus generating bytes objects.
note:
safe unicode encoding/decoding for some kinds of data types is done in Item
class (see item.pyx), so it is enough if we care for bytes objects on the
msgpack level.
also wrap exception handling, so borg code can catch msgpack specific
exceptions even if the upstream msgpack code raises way too generic
exceptions typed Exception, TypeError or ValueError.
We use own Exception classes for this, upstream classes are deprecated
most of the files created via self.create_test_files() are not needed.
the only one is the empty file, so just create it here.
also: move repo init to after test file creation.
It causes problems with the new caching in the py37 logger module.
Removing loggerDict.clear() fixes this and makes the tests work again.
Also, it does not seem to have any negative effect, neither on py36
nor on py37.
See also: https://bugs.python.org/issue34269
this code used to live in borg.xattr and used ctypes
(and was the only ctypes-using code in borg).
the low level code now was converted to cython and
the platform code moved to platform package.
got rid of the code that tried to find the libc.
- reordered code a little, so files are touched in different order
let's see if this changes anything.
- create_regular_file: giving size AND contents is not supported
the code did not create a 4096 bytes long file, but just a 1 byte
long file. adding an assertion to avoid unsupported usage.
(cherry picked from commit 2d490aa8c8)
freebsd12 is unhappy with having to deliver 50MiB random in one go
and fails with BlockingIOError "temporary unavailable" when trying.
for test_lz4_buffer_allocation, it is good enough to fetch 5MiB and
concatenate that 10 times.
the problem was that the upper layer code did not have enough information
about the file, whether it is known or not - and thus, could not decide
correctly whether status should be M)odified or A)dded.
now, file_known_and_unchanged method returns an additional "known"
boolean to fix this.
also: add comment about files cache loading in cache_mode='r'
based on willyvmm's work in PR #3116, but some changes:
- removed any mulithreading changes
- add zstandard in setup.py install_requires
- tests
- fix: minimum compression level is 1 (not 0)
- use 3 for the default compression level
- use ID 03 00 for zstd
- only convert to bytes if we don't have bytes yet
- move zstd code so that code blocks are ordered by ID
- other cosmetic fixes
Fixes CVE-2017-15914 (affects releases 1.1.0, 1.1.1, 1.1.2,
but not 1.0.x).
Thanks to Florian Apolloner for discovering/reporting this!
Also: added tests for this.
(cherry picked from commit ea0203bb0d)
This command works similarly to "git config" - it parses repo and
cache configs to get, set, and delete values. It only works on local
repos so a malicious client can't e.g. override their storage quota
or reset the append_only flag.
Add tests for borg config
Add documentation for borg config
Change manual config edits -> borg config
There were a couple places in the documentation where it was advised
to edit the repository or cache config file, a process that is stream-
lined by borg config.
previous commit did not yet support hardlinks correctly, if the
hardlink master was excluded somehow.
added some tests for this, also refactored related tests slightly.
fixing only the (generic) slice-by-8 crc32 implementation,
it is assumed that CPUs supporting CLMUL can also efficiently
and correctly deal with unaligned accesses.
slice-by-8 is used e.g. on ARM cpus and they might not (efficiently)
support unaligned memory access, leading to bus errors or low
performance.
include item birthtime in archive, fixes#3272
* use `safe_ns` when reading birthtime into attributes
* proper order for `birthtime` in `ITEM_KEYS` list
* use `bigint` wrapper for consistency
* Add tests to verify that birthtime is normally preserved, but not preserved when `--nobirthtime` is passed to `borg create`.
