Commit graph

1760 commits

Author SHA1 Message Date
TW
7a3056092e
Merge pull request #6582 from ThomasWaldmann/deleted-key-1.2
load_key: no key is same as empty key, fixes #6441
2022-04-12 20:09:10 +02:00
Thomas Waldmann
7fc1d75cd3 load_key: no key is same as empty key, fixes #6441
when migrating from repokey to keyfile, we just store an empty key into the repo config,
because we do not have a "delete key" RPC api. thus, empty key means "there is no key".

here we fix load_key, so that it does not behave differently for no key and empty key:
in both cases, it just returns an empty value.

additionally, we strip the value we get from the config, so whitespace does not matter.

All callers now check for the repokey not being empty, otherwise RepoKeyNotFoundError
is raised.
2022-04-12 19:43:48 +02:00
TW
fddd0c93fa
Merge pull request #6581 from ThomasWaldmann/docs-max-duration-1.2
docs: improve borg check --max-duration description
2022-04-12 19:40:51 +02:00
TW
063adc1799
Merge pull request #6579 from ThomasWaldmann/compact-docs-1.2
docs: borg compact --cleanup-commits also runs a normal compaction
2022-04-12 18:53:53 +02:00
Thomas Waldmann
ec90a065fb docs: improve borg check --max-duration description 2022-04-12 18:45:03 +02:00
Thomas Waldmann
3bacf4ac39 docs: borg compact --cleanup-commits also runs a normal compaction, fixes #6324 2022-04-12 18:11:38 +02:00
Thomas Waldmann
c8f160068e check archives: improve error handling for corrupt archive metadata block
this is similar to #4777.

borg check must not crash if an archive metadata block does not decrypt.

Instead, report the archive_id, remove the archive from the manifest and skip to the next archive.
2022-04-12 17:45:48 +02:00
Thomas Waldmann
151fe5f348 check archive: make robust_iterator more robust, fixes #4777
borg check must not crash if an archive metadata chunk does not decrypt.

Instead, report the chunk and skip to the next one.
2022-04-12 17:45:17 +02:00
TW
aa55aef2b9
Merge pull request #6543 from ThomasWaldmann/fix-progress-archivename-1.2
escape % chars in archive name, fixes #6500
2022-04-07 20:22:06 +02:00
Thomas Waldmann
a4f7eeb179 escape % chars in archive name, fixes #6500
also: fix percentage format for float value.
2022-04-07 18:03:25 +02:00
TW
a7328edd58
Merge pull request #6532 from bket/1.2_fix_#2055
1.2 - fix OpenBSD symlink mode test failure (#2055)
2022-04-04 21:41:07 +02:00
Björn Ketelaars
b467d50c35 1.2 - Fix OpenBSD symlink mode test failure (#2055)
OpenBSD does not have `lchmod()` causing `os.lchmod` to be unavailable
on this platform. As a result ArchiverTestCase::test_basic_functionality
fails when run manually (#2055).

OpenBSD does have `fchmodat()`, which has a flag that makes it behave
like `lchmod()`. In Python this can be used via `os.chmod(path, mode,
follow_symlinks=False)`.

As of Python 3.3 `os.lchmod(path, mode)` is equivalent to
`os.chmod(path, mode, follow_symlinks=False)`. As such, switching to the
latter is preferred as it enables more platforms to do the right thing.
2022-04-04 19:24:28 +02:00
Thomas Waldmann
40f1f9ddee use same host regex for ssh and scp style, refactor/clean up
although bug #6526 did not show with ssh style URLs, we should
not have different regexes for the host part for ssh and scp style.

thus i extracted the host_re from both and also cleaned up a bit.
2022-04-03 21:33:26 +02:00
Thomas Waldmann
e332acb334 fix scp repo url parsing for ip v6 addrs, fixes #6526
added a negative lookahead/lookbehind to make sure an ipv6 addr
(enclosed in square brackets) does not get badly matched by the
regex part intended for hostnames and ipv4 addrs only.

the other part of that regex which is actually intended to match
ipv6 addrs only matches if they are enclosed in square brackets.

also added tests for ssh and scp style repo URLs with ipv6 addrs
in brackets.

also: made regex more readable, putting these 2 cases on separate lines.
2022-04-03 21:30:28 +02:00
Thomas Deutschmann
821d814af3 delete: add repository id and location to prompt
Closes: https://github.com/borgbackup/borg/issues/6453
2022-03-28 10:16:53 -04:00
TW
b5443da6cb
Merge pull request #6493 from ThomasWaldmann/use-compare_digest-1.2
use hmac.compare_digest instead of ==, fixes #6470
2022-03-26 19:48:05 +01:00
Thomas Waldmann
b75079c0f8 use hmac.compare_digest instead of ==, fixes #6470 2022-03-26 18:52:41 +01:00
Thomas Waldmann
3e91694a0a avoid losing the key (old crypto)
if we just have a pointer to a bytes object which might go out of scope, we can lose it.

also: cython can directly assign a bytes object into a same-size char array.
2022-03-26 18:32:44 +01:00
Björn Ketelaars
71fa056c47 1.2 - Remove unused code
#6473 Removed AEAD ciphers AES-OCB and CHACHA20-POLY1305 from 1.2. As a
result some additional code can be removed.

Passes regression tests.
2022-03-21 18:54:50 +01:00
Thomas Waldmann
aa57a3cb57 remove AEAD ciphers AES-OCB and CHACHA20-POLY1305, fixes #6472
not used in 1.2.x, we'll start using them in 1.3.
2022-03-20 23:05:30 +01:00
Andrey Bienkowski
a58fd3db5c 1.2: Make switch fallthrough explicit 2022-03-10 08:24:23 +03:00
TW
f698c12129
Merge pull request #6403 from ThomasWaldmann/fix-savefile-mode-1.2
respect umask for files / dirs (1.2)
2022-03-09 16:53:35 +01:00
braewoods
8b4c501d8d
[1.2-maint] Backport implicit warning patches from master (#6427)
* suppress -Wimplicit-fallthrough warning

These instances of implicit switch case fallthrough appear to be
intentional. Add comments that the compiler understands to suppress
the false positive warning.
2022-03-09 16:10:21 +01:00
Thomas Waldmann
dc811bda6f fix "'HMAC_CTX' already defined" cython warning 2022-03-08 21:47:14 +01:00
Thomas Waldmann
cd629d32cb fix "useless trailing comma" cython warnings 2022-03-08 21:27:20 +01:00
Andrey Bienkowski
f05036cbcb ‘PyUnicode_AsUnicode’ is deprecated
Fix compilation warnings:
```
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.9.10/x64/include/python3.9 -c src/borg/platform/posix.c -o build/temp.linux-x86_64-3.9/src/borg/platform/posix.o
src/borg/platform/posix.c: In function ‘__pyx_pf_4borg_8platform_5posix_2swidth’:
src/borg/platform/posix.c:1572:3: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
 1572 |   __pyx_t_2 = __Pyx_PyUnicode_AsUnicode(__pyx_v_s); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 19, __pyx_L1_error)
      |   ^~~~~~~~~
In file included from /opt/hostedtoolcache/Python/3.9.10/x64/include/python3.9/unicodeobject.h:1026,
                 from /opt/hostedtoolcache/Python/3.9.10/x64/include/python3.9/Python.h:93,
                 from src/borg/platform/posix.c:19:
/opt/hostedtoolcache/Python/3.9.10/x64/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
```

Ref: https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUnicode
2022-03-07 21:40:39 +03:00
Thomas Waldmann
ce72a87f2d fixup: fix SELFTEST_COUNT 2022-03-06 11:31:05 +03:00
Thomas Waldmann
65ccbc2c69 fixup: remove now unused import 2022-03-06 11:31:05 +03:00
Thomas Waldmann
45ef108d46 remove our blake2b_128/256 tests
as we're just calling python stdlib there,
we can assume that they have tests for it.

 Conflicts:
	src/borg/testsuite/crypto.py
2022-03-06 11:30:58 +03:00
Thomas Waldmann
c2211a7ef9 remove our hmac_sha256 tests
as we're just calling python stdlib there,
we can assume that they have tests for it.

 Conflicts:
	src/borg/testsuite/crypto.py
2022-03-06 11:28:27 +03:00
Thomas Waldmann
72ccafb4b8 hmac.digest: some more minor optimizations
also some cosmetic changes:
- import hmac module
- use hmac.compare_digest
2022-03-06 11:26:04 +03:00
Thomas Waldmann
bcd1d49d3b hmac_sha256: replace own cython wrapper code by hmac.digest python stdlib (since py38)
i measured performance of both: pretty much the same.
2022-03-06 11:26:04 +03:00
TW
d0fb234f46
Merge pull request #6404 from ThomasWaldmann/import-tar-fixes-1.2
import-tar fixes
2022-03-05 14:26:59 +01:00
TW
8abb0aa7ab
Merge pull request #6375 from ThomasWaldmann/fix-6374-1.2
create: skip with warning if opening the parent dir of recursion root fails, fixes #6374
2022-03-05 14:18:40 +01:00
Thomas Waldmann
7dc800fecd kill filter process in case of borg exceptions, fixes #6401
in the finally-block, we wait for the filter process to die. but it only dies
voluntarily if all data was processed by the filter and it terminates due to EOF.

otoh, if borg has thrown an early exception, e.g. "archive already exists",
we need to kill the filter process to bring it to an early end. in that
case, we also do not need to check the filter rc, because we know we killed it.
2022-03-04 23:33:53 +01:00
Thomas Waldmann
33192e5e5d import-tar: fix mtime type bug
looks like with a .tar file created by the tar tool,
tarinfo.mtime is a float [s]. So, after converting to
nanoseconds, we need to cast to int because that's what
Item.mtime wants.

also added a safe_ns() there to clip values to the safe range.
2022-03-04 23:33:53 +01:00
Thomas Waldmann
f5a47d86e3 ensure_dir: respect umask for created directory modes, fixes #6400
we tried to be very private / secure here, but that created the issue
that a less secure umask (like e.g. 0o007) just did not work.

to make the umask work, we must start from 0o777 mode and let the
umask do its work, like e.g. 0o777 & ~0o007 --> 0o770.

with borg's default umask of 0o077, it usually ends up being 0o700,
so only permissions for the user (not group, not others).
2022-03-04 21:24:42 +01:00
Thomas Waldmann
9ea1d2a7d6 SaveFile: respect umask for final file mode, fixes #6400 2022-03-04 20:53:10 +01:00
Thomas Waldmann
8edb1dc085 manual formatting fixes (cosmetic / pep8) 2022-02-27 21:13:04 +01:00
Thomas Waldmann
c4ed840c7b pyupgrade --py38-plus ./**/*.py 2022-02-27 21:12:44 +01:00
TW
97de7e1fe2
Merge pull request #6380 from ThomasWaldmann/fix-zstd-include-1.2
fix zstd.h include for bundled zstd, fixes #6369
2022-02-27 14:39:09 +01:00
Thomas Waldmann
9390d2712b fix zstd.h include for bundled zstd, fixes #6369 2022-02-27 14:35:19 +01:00
Thomas Waldmann
da0ddacc54 create: skip with warning if opening the parent dir of recursion root fails, fixes #6374 2022-02-27 02:40:15 +01:00
James Buren
1618c98975 src/borg/cache_sync/unpack.h: fix compiler warnings
The key argument being sent to hashindex_get and hashindex_set by
multiple functions is a different signedness from what the functions
expect. This resolves the issue by changing the key type in the
unpack_user struct to unsigned char.
2022-02-26 16:13:07 -06:00
James Buren
4427c16d31 src/borg/_hashindex.c: fix compiler warnings
The value argument of hashindex_set is causing harmless pointer type
mismatches. This resolves the issue by changing the type to void*
which silences these types of warnings.
2022-02-26 16:13:02 -06:00
James Buren
8e640ae0f3 src/borg/compress.pyx: fix compiler warning, closes #6365
This resolves a compiler warning from the generated code that
resulted from a comparison of two local variables of different
signedness. The issue is resolved by changing the type of both
to int since this seems like the safest choice available.
2022-02-26 13:20:39 -06:00
James Buren
067dd10027 src/borg/crypto/low_level.pyx: fix compiler warning
The generated source code was producing a compiler warning due to
the pointers differing in constness. The called function expects
a non-const pointer while the generated code produces a const pointer
via a cast. This changes the cast to drop 'const' to make the compiler
happy.
2022-02-25 17:36:48 -06:00
TW
8a409ec1fb
Merge pull request #6306 from ThomasWaldmann/fix-savefile-races-master
SaveFile: fix race conditions
2022-02-17 16:51:43 +01:00
Laurent
6015ded828
docs: explain the difference between a path that ends with or without a slash (#6297)
docs: explain the difference between a path that ends with or without a slash
2022-02-16 23:53:11 +01:00
Thomas Waldmann
14b5c005d8 SaveFile: fix race conditions
Thanks to Andrey Bienkowski (@hexagonrecursion) for reporting this and writing reproducer code.

Changes:
- use different, randomly (but recognizably) named temp files while writing (securely made by os.mkstemp())
- make sure temp files are cleaned up in normal and error conditions
- SyncFile can now get corresponding pair of path + open os-level fd
- cleaned up: fd now means os-level fd, f means python-file-like object
- fixed a caller of SaveFile
2022-02-16 23:23:47 +01:00