The addend for PT_TLS p_vaddr value should be obj' relocbase and not
mapbase. It does not matter for dso which is linked at the address
zero, but for executables in direct-exec mode with non-zero link
address mapbase is already at the link base. Then, adding mapbase to
phtls->p_vaddr adds twice as much relocbase offset as needed.
PR: 288334
Reported by: Jordan Gordeev <jgopensource@proton.me>
Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D51448
Currently rtld delegates to libc or libthr to initialise the TCBs for
all existing threads when dlopen is called for a library that is using
static TLS. This creates an odd split where rtld manages all of TLS for
dynamically-linked executables except for this specific case, and is
unnecessarily complex, including having to reason about the locking due
to dropping the bind lock so libthr can take the thread list lock
without deadlocking if any of the code run whilst that lock is held ends
up calling back into rtld (such as for lazy PLT resolution).
The only real reason we call out into libc / libthr is that we don't
have a list of threads in rtld and that's how we find the currently used
TCBs to initialise (and at the same time do the copy in the callee
rather than adding overhead with some kind of callback that provides the
TCB to rtld. If we instead keep a list of allocated TCBs in rtld itself
then we no longer need to do this, and can just copy the data in rtld.
How these TCBs are mapped to threads is irrelevant, rtld can just treat
all TCBs equally and ensure that each TCB's static TLS data block
remains in sync with the current set of loaded modules, just as how
_rtld_allocate_tls creates a fresh TCB and associated data without any
embedded threading model assumptions.
As an implementation detail, to avoid a separate allocation for the list
entry and having to find that allocation from the TCB to remove and free
it on deallocation, we allocate a fake TLS offset for it and embed the
list entry there in each TLS block.
This will also make it easier to add a new TLS ABI downstream in
CheriBSD, especially in the presence of library compartmentalisation.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50920
This will be used to allocate additional space for a TAILQ_ENTRY by rtld
at a known offset from the TCB, as if it were TLS data.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D51068
Add the AT_HWCAP3 and AT_HWCAP4 format strings to auxfmt.
Reviewed by: brooks, kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D51007
The only caller already has the current TCB to hand, so just pass it
down rather than get it again. This also makes it clear in the caller
that it depends on the (current) TCB, rather than being storage that
could be assigned to any thread (concurrency issues aside).
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50594
When this code was first written we didn't have even a struct tcb, so to
make it MI a pointer to the DTV pointer in the TCB was passed around.
Now that we have a struct tcb we can simplify the code by instead
passing around a pointer to that, and the MI code can access the tcb_dtv
member wherever it happens to be in the layout. This reduces boilerplate
in all the various callers of tls_get_addr_common/slow and makes it
clearer that tls_get_addr_common/slow are operating on the TCB, rather
than obfuscating it slightly through the double pointer.
Whilst here, clarify the comments in aarch64's TLSDESC dynamic resolver,
which were using tp without clarifying what this was for (previously a
pointer to the DTV pointer, now a pointer to the TCB, which happen to be
the same thing for Variant I TLS, and in the case of AArch64 are what
TPIDR_EL0 point to directly, with no offset/bias).
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50591
This was applying a NetBSD fix to FreeBSD. However, the original code
was correct for FreeBSD. NetBSD's obj->tlsoffset is relative to the end
of the TCB, not the TCB itself, whilst ours is relative to the TCB[1]
itself. For example, our allocate_tls uses (char *)tcb + obj->tlsoffset
for the memcpy and memset calls.
Without this reverted, for dynamically loaded shared objects, Initial
Exec accesses to TLS variables on variant I architectures (non-x86) use
the correct address, whilst General Dynamic and dlsym(3) use the
incorrect address (TLS_TCB_SIZE past the start). Note that, on arm64,
LLVM only supports TLSDESC (including LLD) and TLSDESC will use the
static resolver if the variable ends up allocated to the static TLS
block, even in the presence of dlopen(3), so only dlsym(3) shows the
discrepancy there.
Whilst here, add a comment to explain this difference to try and avoid
the same mistake being made in future.
[1] In the case of variant II, it's the amount to subtract, so still
positive
This reverts commit e9a38ed2fa.
Reviewed by: kib (prior version)
Fixes: e9a38ed2fa ("rtld: fix allocate_module_tls() variant I fallback to static allocation")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50565
Provided you don't run out of extra static TLS space this should work,
but it's wholly unnecessary and not how things are supposed to be done.
Only static TLS relocations should allocate static TLS.
Reviewed by: kib
Fixes: 4b1859c0e9 ("Add support for RISC-V architecture.")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50563
Reported and tested: Alex S <iwtcex@gmail.com>
Tested by: pho
Reviewed by: olce
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D50482
When I added libsys I failed to update rtld's reuse of object files
from libc to use ones from libsys instead. This would have turned up
as a broken system in d7847a8d35, but SHARED_CFLAGS is not being
applied to assembly files.
PR: 286975
Reviewed by: jrtc27, jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D50475
This was previously an Elf_Addr and was turned into a uintptr_t when
really it should have been made a size_t. Even on CHERI both actually
work in this case, but it's better style (and more consistent with
elsewhere in the file) to use size_t instead.
Fixes: 4642b638a5 ("rtld-elf: Consistently use uintptr_t for TLS implementation")
Rather than treating the DTV as a raw array of uintptr_t, use proper
struct types and gain the benefit of having different types for
different members. In particular, the module slots now have real pointer
types so less casting is generally needed.
Note that, whilst struct dtv_slot may seem a little unnecessary, this
will help downstream in CheriBSD where we wish to be able to easily
alter the layout of a module's slot, which this helps abstract.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50231
On traditional architectures where uintptr_t is just a plain integer,
there is no provenance from the order of operations. However, on CHERI
there is even for uintptr_t, and in future this code will use actual
pointer types anyway, where the provenance does technically matter even
for non-CHERI. Commute and associate the operands appropriately to
ensure the provenance is for the new allocation, not the old one.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50230
This lets us access via named struct members rather than magic
hard-coded indices.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50229
Firstly, the second argument to allocate_tls is the old TCB (versus
oldtls, which has less of a clear meaning), so rename it to oldtcb like
Variant I.
Secondly, segbase and oldsegbase are oriented towards what ends up in
the segment registers, but that's not the main concern here, and those
don't convey what they actually point to. Instead, rename segbase to tcb
and change it to a uintptr_t **, and remove oldsegbase as it's always
equal to oldtcb, again both matching Variant I.
Thirdly, rename tls to tls_block, again both matching Variant I.
Finally, similarly rename tls to tcb in free_tls, and oldtls to oldtcb
in the common _rtld_allocate_tls.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50232
Elf_Addr is the format of addresses in the ELF file with the current
ABI's default class. This is normally the same as the format of an
address at run time, though technically exceptions do exist outside of
FreeBSD's currently-supported architectures (for example, IA-64's LP64
supports both ELFCLASS32 and ELFCLASS64 file formats; LP64 vs ILP32 is
an orthogonal EF_IA_64_ABI64 flag). On traditional architectures,
including all currently-supported FreeBSD architectures, addresses and
pointers are synonymous, but on CHERI they are not, as pointers are
capabilities that contain metadata alongside the address. In the cases
here, the quantities are run-time pointers, not addresses (and
definitely not ELF file addresses), so we should use pointer-ish types.
Note that we already use uintptr_t in struct tcb (both Variant I and
Variant II) but still use Elf_Addr in various places here (including
different argument types for tls_get_addr_slow and tls_get_addr_common).
Also use char * for addr rather than even uintptr_t, since most of the
time we want it to be an actual pointer.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50226
Rather than calling tls_get_addr_common with a biased ti_offset and
un-biasing the return value, we can instead un-bias the offset given in
the first place and not need to adjust the return value. This is in fact
how the MIPS implementation worked, and makes this call tail-recursive.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50187
For direct exec mode we reuse map_object, but tls_max_index is
initialised to 1. As a result, the executable ends up being assigned
module 2 (and the generation is pointlessly incremented, unlike in
digest_phdr for the normal case). For most architectures this is
harmless, since TLS linker relaxation will optimise General Dynamic
accesses to Initial Exec or Local Exec for executables, but on RISC-V
this relaxation does not exist, yet the linker will initialise the
tls_index in the GOT with module 1, and at run time the call to
__tls_get_addr will fail with:
ld-elf.so.1: Can't find module with TLS index 1
Fix this by making map_object use 1 for obj->tlsindex when it's loading
the main executable, and don't bother to increment tls_dtv_generation
either, matching digest_phdr (though that one is harmless).
(Note this also applies to MIPS on stable/13)
Reviewed by: kib
Fixes: 0fc65b0ab8 ("Make ld-elf.so.1 directly executable.")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50186
If no extra rtld arguments are provided, rtld_argc will be 1 (for
argv[0] and so we are shifting the entire memory range down by a single
pointer. However, unlike argv and envp, auxp's entries are two pointers
in size, not one, and so in this case the source and destination
overlap, meaning simple assignment is UB (C99 6.5.16.1p3). On many
architectures this ends up being harmless as the compiler will emit
double machine word loads and stores, or if it splits them it may still
schedule them such that it works in this case, but our RISC-V baseline
does not include such instructions and LLVM ends up picking a schedule
that copies the second word before the first word, thereby replacing the
first word with a copy of the second word. This results in direct exec
mode segfaulting on RISC-V when given no arguments.
Fix this by using a temporary in the source and let the compiler safely
elide its use.
Reviewed by: kib
Fixes: 0fc65b0ab8 ("Make ld-elf.so.1 directly executable.")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50185
The implementation of dl_iterate_phdr abuses tls_get_addr_slow to get to
the start of the TLS block, inlining the implementation of
__tls_get_addr as if the tls_index's ti_offset were 0 (historically it
called __tls_get_addr itself but changed due to locking issues). For
most architectures, tls_index's ti_offset (relocated by DTPOFF/DTPREL
for GOT entries) is just the offset within that module's TLS block.
However, for PowerPC and RISC-V, which have a non-zero TLS_DTV_OFFSET
and thus are designed assuming DTV entries are biased by that value,
ti_offset normally has TLS_DTV_OFFSET pre-subtracted, but it's
__tls_get_addr's responsibility to compensate for that. By using an
offset of zero here, tls_get_addr_slow will return a pointer to the
start of the TLS block itself, so by adding TLS_DTV_OFFSET we will point
TLS_DTV_OFFSET past the module's TLS block.
Fix this by removing the extra bias (the alternative would be to pass
-TLS_DTV_OFFSET and keep the addition, which would more closely follow
what __tls_get_addr does, but this is more direct).
(Note this also applies to MIPS on stable/13)
Reviewed by: kib
Fixes: d36d681615 ("rtld dl_iterate_phdr(): dlpi_tls_data is wrong")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50184
The implementation here is meant to mirror what a GOT entry for the
given symbol would use for ti_offset. However, on PowerPC and RISC-V,
TLS_DTV_OFFSET is non-zero, and so the GOT entries are normally biased
by this, but we fail to do so here. As a result we end up getting a
pointer TLS_DTV_OFFSET past where the variable actually is.
(Note this also applies to MIPS on stable/13)
Reviewed by: kib
Fixes: 5ceeeba90c ("Import DragonFly BSD commit")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50183
Internally, initfirst objects and their needed objects are put on the
dedicated initlist, which is prepended to the current regular initlist
at the last moment.
This results in the move of the needed objects into the beginning of the
initlist, which is required for the proper initialization of the
dependencies. It seems that glibc moves only the initfirst object,
which makes its constructors depend on not yet initialized dsos.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D50132
If filter needs to be loader, we restart after the lock upgrade. But
possible binds in the resolver itself would try to recurse on the lock,
which can be only done for the read locks.
PR: 286502
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This allows, for instance, to interpose opendir() with an implementation
that calls into fdopendir(), without causing multiple symbol definitions
when libc is linked in statically.
This is aligned with the quality of implementation principle that single
object file from a static library should not provide more than one
app-visible symbol.
[The copyright for fopendir() was copied, it might be refined by the
author]
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D49089
Previously, __realpathat was in libc and libsys (as is currently
standard), but not exported from libc which meant the stub in libc was
not filtered and thus libc's copy of the syscall was used. This broke
an upcoming change to CheriBSD limiting syscalls to libsys.
The realpath(3) implementation now uses __sys___realpathat so there are no
consumers of __realpathat. Switch it to PSEUDO (only _foo and __sys_foo
symbols) and remove __realpathat from Symbol.map.
This is a corrected version of 58d43a3cd7.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D49049
This should avoid an (almost) false positive from Valgrind, by filling
the padding on LP64.
PR: 284563
Reported by: Paul Floyd <pjfloyd@wanadoo.fr>
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D48854
Iterate over all the program headers in obj_remap_relro and remove the
relro fields from Obj_Entry.
Skip the call to obj_enforce_relro() in relocate_object() for the rtld
object as well as the main program object. obj_enforce_relro() is
called later when it safe to reference globals such as page_size.
Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D47884
Some sanitizers need to be able to use dl_iterate_phdr() after stopping
the rest of the process, but it's very hard to do so reliably as a
non-participant in the main logic of the program.
Introduce _dl_iterate_phdr_locked to bypass the locking that's normally
required for dl_iterate_phdr() and slap some scary warning on it. It
will remain undocumented and probably shouldn't be used for anything
else.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D47558
by making the LD_ENV_DESC() macro variadic.
Suggested by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D47351
