This patch implements the P3860R1 (accepted as DR against C++20) and LWG4472.
The two constraints on the constructor (that T and U are similar types and
is_convertible_v<U*, T*>) are combined into a single check:
is_convertible_v<_Up(*)[1], _Tp(*)[1]>. While this check is not equivalent
for array of known bound to array of unknown bound conversions (T[N] to T[]),
this is irrelevant for atomic_ref, since instantiation with an array type is
ill-formed (due to the return type of load and other members).
The __atomic_ref_base constructor is modified to accept _Tp* instead of _Tp&.
This allows both the atomic_ref(atomic_ref<_Up> __other) and atomic_ref(_Tp& __t)
constructors to delegate to it. Furthermore, such approach does not require
dereferencing *__other._M_ptr, and thus avoid ADL-lookup for operator* and
issues related to it. The precondition check on alignment is moved specifically
to the atomic_ref(_Tp&) constructor, preventing redundant checks during atomic_ref
conversion.
A deleted atomic_ref(_Tp&&) constructor is introduced (per LWG4472).
This prevents the construction of atomic_ref<T> from atomic_ref<volatile T>
via the conversion operator.
libstdc++-v3/ChangeLog:
* include/bits/atomic_base.h
(__atomic_ref_base<const _Tp>::__atomic_ref_base): Accept
pointer instead of reference. Remove precondition check and
mark as noexcept.
(__atomic_ref_base<_Tp>::__atomic_ref_base): Accept pointer
insted of reference, and mark as noexcept.
* include/std/atomic (atomic_ref::atomic_ref(_Tp&)): Add
precondition check and take address of argument.
(atomic_ref::atomic_ref(_Tp&&)): Define as deleted.
(atomic_ref::atomic_ref(atomic_ref<_Up>)): Define.
* include/bits/shared_ptr_atomic.h (_Sp_atomic::_Atomic_count):
Pass address to __atomic_ref constructor.
* include/std/barrier (__tree_barrier_base::_M_arrive)
(__tree_barrier::arrive): Pass address to __atomic_ref constructor.
* testsuite/29_atomics/atomic_ref/ctor.cc: New test.
Reviewed-by: Jonathan Wakely <jwakely@redhat.com>
Signed-off-by: Tomasz Kamiński <tkaminsk@redhat.com>
Previously, atomic<shared_ptr<T>>::wait (and the weak_ptr version) was
equivalent to waiting directly on _M_val, which corresponds to the pointer
to the control block (_M_pi). Consequently, wakeups were not triggered if
the stored pointer value was changed to a pointer that uses the same control
block but stores pointer to a different object. Such a pointer can be
constructed using an aliasing constructor.
To address this, wait now uses a generic proxy wait
std::__atomic_wait_address function, which supports waiting until any
predicate is satisfied. The provided predicate now compares both the control
block (_M_pi) and the stored pointer (_M_ptr). Comparing the latter requires
locking the pointer.
Since this function operates on raw pointers, the type of _M_val was changed
from __atomic_base<uintptr_t> to uintptr_t. Invocations of the corresponding
member functions are now replaced with direct use of __atomic builtins.
PR libstdc++/118757
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (_Atomic_count::_M_wait_unlock):
Add parameter capturing reference to _M_ptr. Reimplement in terms
of __atomic_wait_address.
(_Atomic_count::~_Atomic_count, _Atomic_count::lock)
(_Atomic_count::unlock, _Atomic_count::_M_swap_unlock): Replace
invocation of atomic member funcitons with __atomic builtins.
(_Atomic_count::notify_one, _Atomic_count::notify_all):
Use __atomic_notify_address.
(_Sp_atomic::element_type): Define.
(_Sp_atomic::_M_val): Change type to uintptr_t.
(_Sp_atomic::wait): Pass _M_ptr to _M_wait_unlock.
* python/libstdcxx/v6/printers.py:
* testsuite/20_util/shared_ptr/atomic/pr118757.cc: New test.
* testsuite/20_util/weak_ptr/pr118757.cc: New test.
Reviewed-by: Jonathan Wakely <jwakely@redhat.com>
Signed-off-by: Tomasz Kamiński <tkaminsk@redhat.com>
This implements that changes from P1132R8, including optimized paths for
std::shared_ptr and std::unique_ptr.
For std::shared_ptr we pre-allocate a new control block in the
std::out_ptr_t constructor so that the destructor is non-throwing. This
requires some care because unlike the shared_ptr(Y*, D, A) constructor,
we don't want to invoke the deleter if allocating the control block
throws, because we don't own any pointer yet. In order to avoid the
unwanted deleter invocation, we create the control block manually. We
also want to avoid invoking the deleter on a null pointer on
destruction, so we destroy the control block manually if there is no
pointer to take ownership of.
For std::unique_ptr and for raw pointers, the out_ptr_t object hands out
direct access to the pointer, so that we don't have anything to do
(except possibly assign a new deleter) in the ~out_ptr_t destructor.
These optimizations avoid requiring additional temporary storage for the
pointer (and optional arguments), and avoid additional instructions to
copy that pointer into the smart pointer at the end.
libstdc++-v3/ChangeLog:
PR libstdc++/111667
* include/Makefile.am: Add new header.
* include/Makefile.in: Regenerate.
* include/bits/out_ptr.h: New file.
* include/bits/shared_ptr.h (__is_shared_ptr): Move definition
to here ...
* include/bits/shared_ptr_atomic.h (__is_shared_ptr): ... from
here.
* include/bits/shared_ptr_base.h (__shared_count): Declare
out_ptr_t as a friend.
(_Sp_counted_deleter, __shared_ptr): Likewise.
* include/bits/unique_ptr.h (unique_ptr, unique_ptr<T[], D>):
Declare out_ptr_t and inout_ptr_t as friends.
(__is_unique_ptr): Define new variable template.
* include/bits/version.def (out_ptr): Define.
* include/bits/version.h: Regenerate.
* include/std/memory: Include new header.
* testsuite/20_util/smartptr.adapt/inout_ptr/1.cc: New test.
* testsuite/20_util/smartptr.adapt/inout_ptr/2.cc: New test.
* testsuite/20_util/smartptr.adapt/inout_ptr/shared_ptr_neg.cc:
New test.
* testsuite/20_util/smartptr.adapt/inout_ptr/void_ptr.cc: New
test.
* testsuite/20_util/smartptr.adapt/out_ptr/1.cc: New test.
* testsuite/20_util/smartptr.adapt/out_ptr/2.cc: New test.
* testsuite/20_util/smartptr.adapt/out_ptr/shared_ptr_neg.cc:
New test.
* testsuite/20_util/smartptr.adapt/out_ptr/void_ptr.cc: New
test.
This change moves the definitions of feature test macros (or strictly
speaking, the requests for <bits/version.h> to define them) so that only
standard headers define them. For example, <bits/shared_ptr.h> will no
longer define macros related to std::shared_ptr, only <memory> and
<version> will define them. This means that __cpp_lib_shared_ptr_arrays
will not be defined by <future> or by other headers that include
<bits/shared_ptr.h>. It will only be defined when <memory> has been
included. This will discourage users from relying on transitive
includes.
As a result, internal headers that need to query the macros should use
the internal macros like __glibcxx_shared_ptr_arrays instead of
__cpp_lib_shared_ptr_arrays, as those internal macros are defined by the
internal headers after icluding <bits/version.h>. There are some
exceptions to this rule, because __cpp_lib_is_constant_evaluated is
defined by bits/c++config.h and so is available everywhere, and
__cpp_lib_three_way_comparison is defined by <compare> which several
headers are explicitly specified to include, so its macro is guaranteed
to be usable too.
N.B. not many internal headers actually need an explicit include of
<bits/version.h>, because most of them include <type_traits> and so get
all the __glibcxx_foo internal macros from there.
libstdc++-v3/ChangeLog:
* include/bits/algorithmfwd.h: Do not define standard feature
test macro here.
* include/bits/align.h: Likewise. Test internal macros instead
of standard macros.
* include/bits/alloc_traits.h: Likewise.
* include/bits/allocator.h: Likewise.
* include/bits/atomic_base.h: Likewise.
* include/bits/atomic_timed_wait.h: Likewise.
* include/bits/atomic_wait.h: Likewise.
* include/bits/basic_string.h: Likewise.
* include/bits/basic_string.tcc: Likewise.
* include/bits/char_traits.h: Likewise.
* include/bits/chrono.h: Likewise.
* include/bits/cow_string.h: Likewise.
* include/bits/forward_list.h: Likewise.
* include/bits/hashtable.h: Likewise.
* include/bits/ios_base.h: Likewise.
* include/bits/memory_resource.h: Likewise.
* include/bits/move.h: Likewise.
* include/bits/move_only_function.h: Likewise.
* include/bits/node_handle.h: Likewise.
* include/bits/ptr_traits.h: Likewise.
* include/bits/range_access.h: Likewise.
* include/bits/ranges_algo.h: Likewise.
* include/bits/ranges_cmp.h: Likewise.
* include/bits/ranges_util.h: Likewise.
* include/bits/semaphore_base.h: Likewise.
* include/bits/shared_ptr.h: Likewise.
* include/bits/shared_ptr_atomic.h: Likewise.
* include/bits/shared_ptr_base.h: Likewise.
* include/bits/stl_algo.h: Likewise.
* include/bits/stl_algobase.h: Likewise.
* include/bits/stl_function.h: Likewise.
* include/bits/stl_iterator.h: Likewise.
* include/bits/stl_list.h: Likewise.
* include/bits/stl_map.h: Likewise.
* include/bits/stl_pair.h: Likewise.
* include/bits/stl_queue.h: Likewise.
* include/bits/stl_stack.h: Likewise.
* include/bits/stl_tree.h: Likewise.
* include/bits/stl_uninitialized.h: Likewise.
* include/bits/stl_vector.h: Likewise.
* include/bits/unique_ptr.h: Likewise.
* include/bits/unordered_map.h: Likewise.
* include/bits/uses_allocator_args.h: Likewise.
* include/bits/utility.h: Likewise.
* include/bits/erase_if.h: Add comment.
* include/std/algorithm: Define standard feature test macros
here.
* include/std/atomic: Likewise.
* include/std/array: Likewise.
* include/std/chrono: Likewise.
* include/std/condition_variable: Likewise.
* include/std/deque: Likewise.
* include/std/format: Likewise.
* include/std/functional: Likewise.
* include/std/forward_list: Likewise.
* include/std/ios: Likewise.
* include/std/iterator: Likewise.
* include/std/list: Likewise.
* include/std/map: Likewise.
* include/std/memory: Likewise.
* include/std/numeric: Likewise.
* include/std/queue: Likewise.
* include/std/ranges: Likewise.
* include/std/regex: Likewise.
* include/std/set: Likewise.
* include/std/stack: Likewise.
* include/std/stop_token: Likewise.
* include/std/string: Likewise.
* include/std/string_view:
* include/std/tuple: Likewise.
* include/std/unordered_map:
* include/std/unordered_set:
* include/std/utility: Likewise.
* include/std/vector: Likewise.
* include/std/scoped_allocator: Query internal macros instead of
standard macros.
These overloads are deprecated in C++20 (and likely to be removed for
C++26). The std::atomic<std::shared_ptr<T>> specialization should be
preferred in new code.
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (atomic_is_lock_free)
(atomic_load_explicit, atomic_load, atomic_store_explicit)
(atomic_store, atomic_exchange_explicit, atomic_exchange)
(atomic_compare_exchange_strong, atomic_compare_exchange_weak)
(atomic_compare_exchange_strong_explicit)
(atomic_compare_exchange_weak_explicit): Add deprecated
attribute for C++20 and later.
* testsuite/20_util/shared_ptr/atomic/1.cc: Suppress deprecated
warnings.
* testsuite/20_util/shared_ptr/atomic/2.cc: Likewise.
* testsuite/20_util/shared_ptr/atomic/3.cc: Likewise.
* testsuite/29_atomics/atomic/lwg3220.cc: Likewise.
LWG voted this to Tentatively Ready recently.
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (atomic::operator=(nullptr_t)):
Add overload, as per LWG 3893.
* testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc:
Check assignment from nullptr.
Do not use the __tsan_mutex_not_static flag for annotation functions
where it's not a valid flag. Also use the try_lock and try_lock_failed
flags to more precisely annotate the CAS loop used to acquire a lock.
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (_GLIBCXX_TSAN_MUTEX_PRE_LOCK):
Replace with ...
(_GLIBCXX_TSAN_MUTEX_TRY_LOCK): ... this, add try_lock flag.
(_GLIBCXX_TSAN_MUTEX_TRY_LOCK_FAILED): New macro using
try_lock_failed flag
(_GLIBCXX_TSAN_MUTEX_POST_LOCK): Rename to ...
(_GLIBCXX_TSAN_MUTEX_LOCKED): ... this.
(_GLIBCXX_TSAN_MUTEX_PRE_UNLOCK): Remove invalid flag.
(_GLIBCXX_TSAN_MUTEX_POST_UNLOCK): Remove invalid flag.
(_Sp_atomic::_Atomic_count::lock): Use new macros.
This adds annotations to std::atomic<shared_ptr<T>> to enable TSan to
understand the custom locking. Without this, TSan reports data races for
accesses to the _M_ptr member, even though those are correctly
synchronized using atomic operations on the tagged pointer.
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (_GLIBCXX_TSAN_MUTEX_DESTROY)
(_GLIBCXX_TSAN_MUTEX_PRE_LOCK, _GLIBCXX_TSAN_MUTEX_POST_LOCK)
(_GLIBCXX_TSAN_MUTEX_PRE_UNLOCK, _GLIBCXX_TSAN_MUTEX_POST_UNLOCK)
(_GLIBCXX_TSAN_MUTEX_PRE_SIGNAL, _GLIBCXX_TSAN_MUTEX_POST_SIGNAL):
Define macros for TSan annotation functions.
(_Sp_atomic::_Atomic_count): Add annotations.
libstdc++-v3/ChangeLog:
* include/std/atomic: Suppress doxygen docs for
implementation details.
* include/bits/atomic_base.h: Likewise.
* include/bits/shared_ptr_atomic.h: Use markdown. Fix grouping
so that std::atomic is not added to the pointer abstractions
group.
This DR was approved at the February 2022 plenary.
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (atomic<shared_ptr>): Add
constructor for constant initialization from nullptr_t.
* testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc:
Check for new constructor.
This fixes an on AIX.
The lock function currently just spins, which should be changed to use
back-off, and maybe then _M_val.wait(__current) when supported.
libstdc++-v3/ChangeLog:
PR libstdc++/104101
* include/bits/shared_ptr_atomic.h (_Sp_atomic::_Atomic_count::lock):
Only use __thread_relax if __cpp_lib_atomic_wait is defined.
This adds another piece of C++20, the std::atomic specializations for
std::shared_ptr and std::weak_ptr.
The new _Sp_atomic type mimics the structure of shared_ptr<T> and
weak_ptr<T>, holding a T* pointer (the one returned by get() on a
shared_ptr/weak ptr) and a _Sp_counted_base<>* pointer to the
ref-counted control block. For _Sp_atomic the low bit of the control
block pointer is used as a lock bit, to ensure only one thread will
access the object at a time. The pointer is actually stored as a
uintptr_t to avoid accidental dereferences of the pointer when unlocked
(which would be a race) or when locked (which would dereference the
wrong pointer value due to the low bit being set). To get a raw pointer
to the control block, the lock must be acquired. Converting between a
_Sp_atomic and a shared_ptr or weak_ptr requires manually adjusting the
T* and _Sp_counted_base<>* members of the shared/weak ptr, instead of
going through the public API. This must be done carefully to ensure that
any change in the number of owners is reflected in a ref-count update.
Co-authored-by: Thomas Rodgers <trodgers@redhat.com>
Signed-off-by: Thomas Rodgers <trodgers@redhat.com>
libstdc++-v3/ChangeLog:
* include/bits/shared_ptr_atomic.h (__cpp_lib_atomic_shared_ptr):
New macro.
(_Sp_atomic): New class template.
(atomic<shared_ptr<T>>, atomic<weak_ptr<T>>): New partial
specializations.
* include/bits/shared_ptr_base.h (__shared_count, __weak_count)
(__shared_ptr, __weak_ptr): Declare _Sp_atomic as a friend.
* include/std/version (__cpp_lib_atomic_shared_ptr): New macro.
* testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc: New
test.
* testsuite/20_util/weak_ptr/atomic_weak_ptr.cc: New test.
Jakub Jelinek