mirror of
https://github.com/openRuyi-Project/gcc.git
synced 2026-09-06 05:51:43 +00:00
master
53
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jonathan Wakely
|
59bc37debf |
libstdc++: Improve comments on __wait_args::_M_setup_proxy_wait
libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__wait_args): Improve comments. * src/c++20/atomic.cc (__wait_args::_M_setup_proxy_wait): Improve comment. Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com> |
||
|
Jonathan Wakely
|
5f02ba9258 |
libstdc++: Fix proxy wait detection in atomic wait
A failed assertion was observed with std::atomic<bool>::wait when the loop in __atomic_wait_address is entered and calls _M_setup_wait a second time, after waking from __wait_impl. When the first call to _M_setup_wait makes a call to _M_setup_proxy_wait that function decides that a proxy wait is needed for an object of type bool, and it updates the _M_obj and _M_obj_size members to refer to the futex in the proxy state, instead of referring to the bool object itself. The next time _M_setup_wait is called it calls _M_setup_proxy_wait again but now it sees _M_obj_size == sizeof(futex) and so this time decides a proxy wait is *not* needed, and then fails the __glibcxx_assert(_M_obj == addr) check. The problem is that _M_setup_proxy_wait wasn't correctly handling the case where it's called a second time, after the decision to use a proxy wait has already been made. That can be fixed in _M_setup_proxy_wait by checking if _M_obj != addr, which implies that a proxy wait has already been set up by a previous call. In that case, _M_setup_proxy_wait should only update _M_old to the latest value of the proxy _M_ver. This change means that _M_setup_proxy_wait is safe to call repeatedly for a proxy wait, and will only update _M_wait_state, _M_obj, and _M_obj_size on the first call. On the second and subsequent calls, those variables are already correctly set for the proxy wait so don't need to be set again. For non-proxy waits, calling _M_setup_proxy_wait more than once is safe, but pessimizes performance. The caller shouldn't make a second call to _M_setup_proxy_wait because we don't need to check again if a proxy wait should be used (the answer won't change) and we don't need to load a value from the proxy _M_ver. However, it was difficult to detect the case of a non-proxy wait, because _M_setup_wait doesn't know if it's being called the first time (when _M_setup_proxy_wait is called to make the initial decision) or a subsequent time (in which case _M_obj == addr implies a non-proxy wait was already decided on). As a result, _M_setup_proxy_wait was being used every time to see if it's a proxy wait. We can resolve this by splitting the _M_setup_wait function into _M_setup_wait and _M_on_wake, where the former is only called once to do the initial setup and the latter is called after __wait_impl returns, to prepare to check the predicate and possibly wait again. The new _M_on_wake function can avoid unnecessary calls to _M_setup_proxy_wait by checking _M_obj == addr to identify a non-proxy wait. The three callers of _M_setup_wait are updated to use _M_on_wake instead of _M_setup_wait after waking from a waiting function. This change revealed a latent performance bug in __atomic_wait_address_for which was not passing __res to _M_setup_wait, so a new value was always loaded even when __res._M_has_val was true. By splitting _M_on_wake out of _M_setup_wait this problem became more obvious, because we no longer have _M_setup_wait doing two different jobs, depending on whether it was passed the optional third argument or not. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__atomic_wait_address_until): Use _M_on_wake instead of _M_setup_wait after waking. (__atomic_wait_address_for): Likewise. * include/bits/atomic_wait.h (__atomic_wait_address): Likewise. (__wait_args::_M_setup_wait): Remove third parameter and move code to update _M_old to ... (__wait_args::_M_on_wake): New member function to update _M_old after waking, only calling _M_setup_proxy_wait if needed. (__wait_args::_M_store): New member function to update _M_old from a value, for non-proxy waits. * src/c++20/atomic.cc (__wait_args::_M_setup_proxy_wait): If _M_obj is not addr, only load a new value and return true. Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com> |
||
|
Jonathan Wakely
|
a6c853112d |
libstdc++: Remove redundant return statement after static_assert(false)
Jakub's fix for PR c++/91388 means that we don't need an unreachable return statement after static_assert(false). libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__wait_args::_M_setup_wait): Remove unreachable return statement. Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com> |
||
|
Jakub Jelinek
|
254a858ae7 | Update copyright years. | ||
|
Jonathan Wakely
|
7a2d141876 |
libstdc++: std::atomic should use std::addressof
libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__detail::__atomic_eq): Use std::addressof instead of &. * include/std/atomic (atomic::wait, atomic::notify_one) (atomic::notify_all): Likewise. Reviewed-by: Patrick Palka <ppalka@redhat.com> |
||
|
Jonathan Wakely
|
d07b9e7fe4 |
libstdc++: Future-proof C++20 atomic wait/notify
This will allow us to extend atomic waiting functions to support a possible future 64-bit version of futex, as well as supporting futex-like wait/wake primitives on other targets (e.g. macOS has os_sync_wait_on_address and FreeBSD has _umtx_op). Before this change, the decision of whether to do a proxy wait or to wait on the atomic variable itself was made in the header at compile-time, which makes it an ABI property that would not have been possible to change later. That would have meant that std::atomic<uint64_t> would always have to do a proxy wait even if Linux gains support for 64-bit futex2(2) calls at some point in the future. The disadvantage of proxy waits is that several distinct atomic objects can share the same proxy state, leading to contention between threads even when they are not waiting on the same atomic object, similar to false sharing. It also result in spurious wake-ups because doing a notify on an atomic object that uses a proxy wait will wake all waiters sharing the proxy. For types that are known to definitely not need a proxy wait (e.g. int on Linux) the header can still choose a more efficient path at compile-time. But for other types, the decision of whether to do a proxy wait is deferred to runtime, inside the library internals. This will make it possible for future versions of libstdc++.so to extend the set of types which don't need to use proxy waits, without ABI changes. The way the change works is to stop using the __proxy_wait flag that was set by the inline code in the headers. Instead the __wait_args struct has an extra pointer member which the library internals populate with either the address of the atomic object or the _M_ver counter in the proxy state. There is also a new _M_obj_size member which stores the size of the atomic object, so that the library can decide whether a proxy is needed. So for example if linux gains 64-bit futex support then the library can decide not to use a proxy when _M_obj_size == 8. Finally, the _M_old member of the __wait_args struct is changed to uint64_t so that it has room to store 64-bit values, not just whatever size the __platform_wait_t type is (which is a 32-bit int on Linux). Similarly, the _M_val member of __wait_result_type changes to uint64_t too. libstdc++-v3/ChangeLog: * config/abi/pre/gnu.ver: Adjust exports. * include/bits/atomic_timed_wait.h (_GLIBCXX_HAVE_PLATFORM_TIMED_WAIT): Do not define this macro. (__atomic_wait_address_until_v, __atomic_wait_address_for_v): Adjust assertions to check that __platform_wait_uses_type is true. * include/bits/atomic_wait.h (__waitable): New concept. (__platform_wait_uses_type): Different separately for platforms with and without platform wait. (_GLIBCXX_HAVE_PLATFORM_WAIT): Do not define this macro. (__wait_value_type): New typedef. (__wait_result_type): Change _M_val to __wait_value_type. (__wait_flags): Remove __proxy_wait enumerator. Reduce range reserved for ABI version by the commented-out value. (__wait_args_base::_M_old): Change type to __wait_args_base. (__wait_args_base::_M_obj, __wait_args_base::_M_obj_size): New data members. (__wait_args::__wait_args): Set _M_obj and _M_obj_size on construction. (__wait_args::_M_setup_wait): Change void* parameter to deduced type. Adjust bit_cast to work for types of different sizes. (__wait_args::_M_load_proxy_wait_val): Remove function, replace with ... (__wait_args::_M_setup_proxy_wait): New function. (__wait_args::_S_flags_for): Do not set __proxy_wait flag. (__atomic_wait_address_v): Adjust assertion to check that __platform_wait_uses_type is true. * src/c++20/atomic.cc (_GLIBCXX_HAVE_PLATFORM_WAIT): Define here instead of in header. Check _GLIBCXX_HAVE_PLATFORM_WAIT instead of _GLIBCXX_HAVE_PLATFORM_TIMED_WAIT. (__platform_wait, __platform_notify, __platform_wait_until): Add unused parameter for _M_obj_size. (__spin_impl): Adjust for 64-bit __wait_args_base::_M_old. (use_proxy_wait): New function. (__wait_args::_M_load_proxy_wait_val): Replace with ... (__wait_args::_M_setup_proxy_wait): New function. Call use_proxy_wait to decide at runtime whether to wait on the pointer directly instead of using a proxy. If a proxy is needed, set _M_obj and _M_obj_size to refer to its _M_ver member. Adjust for change to type of _M_old. (__wait_impl): Wait on _M_obj unconditionally. Pass _M_obj_size to __platform_wait. (__notify_impl): Call use_proxy_wait to decide whether to notify on the address parameter or a proxy (__spin_until_impl): Adjust for change to type of _M_val. (__wait_until_impl): Wait on _M_obj unconditionally. Pass _M_obj_size to __platform_wait_until. Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com> |
||
|
Jonathan Wakely
|
ba1ad0ec6f |
libstdc++: Add assertions to atomic waiting functions that need platform wait
These overloads should never be used for proxy waits, so add assertions to ensure that they aren't used accidentally. The reason they can't be used is that they don't call __args._M_setup_wait to obtain a __wait_state pointer. Even if that was changed, they would wait on a proxy wait which is potentially used by many other threads waiting on other addresses, meaning spurious wake ups are likely. In order to make the functions correct they would need to perform additional loads and comparisons of the atomic variable before calling __wait_impl or __wait_until_impl, which would make these functions no faster than the general purpose overloads that take an accessor function and predicate. That would be possible, and I think they would then work for proxy waits, but doesn't seem necessary at this time. In order to preseve the property that these functions are more lightweight and efficient than the general ones, they should not be used for proxy waits. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__atomic_wait_address_until_v): Add assertion to prevent use with proxy waits. (__atomic_wait_address_for_v): Likewise. * include/bits/atomic_wait.h (__atomic_wait_address_v): Likewise. |
||
|
Jonathan Wakely
|
bdae7824cd |
libstdc++: Optimize std::counting_semaphore for futex path
Rename __semaphore_base to __semaphore_impl, because it's not used as a base class. Replace the three identical lambda expressions with a named class, __semaphore_impl::_Available, which stores the most recent value of the counter as a data member, and provides call operators that test whether the value is decrementable (i.e. whether the semaphore can be acquired). Add a new __platform_semaphore_impl class template to be used when __platform_wait is available, which uses __platform_wait_t for the counter and uses more efficient atomic waits for the acquire functions. For a binary semaphore some members are further optimized because we know the counter can only be zero or one. Also add a bare wait flag to __atomic_wait_address_v, for consistency with __atomic_wait_address_until_v and __atomic_wait_address_for_v and to allow semaphores to use it without the redundant overhead of tracking waiters. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__atomic_wait_address_v): Add bare wait flag. * include/bits/semaphore_base.h (__semaphore_base): Rename to __semaphore_impl. Replace local variable and predicate lambdas with _Available struct. (__platform_semaphore_impl): New class template. (__semaphore_impl): Remove alias template. (_Select_semaphore_impl): New alias template. * include/std/semaphore (counting_semaphore): Use _Select_semaphore_impl. |
||
|
Jonathan Wakely
|
225622398a |
libstdc++: Define __wait_result_type for atomic waiting
libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h: Use __wait_result_type. * include/bits/atomic_wait.h (__wait_result_type): New struct. (__wait_args::_M_prep_for_wait_on): Rename to _M_setup_wait, use __wait_result_type. (__atomic_wait_address): Adjust to call _M_setup_wait. * src/c++20/atomic.cc (__spin_impl): Use __wait_result_type. (__wait_impl): Likewise. (__spin_until_impl): Likewise. (__wait_until_impl): Likewise. |
||
|
Jonathan Wakely
|
da9b2ea04c |
libstdc++: Move atomic wait/notify entry points into the library
This moves the implementation details of atomic wait/notify functions
into the library, so that only a small API surface is exposed to users.
This also fixes some race conditions present in the design for proxied
waits:
- The stores to _M_ver in __notify_impl must be protected by the mutex,
and the loads from _M_ver in __wait_impl and __wait_until_impl to
check for changes must also be protected by the mutex. This ensures
that checking _M_ver for updates and waiting on the condition_variable
happens atomically. Otherwise it's possible to have: _M_ver == old
happens-before {++_M_ver; cv.notify;} which happens-before cv.wait.
That scenario results in a missed notification, and so the waiting
function never wakes. This wasn't a problem for Linux, because the
futex wait call re-checks the _M_ver value before sleeping, so the
increment cannot interleave between the check and the wait.
- The initial load from _M_ver that reads the 'old' value used for the
_M_ver == old checks must be done before loading and checking the
value of the atomic variable. Otherwise it's possible to have:
var.load() == val happens-before {++_M_ver; _M_cv.notify_all();}
happens-before {old = _M_ver; lock mutex; if (_M_ver == old) cv.wait}.
This results in the waiting thread seeing the already-incremented
value of _M_ver and then waiting for it to change again, which doesn't
happen. This race was present even for Linux, because using a futex
instead of mutex+condvar doesn't prevent the increment from happening
before the waiting threads checks for the increment.
The first race can be solved locally in the waiting and notifying
functions, by acquiring the mutex lock earlier in the function. The
second race cannot be fixed locally, because the load of the atomic
variable and the check for updates to _M_ver happen in different
functions (one in a function template in the headers and one in the
library). We do have an _M_old data member in the __wait_args_base
struct which was previously only used for non-proxy waits using a futex.
We can add a new entry point into the library to look up the waitable
state for the address and then load its _M_ver into the _M_old member.
This allows the inline function template to ensure that loading _M_ver
happens-before testing whether the atomic variable has been changed, so
that we can reliably tell if _M_ver changes after we've already tested
the atomic variable. This isn't 100% reliable, because _M_ver could be
incremented 2^32 times and wrap back to the same value, but that seems
unlikely in practice. If/when we support waiting on user-defined
predicates (which could execute long enough for _M_ver to wrap) we might
want to always wait with a timeout, so that we get a chance to re-check
the predicate even in the rare case that _M_ver wraps.
Another change is to make the __wait_until_impl function take a
__wait_clock_t::duration instead of a __wait_clock_t::time_point, so
that the __wait_until_impl function doesn't depend on the symbol name of
chrono::steady_clock. Inside the library it can be converted back to a
time_point for the clock. This would potentially allow using a different
clock, if we made a different __abi_version in the __wait_args imply
waiting with a different clock.
This also adds a void* to the __wait_args_base structure, so that
__wait_impl can store the __waitable_state* in there the first time it's
looked up for a given wait, so that it doesn't need to be retrieved
again on each loop. This requires passing the __wait_args_base structure
by non-const reference.
The __waitable_state::_S_track function can be removed now that it's all
internal to the library, and namespace-scope RAII types added for
locking and tracking contention.
libstdc++-v3/ChangeLog:
* config/abi/pre/gnu.ver: Add new symbol version and exports.
* include/bits/atomic_timed_wait.h (__platform_wait_until): Move
to atomic.cc.
(__cond_wait_until, __spin_until_impl): Likewise.
(__wait_until_impl): Likewise. Change __wait_args_base parameter
to non-const reference and change third parameter to
__wait_clock_t::duration.
(__wait_until): Change __wait_args_base parameter to non-const
reference. Change Call time_since_epoch() to get duration from
time_point.
(__wait_for): Change __wait_args_base parameter to non-const
reference.
(__atomic_wait_address_until): Call _M_prep_for_wait_on on args.
(__atomic_wait_address_for): Likewise.
(__atomic_wait_address_until_v): Qualify call to avoid ADL. Do
not forward __vfn.
* include/bits/atomic_wait.h (__platform_wait_uses_type): Use
alignof(T) not alignof(T*).
(__futex_wait_flags, __platform_wait, __platform_notify)
(__waitable_state, __spin_impl, __notify_impl): Move to
atomic.cc.
(__wait_impl): Likewise. Change __wait_args_base parameter to
non-const reference.
(__wait_args_base::_M_wait_state): New data member.
(__wait_args_base::_M_prep_for_wait_on): New member function.
(__wait_args_base::_M_load_proxy_wait_val): New member
function.
(__wait_args_base::_S_memory_order_for): Remove member function.
(__atomic_wait_address): Call _M_prep_for_wait_on on args.
(__atomic_wait_address_v): Qualify call to avoid ADL.
* src/c++20/Makefile.am: Add new file.
* src/c++20/Makefile.in: Regenerate.
* src/c++20/atomic.cc: New file.
* testsuite/17_intro/headers/c++1998/49745.cc: Remove XFAIL for
C++20 and later.
* testsuite/29_atomics/atomic/wait_notify/100334.cc: Remove use
of internal implementation details.
* testsuite/util/testsuite_abi.cc: Add GLIBCXX_3.4.35 version.
|
||
|
Jonathan Wakely
|
5b110bbb70 |
libstdc++: Rename __waiter_pool_impl to __waitable_state
The name __waiter_pool_impl is misleading. An object of that type is a member of the pool, not the pool itself, and it's not an "impl" of any abstract base class or generic concept. Just call it __waitable_state since it maintains the state used for waiting/notifying a waitable atomic object. Similarly, rename _S_impl_for to _S_state_for. Once these functions move into the shared library they won't be exported and so the naming won't matter much anyway. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__wait_until_impl): Adjust to use new naming. * include/bits/atomic_wait.h (__waiter_pool_impl): Rename to __waitable_state. (__waiter_pool_impl::_S_wait): Rename to _M_waiters. (__waiter_pool_impl::_S_impl_for): Rename to _S_state_for. (__waiter_pool_impl::_S_track): Adjust to use new naming. (__wait_impl, __notify_impl): Likewise. * testsuite/29_atomics/atomic/wait_notify/100334.cc: Adjust to use new naming. |
||
|
Jonathan Wakely
|
ba56ed29d9 |
libstdc++: Rename __atomic_compare to __atomic_eq
This is an equality comparison rather than a three-way comparison like memcmp and <=>, so name it more precisely. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__atomic_wait_address_until_v): Replace __atomic_compare with __atomic_eq. (__atomic_wait_address_for_v): Likewise. * include/bits/atomic_wait.h (__atomic_compare): Rename to __atomic_eq. (__atomic_wait_address_v): Replace __atomic_compare with __atomic_eq. |
||
|
Jonathan Wakely
|
5729a28e99 |
libstdc++: Remove reinterpret_cast uses in atomic wait/notify
We can pass around void* instead of casting incompatible pointers to __platform_wait_t*, and then only static_cast to __platform_wait_t* when we know that's valid. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__wait_until_impl): Change first parameter to const void* and then static_cast to const __platform_wait_t* when not using proxied wait. (__wait_until): Change first parameter to const void*. (__wait_for): Likewise. (__atomic_wait_address_until): Remove reinterpret_cast and allow address to implicitly convert to const void* instead. (__atomic_wait_address_for): Likewise. * include/bits/atomic_wait.h: (__wait_impl, __notify_impl): Change first parameter to const void* and then static_cast to const __platform_wait_t* when not using proxied wait. (__atomic_wait_address, __atomic_notify_address) Remove reinterpret_cast and allow address to implicitly convert to const void* instead. |
||
|
Jonathan Wakely
|
f52948bf8e |
libstdc++: Simplify futex wrapper functions for atomic wait/notify
libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__platform_wait): Change function template to a normal function. The parameter is always __platform_wait_t* which is just int* for this implementation of the function. (__platform_notify): Likewise. |
||
|
Jonathan Wakely
|
97ea9d7b02 |
libstdc++: Fix race condition in new atomic notify code
When using a proxy object for atomic waiting and notifying operations, we need to ensure that the _M_ver value is always incremented by a notifying operation, even if we return early without doing the futex wake syscall. Otherwise we get missed wake-ups because the notifying thread doesn't modify the value that other threads are doing a futex wait on. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__notify_impl): Increment the proxy value before returning early for the uncontended case. |
||
|
Jonathan Wakely
|
219bb905a6 |
libstdc++: Various fixes for atomic wait/notify code
Pass __wait_args_base by const reference instead of const pointer. I don't see a reason it needs to be passed by pointer to the internals. We can also avoid constructing a __wait_args from __wait_args_base in some places, instaad just using the latter directly. The code using the __wait_flags bitmask type is broken, because the __spin_only constant includes the __do_spin element. This means that testing (__args & __wait_flags::__spin_only) will be inadvertently true when only __do_spin is set. This causes the __wait_until_impl function to never actually wait on the futex (or condition variable), turning all uses of that function into expensive busy spins. Change __spin_only to be a single bit (i.e. a bitmask element) and adjust the places where that bit is set so that they also use the __do_spin element. Update the __args._M_old value when looping in __atomic_wait_address, so that the next wait doesn't fail spuriously. With the new __atomic_wait_address logic, the value function needs to return the correct type, not just a bool. Without that change, the boolean value returned by the value function is used as the value passed to the futex wait, but that mean we're comparing (_M_a == 0) to _M_a and so can block on the futex when we shouldn't, and then never wake up. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__cond_wait_impl): Add missing inline keyword. (__spin_until_impl): Change parameter from pointer to reference. Replace make_pair with list-initialization. Initialize variable for return value. (__wait_until_impl): Likewise. Remove some preprocessor conditional logic. Use _S_track for contention tracking. Avoid unnecessary const_cast. (__wait_until): Change parameter from pointer to reference. Replace make_pair with list-initialization. (__wait_for): Change parameter from pointer to reference. Add __do_spin flag to args. * include/bits/atomic_wait.h (__waiter_pool_impl::_S_track): New function returning an RAII object for contention tracking. (__wait_flags): Do not set the __do_spin flag in the __spin_only enumerator. Comment out the unused __abi_version_mask enumerator. Define operator| and operator|= overloads. (__wait_args_base::operator&): Define. (__wait_args::operator&, __wait_args::_S_default_flags): Remove. (__wait_args::operator|, __wait_args::operator|=): Remove. (__spin_impl): Change parameter from pointer to reference. Replace make_pair call with list-initialization. (__wait_impl): Likewise. Remove some preprocessor conditional logic. Always store old value in __args._M_old. Avoid unnecessary const_cast. Use _S_track. (__notify_impl): Change parameter to reference. Remove some preprocessor conditional logic. (__atomic_wait_address): Add comment. Update __args._M_old on each iteration. (__atomic_wait_address_v): Add comment. * include/std/latch (latch::wait): Adjust predicates for new logic. * testsuite/29_atomics/atomic_integral/wait_notify.cc: Improve test. |
||
|
Jonathan Wakely
|
8e2c4d22fc |
libstdc++: Whitespace fixes in atomic wait/notify code
libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h: Whitespace fixes. * include/bits/atomic_wait.h: Likewise. |
||
|
Thomas Rodgers
|
e071cd6034 |
libstdc++: Pass __wait_args to internal API by const pointer
This change splits the __wait_args data members to a new struct __wait_args_base and then passes that type by const pointer to the low level implementation functions. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__spin_until_impl): Accept __wait_args as const __wait_args_base*. (__wait_until_impl): Likewise. (__wait_until): Likewise. (__wait_for): Likewise. (__atomic_wait_address_until): Pass __wait_args by address. (__atomic_wait_address_for): Likewise. * include/bits/atomic_wait.h (__wait_args_base): New struct. (__wait_args): Derive from __wait_args_base. (__wait_args::__wait_args()): Adjust ctors to call call base ctor. (__wait_args::__wait_args(const __wait_args_base&)): New ctor. (__wait_args::operator|=): New method. (__wait_args::_S_flags_for): Change return type to __wait_flags. (__spin_impl): Accept __wait_args as const __wait_args_base*. (__wait_impl): Likewise. (__notify_impl): Likewise. (__atomic_wait_address): Pass __wait_args by address. (__atomic_wait_address_v): Likewise. (__atomic_notify_address): Likewise. |
||
|
Thomas Rodgers
|
437c147f72 |
libstdc++: Atomic wait/notify ABI stabilization
This represents a major refactoring of the previous atomic::wait and atomic::notify implementation detail. The aim of this change is to simplify the implementation details and position the resulting implementation so that much of the current header-only detail can be moved into the shared library, while also accounting for anticipated changes to wait/notify functionality for C++26. The previous implementation implemented spin logic in terms of the types __default_spin_policy, __timed_backoff_spin_policy, and the free function __atomic_spin. These are replaced in favor of two new free functions; __spin_impl and __spin_until_impl. These currently inline free functions are expected to be moved into the libstdc++ shared library in a future commit. The previous implementation derived untimed and timed wait implementation detail from __detail::__waiter_pool_base. This is-a relationship is removed in the new version and the previous implementation detail is renamed to reflect this change. The static _S_for member has been renamed as well to indicate that it returns the __waiter_pool_impl entry in the static 'side table' for a given awaited address. This new implementation replaces all of the non-templated waiting detail of __waiter_base, __waiter_pool, __waiter, __enters_wait, and __bare_wait with the __wait_impl free function, and the supporting __wait_flags enum and __wait_args struct. This currenly inline free function is expected to be moved into the libstdc++ shared library in a future commit. This new implementation replaces all of the non-templated notifying detail of __waiter_base, __waiter_pool, and __waiter with the __notify_impl free function. This currently inline free function is expected to be moved into the libstdc++ shared library in a future commit. The __atomic_wait_address template function is updated to account for the above changes and to support the expected C++26 change to pass the most recent observed value to the caller supplied predicate. A new non-templated __atomic_wait_address_v free function is added that only works for atomic types that operate only on __platform_wait_t and requires the caller to supply a memory order. This is intended to be the simplest code path for such types. The __atomic_wait_address_v template function is now implemented in terms of new __atomic_wait_address template and continues to accept a user supplied "value function" to retrieve the current value of the atomic. The __atomic_notify_address template function is updated to account for the above changes. The template __platform_wait_until_impl is renamed to __wait_clock_t. The previous __platform_wait_until template is deleted and the functionality previously provided is moved t the new tempalate function __wait_until. A similar change is made to the __cond_wait_until_impl/__cond_wait_until implementation. This new implementation similarly replaces all of the non-templated waiting detail of __timed_waiter_pool, __timed_waiter, etc. with the new __wait_until_impl free function. This currently inline free function is expected to be moved into the libstdc++ shared library in a future commit. This implementation replaces all templated waiting functions that manage clock conversion as well as relative waiting (wait_for) with the new template functions __wait_until and __wait_for. Similarly the previous implementation detail for the various __atomic_wait_address_Xxx templates is adjusted to account for the implementation changes outlined above. All of the "bare wait" versions of __atomic_wait_Xxx have been removed and replaced with a defaulted boolean __bare_wait parameter on the new version of these templates. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h: (__detail::__platform_wait_until_impl): Rename to __platform_wait_until. (__detail::__platform_wait_until): Remove previous definition. (__detail::__cond_wait_until_impl): Rename to __cond_wait_until. (__detail::__cond_wait_until): Remove previous definition. (__detail::__spin_until_impl): New function. (__detail::__wait_until_impl): New function. (__detail::__wait_until): New function. (__detail::__wait_for): New function. (__detail::__timed_waiter_pool): Remove type. (__detail::__timed_backoff_spin_policy): Remove type. (__detail::__timed_waiter): Remove type. (__detail::__enters_timed_wait): Remove type alias. (__detail::__bare_timed_wait): Remove type alias. (__atomic_wait_address_until): Adjust to new implementation detail. (__atomic_wait_address_until_v): Likewise. (__atomic_wait_address_bare): Remove. (__atomic_wait_address_for): Adjust to new implementation detail. (__atomic_wait_address_for_v): Likewise. (__atomic_wait_address_for_bare): Remove. * include/bits/atomic_wait.h: Include bits/stl_pair.h. (__detail::__default_spin_policy): Remove type. (__detail::__atomic_spin): Remove function. (__detail::__waiter_pool_base): Rename to __waiter_pool_impl. Remove _M_notify. Rename _S_for to _S_impl_for. (__detail::__waiter_base): Remove type. (__detail::__waiter_pool): Remove type. (__detail::__waiter): Remove type. (__detail::__enters_wait): Remove type alias. (__detail::__bare_wait): Remove type alias. (__detail::__wait_flags): New enum. (__detail::__wait_args): New struct. (__detail::__wait_result_type): New type alias. (__detail::__spin_impl): New function. (__detail::__wait_impl): New function. (__atomic_wait_address): Adjust to new implementation detail. (__atomic_wait_address_v): Likewise. (__atomic_notify_address): Likewise. (__atomic_wait_address_bare): Delete. (__atomic_notify_address_bare): Likewise. * include/bits/semaphore_base.h: Adjust implementation to use new __atomic_wait_address_v contract. * include/std/barrier: Adjust implementation to use new __atomic_wait contract. * include/std/latch: Adjust implementation to use new __atomic_wait contract. * testsuite/29_atomics/atomic/wait_notify/100334.cc (main): Adjust to for __detail::__waiter_pool_base renaming. |
||
|
Jakub Jelinek
|
6441eb6dc0 | Update copyright years. | ||
|
Jason Merrill
|
63a598deb0 |
libstdc++: #ifdef out #pragma GCC system_header
In r15-3714-gd3a7302ec5985a I added -Wsystem-headers to the libstdc++ build flags to help catch problems in the library. This patch takes a different approach, of disabling the #pragma system_header unless _GLIBCXX_SYSHDR is defined. As a result, the testsuites will treat them as non-system-headers to get better warning coverage during regression testing of both gcc and libstdc++, not just when building the library. My rationale for the #ifdef instead of just removing the #pragma is the three G++ tests that want to test libstdc++ system header behavior, so we need a way to select it. This doesn't affect installed libraries, as they get their system-header status from the lookup path. But testsuite_flags --build-includes gives -I directives rather than -isystem. This patch doesn't change the headers in config/ because I'm not compiling with most of them, so won't see any warnings that need fixing. Adjusting them could happen later, or we can not bother. libstdc++-v3/ChangeLog: * acinclude.m4 (WARN_FLAGS): Remove -Wsystem-headers. * configure: Regenerate. * include/bits/algorithmfwd.h: #ifdef out #pragma GCC system_header. * include/bits/atomic_base.h * include/bits/atomic_futex.h * include/bits/atomic_timed_wait.h * include/bits/atomic_wait.h * include/bits/basic_ios.h * include/bits/basic_string.h * include/bits/boost_concept_check.h * include/bits/char_traits.h * include/bits/charconv.h * include/bits/chrono.h * include/bits/chrono_io.h * include/bits/codecvt.h * include/bits/concept_check.h * include/bits/cpp_type_traits.h * include/bits/elements_of.h * include/bits/enable_special_members.h * include/bits/erase_if.h * include/bits/forward_list.h * include/bits/functional_hash.h * include/bits/gslice.h * include/bits/gslice_array.h * include/bits/hashtable.h * include/bits/indirect_array.h * include/bits/invoke.h * include/bits/ios_base.h * include/bits/iterator_concepts.h * include/bits/locale_classes.h * include/bits/locale_facets.h * include/bits/locale_facets_nonio.h * include/bits/localefwd.h * include/bits/mask_array.h * include/bits/max_size_type.h * include/bits/memory_resource.h * include/bits/memoryfwd.h * include/bits/move_only_function.h * include/bits/node_handle.h * include/bits/ostream_insert.h * include/bits/out_ptr.h * include/bits/parse_numbers.h * include/bits/postypes.h * include/bits/quoted_string.h * include/bits/range_access.h * include/bits/ranges_base.h * include/bits/refwrap.h * include/bits/sat_arith.h * include/bits/semaphore_base.h * include/bits/slice_array.h * include/bits/std_abs.h * include/bits/std_function.h * include/bits/std_mutex.h * include/bits/std_thread.h * include/bits/stl_iterator_base_funcs.h * include/bits/stl_iterator_base_types.h * include/bits/stl_tree.h * include/bits/stream_iterator.h * include/bits/streambuf_iterator.h * include/bits/stringfwd.h * include/bits/this_thread_sleep.h * include/bits/unique_lock.h * include/bits/uses_allocator_args.h * include/bits/utility.h * include/bits/valarray_after.h * include/bits/valarray_array.h * include/bits/valarray_before.h * include/bits/version.h * include/c_compatibility/fenv.h * include/c_compatibility/inttypes.h * include/c_compatibility/stdint.h * include/decimal/decimal.h * include/experimental/bits/net.h * include/experimental/bits/shared_ptr.h * include/ext/aligned_buffer.h * include/ext/alloc_traits.h * include/ext/atomicity.h * include/ext/concurrence.h * include/ext/numeric_traits.h * include/ext/pod_char_traits.h * include/ext/pointer.h * include/ext/stdio_filebuf.h * include/ext/stdio_sync_filebuf.h * include/ext/string_conversions.h * include/ext/type_traits.h * include/ext/vstring.h * include/ext/vstring_fwd.h * include/ext/vstring_util.h * include/parallel/algorithmfwd.h * include/parallel/numericfwd.h * include/tr1/functional_hash.h * include/tr1/hashtable.h * include/tr1/random.h * libsupc++/exception.h * libsupc++/hash_bytes.h * include/bits/basic_ios.tcc * include/bits/basic_string.tcc * include/bits/fstream.tcc * include/bits/istream.tcc * include/bits/locale_classes.tcc * include/bits/locale_facets.tcc * include/bits/locale_facets_nonio.tcc * include/bits/ostream.tcc * include/bits/sstream.tcc * include/bits/streambuf.tcc * include/bits/string_view.tcc * include/bits/version.tpl * include/experimental/bits/string_view.tcc * include/ext/pb_ds/detail/resize_policy/hash_prime_size_policy_imp.hpp * include/ext/random.tcc * include/ext/vstring.tcc * include/tr2/bool_set.tcc * include/tr2/dynamic_bitset.tcc * include/bits/c++config * include/c/cassert * include/c/cctype * include/c/cerrno * include/c/cfloat * include/c/ciso646 * include/c/climits * include/c/clocale * include/c/cmath * include/c/csetjmp * include/c/csignal * include/c/cstdarg * include/c/cstddef * include/c/cstdio * include/c/cstdlib * include/c/cstring * include/c/ctime * include/c/cuchar * include/c/cwchar * include/c/cwctype * include/c_global/cassert * include/c_global/ccomplex * include/c_global/cctype * include/c_global/cerrno * include/c_global/cfenv * include/c_global/cfloat * include/c_global/cinttypes * include/c_global/ciso646 * include/c_global/climits * include/c_global/clocale * include/c_global/cmath * include/c_global/csetjmp * include/c_global/csignal * include/c_global/cstdalign * include/c_global/cstdarg * include/c_global/cstdbool * include/c_global/cstddef * include/c_global/cstdint * include/c_global/cstdio * include/c_global/cstdlib * include/c_global/cstring * include/c_global/ctgmath * include/c_global/ctime * include/c_global/cuchar * include/c_global/cwchar * include/c_global/cwctype * include/c_std/cassert * include/c_std/cctype * include/c_std/cerrno * include/c_std/cfloat * include/c_std/ciso646 * include/c_std/climits * include/c_std/clocale * include/c_std/cmath * include/c_std/csetjmp * include/c_std/csignal * include/c_std/cstdarg * include/c_std/cstddef * include/c_std/cstdio * include/c_std/cstdlib * include/c_std/cstring * include/c_std/ctime * include/c_std/cuchar * include/c_std/cwchar * include/c_std/cwctype * include/debug/array * include/debug/bitset * include/debug/deque * include/debug/forward_list * include/debug/list * include/debug/map * include/debug/set * include/debug/string * include/debug/unordered_map * include/debug/unordered_set * include/debug/vector * include/decimal/decimal * include/experimental/algorithm * include/experimental/any * include/experimental/array * include/experimental/buffer * include/experimental/chrono * include/experimental/contract * include/experimental/deque * include/experimental/executor * include/experimental/filesystem * include/experimental/forward_list * include/experimental/functional * include/experimental/internet * include/experimental/io_context * include/experimental/iterator * include/experimental/list * include/experimental/map * include/experimental/memory * include/experimental/memory_resource * include/experimental/net * include/experimental/netfwd * include/experimental/numeric * include/experimental/propagate_const * include/experimental/ratio * include/experimental/regex * include/experimental/scope * include/experimental/set * include/experimental/socket * include/experimental/string * include/experimental/string_view * include/experimental/synchronized_value * include/experimental/system_error * include/experimental/timer * include/experimental/tuple * include/experimental/type_traits * include/experimental/unordered_map * include/experimental/unordered_set * include/experimental/vector * include/ext/algorithm * include/ext/cmath * include/ext/functional * include/ext/iterator * include/ext/memory * include/ext/numeric * include/ext/random * include/ext/rb_tree * include/ext/rope * include/parallel/algorithm * include/std/algorithm * include/std/any * include/std/array * include/std/atomic * include/std/barrier * include/std/bit * include/std/bitset * include/std/charconv * include/std/chrono * include/std/codecvt * include/std/complex * include/std/concepts * include/std/condition_variable * include/std/coroutine * include/std/deque * include/std/execution * include/std/expected * include/std/filesystem * include/std/format * include/std/forward_list * include/std/fstream * include/std/functional * include/std/future * include/std/generator * include/std/iomanip * include/std/ios * include/std/iosfwd * include/std/iostream * include/std/istream * include/std/iterator * include/std/latch * include/std/limits * include/std/list * include/std/locale * include/std/map * include/std/memory * include/std/memory_resource * include/std/mutex * include/std/numbers * include/std/numeric * include/std/optional * include/std/ostream * include/std/print * include/std/queue * include/std/random * include/std/ranges * include/std/ratio * include/std/regex * include/std/scoped_allocator * include/std/semaphore * include/std/set * include/std/shared_mutex * include/std/span * include/std/spanstream * include/std/sstream * include/std/stack * include/std/stacktrace * include/std/stdexcept * include/std/streambuf * include/std/string * include/std/string_view * include/std/syncstream * include/std/system_error * include/std/text_encoding * include/std/thread * include/std/tuple * include/std/type_traits * include/std/typeindex * include/std/unordered_map * include/std/unordered_set * include/std/utility * include/std/valarray * include/std/variant * include/std/vector * include/std/version * include/tr1/array * include/tr1/cfenv * include/tr1/cinttypes * include/tr1/cmath * include/tr1/complex * include/tr1/cstdbool * include/tr1/cstdint * include/tr1/cstdio * include/tr1/cstdlib * include/tr1/cwchar * include/tr1/cwctype * include/tr1/functional * include/tr1/memory * include/tr1/random * include/tr1/regex * include/tr1/tuple * include/tr1/type_traits * include/tr1/unordered_map * include/tr1/unordered_set * include/tr1/utility * include/tr2/bool_set * include/tr2/dynamic_bitset * include/tr2/type_traits * libsupc++/atomic_lockfree_defines.h * libsupc++/compare * libsupc++/cxxabi.h * libsupc++/cxxabi_forced.h * libsupc++/cxxabi_init_exception.h * libsupc++/exception * libsupc++/initializer_list * libsupc++/new * libsupc++/typeinfo: Likewise. * testsuite/20_util/ratio/operations/ops_overflow_neg.cc * testsuite/23_containers/array/tuple_interface/get_neg.cc * testsuite/23_containers/vector/cons/destructible_debug_neg.cc * testsuite/24_iterators/operations/prev_neg.cc * testsuite/ext/type_traits/add_unsigned_floating_neg.cc * testsuite/ext/type_traits/add_unsigned_integer_neg.cc * testsuite/ext/type_traits/remove_unsigned_floating_neg.cc * testsuite/ext/type_traits/remove_unsigned_integer_neg.cc: Adjust line numbers. gcc/testsuite/ChangeLog * g++.dg/analyzer/fanalyzer-show-events-in-system-headers-default.C * g++.dg/analyzer/fanalyzer-show-events-in-system-headers-no.C * g++.dg/diagnostic/disable.C: #define _GLIBCXX_SYSHDR. |
||
|
Jonathan Wakely
|
3a817a4a5a |
libstdc++: Remove unnecessary uses of <stdint.h>
We don't need to include all of <stdint.h> when we only need uintptr_t from it. By using GCC's internal macro we avoid unnecessarily declaring everything in <stdint.h>. This helps users to avoid accidentally relying on those names being declared without explicitly including the header. libstdc++-v3/ChangeLog: * include/bits/align.h (align, assume_aligned): Use __UINTPTR_TYPE__ instead of uintptr_t. Do not include <stdint.h>. * include/bits/atomic_base.h (__atomic_ref): Likewise. * include/bits/atomic_wait.h (__waiter_pool_base::_S_for): Likewise. * include/std/atomic: Include <cstdint>. |
||
|
Jakub Jelinek
|
a945c346f5 | Update copyright years. | ||
|
Jonathan Wakely
|
2f512f6fcd |
libstdc++: Use <cstdint> instead of <stdint.h> in <bits/atomic_wait.h>
In r14-5922-g6c8f2d3a08bc01 I added <stdint.h> to <bits/atomic_wait.h>,
so that uintptr_t is declared if that header is compiled as a header
unit. I used <stdint.h> because that's what <atomic> already includes,
so it seemed simpler to be consistent. However, this means that name
lookup for uintptr_t in <bits/atomic_wait.h> depends on whether
<cstdint> has been included by another header first. Whether name lookup
finds std::uintptr_t or ::uintptr_t will depend on include order. This
causes problems when compiling modules with Clang:
bits/atomic_wait.h:251:7: error: 'std::__detail::__waiter_pool_base' has different definitions in different modules; first difference is defined here found method '_S_for' with body
_S_for(const void* __addr) noexcept
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bits/atomic_wait.h:251:7: note: but in 'tm.<global>' found method '_S_for' with different body
_S_for(const void* __addr) noexcept
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By including <cstdint> we would ensure that name lookup always finds the
name in namespace std. Alternatively, we can stop including <stdint.h>
for those types, so that we don't declare the entire contents of
<stdint.h> when we only need a couple of types from it. This patch does
the former, which is appropriate for backporting.
libstdc++-v3/ChangeLog:
* include/bits/atomic_wait.h: Include <cstdint> instead of
<stdint.h>.
|
||
|
Jonathan Wakely
|
6c8f2d3a08 |
libstdc++: Include <stdint.h> in <bits/atomic_wait.h>
This is needed in order to compile it as a header-unit, which might be desired because it's included by both <atomic> and <semaphore>. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h: Include <stdint.h>. |
||
|
Jonathan Wakely
|
7ffa63df8f |
libstdc++: Only declare feature test macros in standard headers
This change moves the definitions of feature test macros (or strictly speaking, the requests for <bits/version.h> to define them) so that only standard headers define them. For example, <bits/shared_ptr.h> will no longer define macros related to std::shared_ptr, only <memory> and <version> will define them. This means that __cpp_lib_shared_ptr_arrays will not be defined by <future> or by other headers that include <bits/shared_ptr.h>. It will only be defined when <memory> has been included. This will discourage users from relying on transitive includes. As a result, internal headers that need to query the macros should use the internal macros like __glibcxx_shared_ptr_arrays instead of __cpp_lib_shared_ptr_arrays, as those internal macros are defined by the internal headers after icluding <bits/version.h>. There are some exceptions to this rule, because __cpp_lib_is_constant_evaluated is defined by bits/c++config.h and so is available everywhere, and __cpp_lib_three_way_comparison is defined by <compare> which several headers are explicitly specified to include, so its macro is guaranteed to be usable too. N.B. not many internal headers actually need an explicit include of <bits/version.h>, because most of them include <type_traits> and so get all the __glibcxx_foo internal macros from there. libstdc++-v3/ChangeLog: * include/bits/algorithmfwd.h: Do not define standard feature test macro here. * include/bits/align.h: Likewise. Test internal macros instead of standard macros. * include/bits/alloc_traits.h: Likewise. * include/bits/allocator.h: Likewise. * include/bits/atomic_base.h: Likewise. * include/bits/atomic_timed_wait.h: Likewise. * include/bits/atomic_wait.h: Likewise. * include/bits/basic_string.h: Likewise. * include/bits/basic_string.tcc: Likewise. * include/bits/char_traits.h: Likewise. * include/bits/chrono.h: Likewise. * include/bits/cow_string.h: Likewise. * include/bits/forward_list.h: Likewise. * include/bits/hashtable.h: Likewise. * include/bits/ios_base.h: Likewise. * include/bits/memory_resource.h: Likewise. * include/bits/move.h: Likewise. * include/bits/move_only_function.h: Likewise. * include/bits/node_handle.h: Likewise. * include/bits/ptr_traits.h: Likewise. * include/bits/range_access.h: Likewise. * include/bits/ranges_algo.h: Likewise. * include/bits/ranges_cmp.h: Likewise. * include/bits/ranges_util.h: Likewise. * include/bits/semaphore_base.h: Likewise. * include/bits/shared_ptr.h: Likewise. * include/bits/shared_ptr_atomic.h: Likewise. * include/bits/shared_ptr_base.h: Likewise. * include/bits/stl_algo.h: Likewise. * include/bits/stl_algobase.h: Likewise. * include/bits/stl_function.h: Likewise. * include/bits/stl_iterator.h: Likewise. * include/bits/stl_list.h: Likewise. * include/bits/stl_map.h: Likewise. * include/bits/stl_pair.h: Likewise. * include/bits/stl_queue.h: Likewise. * include/bits/stl_stack.h: Likewise. * include/bits/stl_tree.h: Likewise. * include/bits/stl_uninitialized.h: Likewise. * include/bits/stl_vector.h: Likewise. * include/bits/unique_ptr.h: Likewise. * include/bits/unordered_map.h: Likewise. * include/bits/uses_allocator_args.h: Likewise. * include/bits/utility.h: Likewise. * include/bits/erase_if.h: Add comment. * include/std/algorithm: Define standard feature test macros here. * include/std/atomic: Likewise. * include/std/array: Likewise. * include/std/chrono: Likewise. * include/std/condition_variable: Likewise. * include/std/deque: Likewise. * include/std/format: Likewise. * include/std/functional: Likewise. * include/std/forward_list: Likewise. * include/std/ios: Likewise. * include/std/iterator: Likewise. * include/std/list: Likewise. * include/std/map: Likewise. * include/std/memory: Likewise. * include/std/numeric: Likewise. * include/std/queue: Likewise. * include/std/ranges: Likewise. * include/std/regex: Likewise. * include/std/set: Likewise. * include/std/stack: Likewise. * include/std/stop_token: Likewise. * include/std/string: Likewise. * include/std/string_view: * include/std/tuple: Likewise. * include/std/unordered_map: * include/std/unordered_set: * include/std/utility: Likewise. * include/std/vector: Likewise. * include/std/scoped_allocator: Query internal macros instead of standard macros. |
||
|
Arsen Arsenović
|
083b7f2833 |
libstdc++: Replace all manual FTM definitions and use
libstdc++-v3/ChangeLog:
* libsupc++/typeinfo: Switch to bits/version.h for
__cpp_lib_constexpr_typeinfo.
* libsupc++/new: Switch to bits/version.h for
__cpp_lib_{launder,hardware_interference_size,destroying_delete}.
(launder): Guard behind __cpp_lib_launder.
(hardware_destructive_interference_size)
(hardware_constructive_interference_size): Guard behind
__cpp_lib_hardware_interference_size.
* libsupc++/exception: Switch to bits/version.h for
__cpp_lib_uncaught_exceptions.
(uncaught_exceptions): Guard behind __cpp_lib_uncaught_exceptions.
* libsupc++/compare: Switch to bits/version.h for
__cpp_lib_three_way_comparison.
(three_way_comparable, three_way_comparable_with)
(compare_three_way, weak_order, strong_order, partial_order):
Guard behind __cpp_lib_three_way_comparison >= 201907L.
* include/std/chrono: Drop __cpp_lib_chrono definition.
* include/std/vector: Switch to bits/version.h for
__cpp_lib_erase_if.
(erase, erase_if): Guard behind __cpp_lib_erase_if.
* include/std/variant: Switch to bits/version.h for
__cpp_lib_variant. Guard whole header behind that FTM.
* include/std/utility: Switch to bits/version.h for
__cpp_lib_{exchange_function,constexpr_algorithms,as_const},
__cpp_lib_{integer_comparison_functions,to_underlying}, and
__cpp_lib_unreachable.
(exchange): Guard behind __cpp_lib_exchange_function.
(cmp_equal, cmp_not_equal, cmp_less, cmp_greater, cmp_less_equal)
(cmp_greater_equal, in_range): Guard behind
__cpp_lib_integer_comparison_functions.
(to_underlying): Guard behind __cpp_lib_to_underlying.
(unreachable): Guard behind __cpp_lib_unreachable.
* include/std/type_traits: Switch to bits/version.h for
__cpp_lib_is_{null_pointer,final,nothrow_convertible,aggregate},
__cpp_lib_is_{constant_evaluated,invocable,layout_compatible},
__cpp_lib_is_{pointer_interconvertible,scoped_enum,swappable},
__cpp_lib_{logical_traits,reference_from_temporary,remove_cvref},
__cpp_lib_{result_of_sfinae,transformation_trait_aliases},
__cpp_lib_{type_identity,type_trait_variable_templates},
__cpp_lib_{unwrap_ref,void_t,integral_constant_callable},
__cpp_lib_{bool_constant,bounded_array_traits}, and
__cpp_lib_has_unique_object_representations.
(integral_constant::operator()): Guard behind
__cpp_lib_integral_constant_callable.
(bool_constant): Guard behind __cpp_lib_bool_constant.
(conjunction, disjunction, negation, conjunction_v, disjunction_v)
(negation_v): Guard behind __cpp_lib_logical_traits.
(is_null_pointer): Guard behind __cpp_lib_is_null_pointer.
(is_final): Guard behind __cpp_lib_is_final.
(is_nothrow_convertible, is_nothrow_convertible_v): Guard behind
__cpp_lib_is_nothrow_convertible.
(remove_const_t, remove_volatile_t, remove_cv_t)
(add_const_t, add_volatile_t, add_cv_t): Guard behind
__cpp_lib_transformation_trait_aliases.
(void_t): Guard behind __cpp_lib_void_t.
(is_swappable_with_v, is_nothrow_swappable_with_v)
(is_swappable_with, is_nothrow_swappable_with): Guard behind
__cpp_lib_is_swappable.
(is_nothrow_invocable_r, is_invocable_r, invoke_result)
(is_invocable, invoke_result_t): Guard behind
__cpp_lib_is_invocable.
(alignment_of_v, extent_v, has_virtual_destructor_v)
(is_abstract_v, is_arithmetic_v, is_array_v)
(is_assignable_v, is_base_of_v, is_class_v, is_compound_v)
(is_constructible_v, is_const_v, is_convertible_v)
(is_copy_assignable_v, is_copy_constructible_v)
(is_default_constructible_v, is_destructible_v)
(is_empty_v, is_enum_v, is_final_v, is_floating_point_v)
(is_function_v, is_fundamental_v, is_integral_v)
(is_invocable_r_v, is_invocable_v, is_literal_type_v)
(is_lvalue_reference_v, is_member_function_pointer_v)
(is_member_object_pointer_v, is_member_pointer_v)
(is_move_assignable_v, is_move_constructible_v)
(is_nothrow_assignable_v, is_nothrow_constructible_v)
(is_nothrow_copy_assignable_v, is_nothrow_copy_constructible_v)
(is_nothrow_default_constructible_v, is_nothrow_destructible_v)
(is_nothrow_invocable_r_v, is_nothrow_invocable_v)
(is_nothrow_move_assignable_v, is_nothrow_move_constructible_v)
(is_null_pointer_v, is_object_v, is_pod_v, is_pointer_v)
(is_polymorphic_v, is_reference_v, is_rvalue_reference_v)
(is_same_v, is_scalar_v, is_signed_v, is_standard_layout_v)
(is_trivially_assignable_v, is_trivially_constructible_v)
(is_trivially_copyable_v, is_trivially_copy_assignable_v)
(is_trivially_copy_constructible_v)
(is_trivially_default_constructible_v)
(is_trivially_destructible_v, is_trivially_move_assignable_v)
(is_trivially_move_constructible_v, is_trivial_v, is_union_v)
(is_unsigned_v, is_void_v, is_volatile_v, rank_v, as variadic):
Guard behind __cpp_lib_type_trait_variable_templates.
(has_unique_object_representations)
(has_unique_object_representations_v): Guard behind
__cpp_lib_has_unique_object_representation.
(is_aggregate): Guard behind __cpp_lib_is_aggregate.
(remove_cvref, remove_cvref_t): Guard behind
__cpp_lib_remove_cvref.
(type_identity, type_identity_t): Guard behind
__cpp_lib_type_identity.
(unwrap_reference, unwrap_reference_t, unwrap_ref_decay)
(unwrap_ref_decay_t): Guard behind __cpp_lib_unwrap_ref.
(is_bounded_array_v, is_unbounded_array_v, is_bounded_array)
(is_unbounded_array): Guard behind __cpp_lib_bounded_array_traits.
(is_scoped_enum, is_scoped_enum_v): Guard behind
__cpp_lib_is_scoped_enum.
(reference_constructs_from_temporary)
(reference_constructs_from_temporary_v): Guard behind
__cpp_lib_reference_from_temporary.
* include/std/tuple: Switch to bits/version.h for
__cpp_lib_{constexpr_tuple,tuple_by_type,apply_make_from_tuple}.
(get<T>): Guard behind __cpp_lib_tuple_by_type.
(apply): Guard behind __cpp_lib_apply.
(make_from_tuple): Guard behind __cpp_lib_make_from_tuple.
* include/std/syncstream: Switch to bits/version.h for
__cpp_lib_syncbuf. Guard header behind that FTM.
* include/std/string_view: Switch to bits/version.h for
__cpp_lib_{string_{view,contains},constexpr_string_view} and
__cpp_lib_starts_ends_with.
(basic_string_view::starts_with, basic_string_view::ends_with):
Guard behind __cpp_lib_starts_ends_with.
[C++23 && _GLIBCXX_HOSTED && !defined(__cpp_lib_string_contains)]:
Assert as impossible ithout a bug in C++23.
* include/std/string: Switch to bits/version.h for
__cpp_lib_erase_if.
(erase, erase_if): Guard behind __cpp_lib_erase_if.
* include/std/thread: Switch to bits/version.h for
__cpp_lib_jthread.
* include/std/stop_token: Switch to bits/version.h for
__cpp_lib_jthread.
* include/std/spanstream: Switch to bits/version.h for
__cpp_lib_spanstream. Guard header behind that FTM.
* include/std/span: Switch to bits/version.h for __cpp_lib_span.
Guard header behind that FTM.
* include/std/source_location: Switch to bits/version.h for
__cpp_lib_source_location. Guard header with that FTM.
* include/std/shared_mutex: Switch to bits/version.h for
__cpp_lib_shared{,_timed}_mutex.
(shared_mutex): Guard behind __cpp_lib_shared_mutex.
* include/std/semaphore: Switch to bits/version.h for
__cpp_lib_semaphore. Guard header behind that FTM.
* include/std/ranges: Switch to bits/version.h for
__cpp_lib_ranges_{zip,chunk{,_by},slide,join_with},
__cpp_lib_ranges_{repeat_stride,cartesian_product,as_rvalue},
and __cpp_lib_ranges_{as_const,enumerate,iota}.
(ranges::zip et al, ranges::chunk et al, ranges::slide et al)
(ranges::chunk_by et al, ranges::join_with et al)
(ranges::stride et al, ranges::cartesian_product et al)
(ranges::as_rvalue et al, ranges::as_const et al)
(ranges::enumerate et al): Guard behind appropriate FTM.
* include/std/optional: Switch to bits/version.h for
__cpp_lib_optional. Guard header behind that FTM.
* include/std/numeric: Switch to bits/version.h for
__cpp_lib_{gcd{,_lcm},lcm,constexpr_numeric,interpolate}
and __cpp_lib_parallel_algorithm.
(gcd, lcm): Guard behind __cpp_lib_gcd_lcm.
(midpoint): Guard behind __cpp_lib_interpolate.
* include/std/numbers: Switch to bits/version.h for
__cpp_lib_math_constants. Guard header behind that FTM.
* include/std/mutex: Switch to bits/version.h for
__cpp_lib_scoped_lock.
(scoped_Lock): Guard behind __cpp_lib_scoped_lock.
* include/std/memory_resource: Switch to bits/version.h for
__cpp_lib_{polymorphic_allocator,memory_resource}.
(synchronized_pool_resource): Guard behind
__cpp_lib_memory_resource >= 201603L.
(polymorphic_allocator): Guard behind
__cpp_lib_polymorphic_allocator.
* include/std/memory: Switch to bits/version.h for
__cpp_lib_{parallel_algorithm,atomic_value_initialization}.
* include/std/list: Switch to bits/version.h for
__cpp_lib_erase_if.
(erase, erase_if): Guard behind __cpp_lib_erase_if.
* include/std/latch: Switch to bits/version.h for __cpp_lib_latch.
Guard header behind that FTM.
* include/std/iterator: Switch to bits/version.h for
__cpp_lib_null_iterators.
* include/std/iomanip: Switch to bits/version.h for
__cpp_lib_quoted_string_io.
(quoted): Guard behind __cpp_lib_quoted_string_io.
* include/std/functional: Switch to bits/version.h for
__cpp_lib_{invoke{,_r},constexpr_functional,bind_front} and
__cpp_lib_{not_fn,booyer_moore_searcher}.
(invoke): Guard behind __cpp_lib_invoke.
(invoke_r): Guard behind __cpp_lib_invoke_r.
(bind_front): Guard behind __cpp_lib_bind_front.
(not_fn): Guard behind __cpp_lib_not_fn.
(boyer_moore_searcher, boyer_moore_horspool_searcher): Guard
definition behind __cpp_lib_boyer_moore_searcher.
* include/std/forward_list: Switch to bits/version.h for
__cpp_lib_erase_if.
(erase, erase_if): Guard behind __cpp_lib_erase_if.
* include/std/format: Switch to bits/version.h for
__cpp_lib_format. Guard header behind that FTM.
* include/std/filesystem: Switch to bits/version.h for
__cpp_lib_filesystem. Guard header behind that FTM.
* include/std/expected: Switch to bits/version.h for
__cpp_lib_expected. Guard header behind it.
* include/std/execution: Switch to bits/version.h for
__cpp_lib_{execution,parallel_algorithm}. Guard header behind
either.
* include/std/deque: Switch to bits/version.h for
__cpp_lib_erase_if.
(erase, erase_if): Guard behind __cpp_lib_erase_if.
* include/std/coroutine: Switch to bits/version.h for
__cpp_lib_coroutine. Guard header behind that FTM.
* include/std/concepts: Switch to bits/version.h for
__cpp_lib_concepts. Guard header behind that FTM.
* include/std/complex: Switch to bits/version.h for
__cpp_lib_{complex_udls,constexpr_complex}.
(operator""if, operator""i, operator""il): Guard behind
__cpp_lib_complex_udls.
* include/std/charconv: Swtich to bits/version.h for
__cpp_lib_{to_chars,constexpr_charconv}.
* include/std/bitset: Switch to bits/version.h for
__cpp_lib_constexpr_bitset.
* include/std/bit: Switch to bits/version.h for
__cpp_lib_{bit_cast,byteswap,bitops,int_pow2,endian}.
(bit_cast): Guard behind __cpp_lib_bit_cast.
(byteswap): Guard behind __cpp_lib_byteswap.
(rotl, rotr, countl_zero, countl_one, countr_zero, countr_one)
(popcount): Guard behind __cpp_lib_bitops.
(has_single_bit, bit_ceil, bit_floor, bit_width): Guard behind
__cpp_lib_int_pow2.
(endian): Guard behind __cpp_lib_endian.
* include/std/barrier: Switch to bits/version.h for
__cpp_lib_barrier. Guard header behind that FTM.
* include/std/atomic: Switch to bits/version.h for
__cpp_lib_atomic_{is_always_lock_free,float,ref}
and __cpp_lib_lock_free_type_aliases.
(*::is_always_lock_free): Guard behind
__cpp_lib_atomic_is_always_lock_free.
(atomic<float>): Guard behind __cpp_lib_atomic_float.
(atomic_ref): Guard behind __cpp_lib_atomic_ref.
(atomic_signed_lock_free, atomic_unsigned_lock_free): Guard behind
__cpp_lib_atomic_lock_free_type_aliases.
* include/std/array: Switch to bits/version.h for
__cpp_lib_to_array.
(to_array): Guard behind __cpp_lib_to_array.
* include/std/any: Switch to bits/version.h for __cpp_lib_any.
Guard header behind that FTM.
* include/std/algorithm: Switch to bits/version.h for
__cpp_lib_parallel_algorithm.
* include/c_global/cstddef: Switch to bits/version.h for
__cpp_lib_byte.
(byte): Guard behind __cpp_lib_byte.
* include/c_global/cmath: Switch to bits/version.h for
__cpp_lib_{hypot,interpolate}.
(hypot3): Guard behind __cpp_lib_hypot.
(lerp): Guard behind __cpp_lib_interpolate.
* include/c_compatibility/stdatomic.h: Switch to
bits/stl_version.h for __cpp_lib_atomic. Guard header behind that
FTM.
* include/bits/utility.h: Switch to bits/version.h for
__cpp_lib_{tuple_element_t,integer_sequence,ranges_zip}.
(tuple_element_t): Guard behind __cpp_lib_tuple_element_t.
(integer_sequence et al): Guard behind __cpp_lib_integer_sequence.
* include/bits/uses_allocator_args.h: Switch to bits/version.h for
__cpp_lib_make_obj_using_allocator. Guard header behind that FTM.
* include/bits/unordered_map.h: Switch to bits/version.h for
__cpp_lib_unordered_map_try_emplace.
(try_emplace): Guard behind __cpp_lib_unordered_map_try_emplace.
* include/bits/unique_ptr.h: Switch to bits/version.h for
__cpp_lib_{constexpr_memory,make_unique}.
(make_unique): Guard behind __cpp_lib_make_unique.
* include/bits/stl_vector.h: Switch to bits/version.h for
__cpp_lib_constexpr_vector.
* include/bits/stl_uninitialized.h: Switch to bits/version.h for
__cpp_lib_raw_memory_algorithms.
(uninitialized_default_construct)
(uninitialized_default_construct_n, uninitialized_move)
(uninitialized_move_n, uninitialized_value_construct)
(uninitialized_value_construct_n): Guard behind
__cpp_lib_raw_memory_algorithms.
* include/bits/stl_tree.h: Switch to bits/version.h for
__cpp_lib_generic_associative_lookup.
* include/bits/stl_stack.h: Switch to bits/version.h for
__cpp_lib_adaptor_iterator_pair_constructor.
(stack): Guard iterator-pair constructor behind
__cpp_lib_adaptor_iterator_pair_constructor.
* include/bits/stl_queue.h: Switch to bits/version.h for
__cpp_lib_adaptor_iterator_pair_constructor.
(queue): Guard iterator-pair constructor behind
__cpp_lib_adaptor_iterator_pair_constructor.
* include/bits/stl_pair.h: Switch to bits/version.h for
__cpp_lib_{concepts,tuples_by_type}.
(get): Guard type-getting overloads behind
__cpp_lib_tuples_by_type.
* include/bits/stl_map.h: Switch to bits/version.h for
__cpp_lib_map_try_emplace.
(map<>::try_emplace): Guard behind __cpp_lib_map_try_emplace.
* include/bits/stl_list.h: Switch to bits/version.h for
__cpp_lib_list_remove_return_type.
(__remove_return_type, _GLIBCXX_LIST_REMOVE_RETURN_TYPE_TAG)
[C++20]: guard behind __cpp_lib_list_remove_return_type instead.
* include/bits/stl_iterator.h: Switch to bits/version.h for
__cpp_lib_{constexpr_iterator,array_constexpr} and
__cpp_lib_{make_reverse_iterator,move_iterator_concept}.
(make_reverse_iterator): Guard behind
__cpp_lib_make_reverse_iterator.
(iterator_concept et al): Guard __cpp_lib_move_iterator_concept
changes behind that FTM.
* include/bits/stl_function.h: Switch to bits/version.h for
__cpp_lib_transparent_operators.
(equal_to, not_equal_to, greater, less, greater_equal)
(less_equal, bit_and, bit_or, bit_xor, bit_not, logical_and)
(logical_or, logical_not, plus, minus, multiplies, divides)
(modulus, negate): Guard '= void' fwdecls behind
__cpp_lib_transparent_operators.
(plus<void>, minus<void>, multiplies<void>, divides<void>)
(modulus<void>, negate<void>, logical_and<void>, logical_or<void>)
(logical_not<void>, bit_and<void>, bit_or<void>, bit_xor<void>)
(equal_to<void>, not_equal_to<void>, greater<void>, less<void>)
(greater_equal<void>, less_equal<void>, bit_not<void>)
(__has_is_transparent): Guard behind
__cpp_lib_transparent_operators.
* include/bits/stl_algobase.h: Switch to bits/version.h for
__cpp_lib_robust_nonmodifying_seq_ops.
(robust equal, mismatch): Guard behind
__cpp_lib_nonmember_container_access.
* include/bits/stl_algo.h: Swtich to bits/version.h for
__cpp_lib_{clamp,sample}.
(clamp): Guard behind __cpp_lib_clamp.
(sample): Guard behind __cpp_lib_sample.
* include/bits/specfun.h: Switch to bits/version.h for
__cpp_lib_math_special_functions and __STDCPP_MATH_SPEC_FUNCS__.
* include/bits/shared_ptr_base.h: Switch to bits/version.h for
__cpp_lib_{smart_ptr_for_overwrite,shared_ptr_arrays}.
(_Sp_overwrite_tag): Guard behind
__cpp_lib_smart_ptr_for_overwrite.
* include/bits/shared_ptr_atomic.h: Switch to bits/version.h for
__cpp_lib_atomic_shared_ptr.
* include/bits/shared_ptr.h: Switch to bits/version.h for
__cpp_lib_{enable_shared_from_this,shared_ptr_weak_type}.
(shared_ptr<T>::weak_type): Guard behind
__cpp_lib_shared_ptr_weak_type.
(enable_shared_from_this<T>::weak_from_this): Guard behind
__cpp_lib_enable_shared_from_this.
* include/bits/ranges_cmp.h: Switch to bits/version.h for
__cpp_lib_ranges.
* include/bits/ranges_algo.h: Switch to bits/version.h for
__cpp_lib_{shift,ranges_{contains,find_last,fold,iota}}.
* include/bits/range_access.h: Switch to bits/version.h for
__cpp_lib_nonmember_container_access
(size, empty, data): Guard behind
__cpp_lib_nonmember_container_access.
(ssize): Guard behind __cpp_lib_ssize.
* include/bits/ptr_traits.h: Switch to bits/version.h. for
__cpp_lib_{constexpr_memory,to_address}.
(to_address): Guard behind __cpp_lib_to_address.
* include/bits/node_handle.h: Switch to bits/version.h for
__cpp_lib_node_extract. Guard header behind that FTM.
* include/bits/move_only_function.h: Switch to bits/version.h for
__cpp_lib_move_only_function. Guard header behind that FTM.
* include/bits/move.h: Switch to bits/version.h for
__cpp_lib_addressof_constexpr.
* include/bits/ios_base.h: Switch to bits/version.h for
__cpp_lib_ios_noreplace.
(noreplace): Guard with __cpp_lib_ios_noreplace.
* include/bits/hashtable.h: Switch to bits/version.h for
__cpp_lib_generic_unordered_lookup.
(_M_equal_range_tr, _M_count_tr, _M_find_tr): Guard behind
__cpp_lib_generic_unordered_lookup.
* include/bits/forward_list.h: Switch to bits/version.h for
__cpp_lib_list_remove_return_type.
(__remove_return_type): Guard behind
__cpp_lib_list_remove_return_type.
* include/bits/erase_if.h: Switch to bits/version.h for
__cpp_lib_erase_if.
* include/bits/cow_string.h: Switch to bits/version.h for
__cpp_lib_constexpr_string.
* include/bits/chrono.h: Swtich to bits/version.h for
__cpp_lib_chrono{,_udls}.
(ceil): Guard behind __cpp_lib_chrono.
(operator""ns et al): Guard behind __cpp_lib_chrono_udls.
* include/bits/char_traits.h: Switch to bits/version.h for
__cpp_lib_constexpr_char_traits.
* include/bits/basic_string.h: Switch to bits/version.h for
__cpp_lib_{constexpr_string,string_{resize_and_overwrite,udls}}.
(resize_and_overwrite): Guard behind
__cpp_lib_string_resize_and_overwrite.
(operator""s): Guard behind __cpp_lib_string_udls.
* include/bits/atomic_wait.h: Switch to bits/version.h for
__cpp_lib_atomic_wait. Guard header behind that FTM.
* include/bits/atomic_base.h: Switch to bits/version.h for
__cpp_lib_atomic_value_initialization and
__cpp_lib_atomic_flag_test.
(atomic_flag::test): Guard behind __cpp_lib_atomic_flag_test,
rather than C++20.
* include/bits/allocator.h: Switch to bits/version.h for
__cpp_lib_incomplete_container_elements.
* include/bits/alloc_traits.h: Switch to using bits/version.h for
__cpp_lib_constexpr_dynamic_alloc and
__cpp_lib_allocator_traits_is_always_equal.
* include/bits/align.h: Switch to bits/version.h for defining
__cpp_lib_assume_aligned.
(assume_aligned): Guard with __cpp_lib_assume_aligned.
* include/bits/algorithmfwd.h: Switch to bits/version.h for
defining __cpp_lib_constexpr_algorithms.
* include/std/stacktrace: Switch to bits/version.h for
__cpp_lib_stacktrace. Guard header behind that FTM.
* testsuite/23_containers/array/tuple_interface/get_neg.cc:
Update line numbers.
|
||
|
Jakub Jelinek
|
83ffe9cde7 | Update copyright years. | ||
|
Jonathan Wakely
|
dc22cf4612 |
libstdc++: Use lock-free type for __platform_wait_t
For non-futex targets the __platform_wait_t type is currently uint64_t, but that requires a lock in libatomic for some 32-bit targets. We don't really need a 64-bit type, so use unsigned long if that is lock-free, and int otherwise. This should mean it's lock-free on a wider set of targets. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__detail::__platform_wait_t): Define as unsigned long if always lock-free, and unsigned int otherwise. |
||
|
Patrick Palka
|
e3b1024911 |
libstdc++: Declare const global variables inline
The changes inside the regex_constants and execution namespaces seem to be (the only) unimplemented parts of P0607R0 "Inline Variable for the Standard Library"; the rest of the changes are to implementation details. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (_detail::__platform_wait_alignment): Declare inline. Remove redundant static specifier. (__detail::__atomic_spin_count_relax): Declare inline. (__detail::__atomic_spin_count): Likewise. * include/bits/regex_automaton.h (__detail::_S_invalid_state_id): Declare inline for C++17. Declare constexpr. Remove redundant const and static specifiers. * include/bits/regex_error.h (regex_constants::error_collate): Declare inline for C++17 as per P0607R0. (regex_constants::error_ctype): Likewise. (regex_constants::error_escape): Likewise. (regex_constants::error_backref): Likewise. (regex_constants::error_brack): Likewise. (regex_constants::error_paren): Likewise. (regex_constants::error_brace): Likewise. (regex_constants::error_badbrace): Likewise. (regex_constants::error_range): Likewise. (regex_constants::error_space): Likewise. (regex_constants::error_badrepeat): Likewise. (regex_constants::error_complexity): Likewise. (regex_constants::error_stack): Likewise. * include/ext/concurrence.h (__gnu_cxx::__default_lock_policy): Likewise. Remove redundant static specifier. * include/pstl/execution_defs.h (execution::seq): Declare inline for C++17 as per P0607R0. (execution::par): Likewise. (execution::par_unseq): Likewise. (execution::unseq): Likewise. |
||
|
Jonathan Wakely
|
af98cb88eb |
libstdc++: Unblock atomic wait on non-futex platforms [PR106183]
When using a mutex and condition variable, the notifying thread needs to increment _M_ver while holding the mutex lock, and the waiting thread needs to re-check after locking the mutex. This avoids a missed notification as described in the PR. By moving the increment of _M_ver to the base _M_notify we can make the use of the mutex local to the use of the condition variable, and simplify the code a little. We can use a relaxed store because the mutex already provides sequential consistency. Also we don't need to check whether __addr == &_M_ver because we know that's always true for platforms that use a condition variable, and so we also know that we always need to use notify_all() not notify_one(). Reviewed-by: Thomas Rodgers <trodgers@redhat.com> libstdc++-v3/ChangeLog: PR libstdc++/106183 * include/bits/atomic_wait.h (__waiter_pool_base::_M_notify): Move increment of _M_ver here. [!_GLIBCXX_HAVE_PLATFORM_WAIT]: Lock mutex around increment. Use relaxed memory order and always notify all waiters. (__waiter_base::_M_do_wait) [!_GLIBCXX_HAVE_PLATFORM_WAIT]: Check value again after locking mutex. (__waiter_base::_M_notify): Remove increment of _M_ver. |
||
|
Thomas Rodgers
|
e75da2ace6 |
libstdc++: Minor codegen improvement for atomic wait spinloop
This patch merges the spin loops in the atomic wait implementation which is a minor codegen improvement. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__atomic_spin): Merge spin loops. |
||
|
Thomas Rodgers
|
b25a7c05f1 |
libstdc++: Strengthen memory order for atomic<T>::wait/notify
This changes the memory order used in the spin wait code to match that of libc++. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__waiter_base::_S_do_spin, __waiter_base::_S_do_spin_v): Change memory order from relaxed to acquire. |
||
|
Thomas Rodgers
|
4cf3c33981 |
libstdc++: Fix deadlock in atomic wait [PR104442]
This issue was observed as a deadlock in 29_atomics/atomic/wait_notify/100334.cc on vxworks. When a wait is "laundered" (e.g. type T* does not suffice as a waitable address for the platform's native waiting primitive), the address waited is that of the _M_ver member of __waiter_pool_base, so several threads may wait on the same address for unrelated atomic<T> objects. As noted in the PR, the implementation correctly exits the wait for the thread whose data changed, but not for any other threads waiting on the same address. As noted in the PR the __waiter::_M_do_wait_v member was correctly exiting but the other waiters were not reloading the value of _M_ver before re-entering the wait. Moving the spin call inside the loop accomplishes this, and is consistent with the predicate accepting version of __waiter::_M_do_wait. libstdc++-v3/ChangeLog: PR libstdc++/104442 * include/bits/atomic_wait.h (__waiter::_M_do_wait_v): Move spin loop inside do loop so that threads failing the wait, reload _M_ver. |
||
|
Thomas Rodgers
|
07a971b28c |
Strengthen memory order for atomic<T>::wait/notify
This matches the memory order in libc++. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h: Change memory order from Acquire/Release with relaxed loads to SeqCst+Release for accesses to the waiter's count. |
||
|
Jakub Jelinek
|
7adcbafe45 | Update copyright years. | ||
|
Jonathan Wakely
|
0e90799071 |
libstdc++: Avoid use of hardware interference non-constant [PR102377]
libstdc++-v3/ChangeLog: PR libstdc++/102377 * include/bits/atomic_wait.h (__waiter_pool_base:_S_align): Hardcode to 64 instead of using non-constant constant. |
||
|
Jonathan Wakely
|
aeaea265ce |
libstdc++: Define macro before it is first checked
Signed-off-by: Jonathan Wakely <jwakely@redhat.com> libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (_GLIBCXX_HAVE_PLATFORM_WAIT): Define before first attempt to check it. |
||
|
Thomas Rodgers
|
346cbaf578 |
libstdc++: Fix wrong thread waking on notify [PR100334]
Fixes https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100334 libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__waiter::_M_do_wait_v): loop until value change observed. (__waiter_base::_M_laundered): New member. (__waiter_base::_M_notify): Check _M_laundered to determine whether to wake one or all. (__detail::__atomic_compare): Return true if call to __builtin_memcmp() == 0. (__waiter_base::_S_do_spin_v): Adjust predicate. * testsuite/29_atomics/atomic/wait_notify/100334.cc: New test. |
||
|
Thomas Rodgers
|
ae2f6e0174 |
libstdc++: Fix "bare" notifications dropped by waiters check
For types that track whether or not there extant waiters (e.g. semaphore) internally, the __atomic_notify_address_bare() call was introduced to avoid the overhead of loading the atomic count of waiters. For platforms that don't have Futex, however, there was still a check for waiters, and seeing that there are none (because in the bare case, the count is not incremented), the notification is dropped. This commit addresses that case. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h: Always notify waiters in the case of 'bare' address notification. |
||
|
Thomas Rodgers
|
b52aef3a8c |
libstdc++: Refactor/cleanup of C++20 atomic wait implementation
This is a substantial rewrite of the atomic wait/notify (and timed wait
counterparts) implementation.
The previous __platform_wait looped on EINTR however this behavior is
not required by the standard. A new _GLIBCXX_HAVE_PLATFORM_WAIT macro
now controls whether wait/notify are implemented using a platform
specific primitive or with a platform agnostic mutex/condvar. This
patch only supplies a definition for linux futexes. A future update
could add support __ulock_wait/wake on Darwin, for instance.
The members of __waiters were lifted to a new base class. The members
are now arranged such that overall sizeof(__waiter_pool_base) fits in
two cache lines (on platforms with at least 64 byte cache lines). The
definition will also use destructive_interference_size for this if it is
available.
The __waiters type is now specific to untimed waits, and is renamed to
__waiter_pool. Timed waits have a corresponding __timed_waiter_pool
type. Much of the code has been moved from the previous __atomic_wait()
free function to the __waiter_base template and a __waiter derived type
is provided to implement the un-timed wait operations. A similar change
has been made to the timed wait implementation.
The __atomic_spin code has been extended to take a spin policy which is
invoked after the initial busy wait loop. The default policy is to
return from the spin. The timed wait code adds a timed backoff spinning
policy. The code from <thread> which implements this_thread::sleep_for,
sleep_until has been moved to a new <bits/std_thread_sleep.h> header
which allows the thread sleep code to be consumed without pulling in the
whole of <thread>.
The entry points into the wait/notify code have been restructured to
support either -
* Testing the current value of the atomic stored at the given address
and waiting on a notification.
* Applying a predicate to determine if the wait was satisfied.
The entry points were renamed to make it clear that the wait and wake
operations operate on addresses. The first variant takes the expected
value and a function which returns the current value that should be used
in comparison operations, these operations are named with a _v suffix
(e.g. 'value'). All atomic<_Tp> wait/notify operations use the first
variant. Barriers, latches and semaphores use the predicate variant.
This change also centralizes what it means to compare values for the
purposes of atomic<T>::wait rather than scattering through individual
predicates.
This change also centralizes the repetitive code which adjusts for
different user supplied clocks (this should be moved elsewhere
and all such adjustments should use a common implementation).
This change also removes the hashing of the pointer and uses
the pointer value directly for indexing into the waiters table.
libstdc++-v3/ChangeLog:
* include/Makefile.am: Add new <bits/this_thread_sleep.h> header.
* include/Makefile.in: Regenerate.
* include/bits/this_thread_sleep.h: New file.
* include/bits/atomic_base.h: Adjust all calls
to __atomic_wait/__atomic_notify for new call signatures.
* include/bits/atomic_timed_wait.h: Extensive rewrite.
* include/bits/atomic_wait.h: Likewise.
* include/bits/semaphore_base.h: Adjust all calls
to __atomic_wait/__atomic_notify for new call signatures.
* include/std/atomic: Likewise.
* include/std/barrier: Likewise.
* include/std/latch: Likewise.
* include/std/semaphore: Likewise.
* include/std/thread (this_thread::sleep_for)
(this_thread::sleep_until): Move to new header.
* testsuite/29_atomics/atomic/wait_notify/bool.cc: Simplify
test.
* testsuite/29_atomics/atomic/wait_notify/generic.cc: Likewise.
* testsuite/29_atomics/atomic/wait_notify/pointers.cc: Likewise.
* testsuite/29_atomics/atomic_flag/wait_notify/1.cc: Likewise.
* testsuite/29_atomics/atomic_float/wait_notify.cc: Likewise.
* testsuite/29_atomics/atomic_integral/wait_notify.cc: Likewise.
* testsuite/29_atomics/atomic_ref/wait_notify.cc: Likewise.
|
||
|
Jonathan Wakely
|
9d449189ee |
libstdc++: Fix __thread_relax for non-gthreads non-x86 targets
My recent change to the preprocessor conditions in __thread_relax() was supposed to also change the __gthread_yield() call to __thread_yield(), which has the right preprocessor checks. Instead I just removed the check for _GLIBCXX_USE_SCHED_YIELD which means the __gthread_yield() call will be ill-formed for non-gthreads targets, and targets without sched_yield(). This fixes it properly. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__thread_relax()): Call __thread_yield() not __gthread_yield(). |
||
|
Jonathan Wakely
|
cc9a0a3d79 |
libstdc++: Fix __thread_yield for non-gthreads targets
The __gthread_yield() function is only defined for gthreads targets, so check _GLIBCXX_HAS_GTHREADS before using it. Also reorder __thread_relax and __thread_yield so that the former can use the latter instead of repeating the same preprocessor checks. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__thread_yield()): Check _GLIBCXX_HAS_GTHREADS before using __gthread_yield. (__thread_relax()): Use __thread_yield() instead of repeating the preprocessor checks for __gthread_yield. |
||
|
Jakub Jelinek
|
99dee82307 | Update copyright years. | ||
|
Thomas Rodgers
|
6591e42269 |
Add feature test macro for atomic<T>::wait
Adds __cpp_lib_atomic_wait feature test macro which was overlooked in the initial commit of this feature. Replaces uses of _GLIBCXX_HAVE_ATOMIC_WAIT. libstdc++-v3/ChangeLog: * include/bits/atomic_base.h: Replace usage of _GLIBCXX_HAVE_ATOMIC_WAIT with __cpp_lib_atomic_wait. * include/bits/atomic_timed_wait.h: Likewise. * include/bits/atomic_wait.h: Define __cpp_lib_atomic_wait feature test macro. * include/bits/semaphore_base.h: Replace usage of _GLIBCXX_HAVE_ATOMIC_WAIT with __cpp_lib_atomic_wait. * include/std/atomic: Likewise. * include/std/latch: Likewise. * include/std/semaphore: Likewise. * include/std/version: Define __cpp_lib_atomic wait feature test macro and replace usage of _GLIBCXX_HAVE_ATOMIC_WAIT. * testsuite/29_atomics/atomic/wait_notify/1.cc: New test. * testsuite/29_atomics/atomic/wait_notify/2.cc: Likewise. |
||
|
Jonathan Wakely
|
61c71a6245 |
libstdc++: Define (and use) _GLIBCXX_HAVE_ATOMIC_WAIT
In order to simplify the preprocessor checks for whether __atomic_wait is available, this commit does: -#if defined _GLIBCXX_HAS_GTHREADS || _GLIBCXX_HAVE_LINUX_FUTEX +#ifdef _GLIBCXX_HAVE_ATOMIC_WAIT The original was wrong anyway, as it should have used 'defined' to check _GLIBCXX_HAVE_LINUX_FUTEX (for consistency with how that's used elsewhere). The new macro is defined in <bits/atomic_wait.h> when the file is defines __atomic_wait and related facilities. All other code that depends on those features can just check the one macro. libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (_GLIBCXX_HAVE_ATOMIC_WAIT): Define. * include/bits/atomic_base.h: Check _GLIBCXX_HAVE_ATOMIC_WAIT. * include/bits/atomic_timed_wait.h: Likewise. * include/bits/semaphore_base.h: Likewise. * include/std/atomic: Likewise. * include/std/latch: Likewise. * include/std/semaphore: Likewise. |
||
|
Jonathan Wakely
|
7d2a98a727 |
libstdc++: Encapsulate __gthread_cond_t as std::__condvar
This introduces a new internal utility, std::__condvar, which is a simplified form of std::condition_variable. It has no dependency on <chrono> or std::unique_lock, which allows it to be used in <bits/atomic_wait.h>. This avoids repeating the #ifdef __GTHREAD_COND_INIT preprocessor conditions and associated logic for initializing a __gthread_cond_t correctly. It also encapsulates most of the __gthread_cond_xxx functions as member functions of __condvar. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__cond_wait_until_impl): Do not define when _GLIBCXX_HAVE_LINUX_FUTEX is defined. Use __condvar and mutex instead of __gthread_cond_t and unique_lock<mutex>. (__cond_wait_until): Likewise. Fix test for return value of __cond_wait_until_impl. (__timed_waiters::_M_do_wait_until): Use __condvar instead of __gthread_cond_t. * include/bits/atomic_wait.h: Remove <bits/unique_lock.h> include. Only include <bits/std_mutex.h> if not using futexes. (__platform_wait_max_value): Remove unused variable. (__waiters::lock_t): Use lock_guard instead of unique_lock. (__waiters::_M_cv): Use __condvar instead of __gthread_cond_t. (__waiters::_M_do_wait(__platform_wait_t)): Likewise. (__waiters::_M_notify()): Likewise. Use notify_one() if not asked to notify all. * include/bits/std_mutex.h (__condvar): New type. * include/std/condition_variable (condition_variable::_M_cond) (condition_variable::wait_until): Use __condvar instead of __gthread_cond_t. * src/c++11/condition_variable.cc (condition_variable): Define default constructor and destructor as defaulted. (condition_variable::wait, condition_variable::notify_one) (condition_variable::notify_all): Forward to corresponding member function of __condvar. |
||
|
Jonathan Wakely
|
a5ccfd0460 |
libstdc++: Fix silly typos [PR 97936]
libstdc++-v3/ChangeLog: PR libstdc++/97936 * include/bits/atomic_wait.h (__platform_wait): Check errno, not just the value of EAGAIN. (__waiters::__waiters()): Fix name of data member. |
||
|
Jonathan Wakely
|
ad9cbcee54 |
libstdc++: Fix handling of futex wake [PR 97936]
The __platform_wait function is supposed to wait until *addr != old. The futex syscall checks the initial value and returns EAGAIN if *addr != old is already true, which should cause __platform_wait to return. Instead it loops and keeps doing a futex wait, which keeps returning EAGAIN. libstdc++-v3/ChangeLog: PR libstdc++/97936 * include/bits/atomic_wait.h (__platform_wait): Return if futex sets EAGAIN. * testsuite/30_threads/latch/3.cc: Re-enable test. * testsuite/30_threads/semaphore/try_acquire_until.cc: Likewise. |
||
|
Stephan Bergmann
|
0986d3bc62 |
libstdc++: Fix linker errors due to missing 'inline' keywords
libstdc++-v3/ChangeLog: * include/bits/atomic_wait.h (__thread_relax, __thread_yield): Add 'inline'. |