This mechanical patch removes _Executor's __dfs_mode template parameter.
In passing we can also get rid of the _V2 inline namespace because this
removal alone achieve the goal of the inline namespace which is to make
all of _Executor's member function signatures different from the
previous (GCC < 16) recursive implementation, allowing for safe mixing
of the two incompatible implementations.
libstdc++-v3/ChangeLog:
* include/bits/regex.h (_Executor): Remove __dfs_mode template
parameter and _V2 inline namespace.
* include/bits/regex.tcc (__regex_algo_impl): Adjust after
__dfs_mode template parameter removal.
* include/bits/regex_executor.h (_Executor): Remove __dfs_mode
parameter and _V2 inline namespace.
* include/bits/regex_executor.tcc (_Executor): Likewise.
Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com>
Reviewed-by: Jonathan Wakely <jwakely@redhat.com>
That _Executor's __dfs_mode flag is a template parameter means
__regex_algo_impl has to instantiate two different (but largely the
same) versions of _Executor even though the BFS version is rarely used.
And it seems the compiler has trouble DCEing the unused version of
_Executor, which needlessly increases code size and burdens the optimizer.
This patch replaces the template parameter with a run-time data member
_M_search_mode. We in turn need to inline the _Executor::_State_info
member functions and data members into _Executor so that they depend on
the run-time instead of compile-time flag. This means _Executor is
three pointers bigger in DFS mode (due to the unused _M_match_queue and
_M_visited_states members), which we can reduce to one pointer if we
really want to, but that doesn't seem ver worthwhile.
The __dfs_mode template parameter is now unused but not yet removed,
that'll be done in the next patch.
After this patch, both code size and run time for the microbenchmark
for (int i = 0; i < 10000; i++)
regex_match(string(200, 'a'), regex("(a|b|c)*"));
decreases by about 15% each (with -O2). Compile time/memory use decreases
by 5-10%.
libstdc++-v3/ChangeLog:
* include/bits/regex.tcc (__regex_algo_impl): Pass __use_dfs
parameter to _Executor's constructor.
* include/bits/regex_executor.h (_Executor::_Search_mode): New.
(_Executor::_Executor): Add __use_dfs parameter and initialize
_M_search_mode. Adjust after inlining _State_info members into
_Executor.
(_Executor::~_Executor): Free _M_visted_states.
(_Executor::_M_main): Adjust after renaming _M_main_dispatch
overloads to _M_main_dfs and _M_main_bfs.
(_Executor::_State_info): Remove.
(_Executor::_M_visited): Inlined from _State_info.
(_Executor::_M_get_sol_pos): Likewise.
(_Executor::_M_states): Remove.
(_Executor::_M_start): Inlined from _State_info.
(_Executor::_M_sol_pos): Likewise.
(_Executor::_M_match_queue): Likewise.
(_Executor::_M_search_mode): New.
* include/bits/regex_executor.tcc (_Executor::_M_main_dispatch):
Renamed to...
(_Executor::_M_main_dfs, _Executor::_M_main_bfs): ... these.
(_Executor::_M_*): Adjust after _M_states removal.
(_Executor::_M_lookhead): Also adjust _Executor constructor
call.
Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com>
Reviewed-by: Jonathan Wakely <jwakely@redhat.com>
This patch replaces the recursive implementation of the DFS executor
with an iterative one using an explicit heap-based stack. System
stack usage of the executor is now constant with respect to input size
rather than linear, avoding stack overflow errors when processing
long inputs.
PR libstdc++/86164
libstdc++-v3/ChangeLog:
* include/bits/regex.h (__detail::_Executor): Use inline
namespace _V2.
* include/bits/regex_executor.h (__detail::_ExecutorFrame):
Declare.
(__detail::_Executor): Use inline namespace _V2.
(__detail::_Executor::_M_node): Declare.
(__detail::_Executor::_M_frames): New data member.
* include/bits/regex_executor.tcc (__detail::_ExecutorFrameOpcode):
New.
(__detail::_ExecutorFrameBase): New.
(__detail::_ExecutorFrame): New.
(__detail::_Executor): Use inline namespace _V2.
(__detail::_Executor::_M_rep_once_more): Replace recursive
_M_dfs calls with an _S_opcode_next frame push, and any work
after such calls with an appropriate frame push.
(__detail::_M_handle_repeat): Likewise.
(__detail::_M_handle_subexpr_begin): Likewise.
(__detail::_M_handle_subexpr_end): Likewise.
(__detail::_M_handle_line_begin_assertion): Likewise.
(__detail::_M_handle_line_end_assertion): Likewise.
(__detail::_M_handle_word_boundary): Likewise.
(__detail::_M_handle_subexpr_lookahead): Likewise.
(__detail::_M_handle_match): Likewise.
(__detail::_M_handle_backref): Likewise.
(__detail::_M_handle_accept): Likewise.
(__detail::_M_handle_alternative): Likewise.
(__detail::_M_node): Factored out from _M_dfs.
(__detail::_M_dfs): Push an initial frame to _M_frames that
visits the starting node and pass this stack each subroutine.
Pop the latest _ExecutorFrame from _M_frames and handle
appropriately according to its _ExecutorFrameOpcode. Loop until
_M_frames is empty.
Reviewed-by: Tomasz Kamiński <tkaminsk@redhat.com>
Reviewed-by: Jonathan Wakely <jwakely@redhat.com>
The PR points out that we assume the match_results allocator is default
constuctible, which might not be true. We also have a related issue with
unwanted propagation from an object that might have an unequal
allocator.
Ideally we use the same allocator type for _State_info::_M_match_queue
but that would be an ABI change now. We should investigate if that can
be done without breaking anything, which might be possible because the
_Executor object is short-lived and never leaks out of the regex_match,
regex_search, and regex_replace algorithms. If we change the mangled
name for _Executor then there would be no ODR violations when mixing old
and new definitions. This commit does not attempt that.
libstdc++-v3/ChangeLog:
PR libstdc++/107376
* include/bits/regex_executor.h (_Executor::_Executor): Use same
allocator for _M_cur_results and _M_results.
* include/bits/regex_executor.tcc (_Executor::_M_main_dispatch):
Prevent possibly incorrect allocator propagating to
_M_cur_results.
* testsuite/28_regex/algorithms/regex_match/107376.cc: New test.
This implements LWG 2503, which allows ^ and $ to match line terminator
characters, rather than only matching the beginning and end of the
entire input. The multiline option is only valid for ECMAScript, but
for other grammars we ignore it rather than throwing an exception.
This is related to PR libstdc++/102480, which incorrectly said that
ECMAscript should match the beginning of a line when match_prev_avail
is used. I think that's only supposed to happen when multiline is used.
The new regex_constants::multiline and basic_regex::multiline constants
are not defined for strict -std=c++11 and -std=c++14 modes, but
regex_constants::__multiline is always defined, so that the
implementation can use it internally.
Signed-off-by: Jonathan Wakely <jwakely@redhat.com>
libstdc++-v3/ChangeLog:
* include/bits/regex.h (basic_regex::multiline): Define constant
for C++17.
* include/bits/regex_constants.h (regex_constants::multiline):
Define constant for C++17.
(regex_constants::__multiline): Define duplicate constant for
internal use in C++11 and C++14.
* include/bits/regex_executor.h (_Executor::_M_match_multiline()):
New member function.
(_Executor::_M_is_line_terminator(_CharT)): New member function.
(_Executor::_M_at_begin(), _Executor::_M_at_end()): Use new
member functions to support multiline matches.
* testsuite/28_regex/algorithms/regex_match/multiline.cc: New test.
The std::regex code uses std::map and std::vector, which means that when
_GLIBCXX_DEBUG is defined it uses the debug versions of those
containers. That no longer compiles, because I changed <regex> to
include <bits/stl_map.h> and <bits/stl_vector.h> instead of <map> and
<vector>, so the debug versions aren't defined, and std::map doesn't
compile. There is also a use of std::stack, which defaults to std::deque
which is the debug deque when _GLIBCXX_DEBUG is defined.
Using std::map, std::vector, and std::deque is probably a mistake, and
we should qualify them with _GLIBCXX_STD_C instead so that the debug
versions aren't used. We do not need the overhead of checking our own
uses of those containers, which should be correct anyway. The exception
is the vector base class of std::match_results, which exposes iterators
to users, so can benefit from debug mode checks for its iterators. For
other accesses to the vector elements, match_results already does its
own checks, so can access the _GLIBCXX_STD_C::vector base class
directly.
Signed-off-by: Jonathan Wakely <jwakely@redhat.com>
libstdc++-v3/ChangeLog:
* include/bits/regex.h (basic_regex::transform_primary): Use
_GLIBCXX_STD_C::vector for local variable.
* include/bits/regex.tcc (__regex_algo_impl): Use reference to
_GLIBCXX_STD_C::vector base class of match_results.
* include/bits/regex_automaton.tcc (_StateSeq:_M_clone): Use
_GLIBCXX_STD_C::map and _GLIBCXX_STD_C::deque for local
variables.
* include/bits/regex_compiler.h (_BracketMatcher): Use
_GLIBCXX_STD_C::vector for data members.
* include/bits/regex_executor.h (_Executor): Likewise.
* include/std/regex [_GLIBCXX_DEBUG]: Include <debug/vector>.
This reduces the size of <regex> a little. This is one of the largest
and slowest headers in the library.
By using <bits/stl_algobase.h> and <bits/stl_algo.h> instead of
<algorithm> we don't need to parse all the parallel algorithms and
std::ranges:: algorithms that are not needed by <regex>. Similarly, by
using <bits/stl_tree.h> and <bits/stl_map.h> instead of <map> we don't
need to parse the definition of std::multimap.
The _State_info type is not movable or copyable, so doesn't need to use
std::unique_ptr<bool[]> to manage a bitset, we can just delete it in the
destructor. It would use a lot less space if we used a bitset instead,
but that would be an ABI break. We could do it for the versioned
namespace, but this patch doesn't do so. For future reference, using
vector<bool> would work, but would increase sizeof(_State_info) by two
pointers, because it's three times as large as unique_ptr<bool[]>. We
can't use std::bitset because the length isn't constant. We want a
bitset with a non-constant but fixed length.
Signed-off-by: Jonathan Wakely <jwakely@redhat.com>
libstdc++-v3/ChangeLog:
* include/bits/regex_executor.h (_State_info): Replace
unique_ptr<bool[]> with array of bool.
* include/bits/regex_executor.tcc: Likewise.
* include/bits/regex_scanner.tcc: Replace std::strchr with
__builtin_strchr.
* include/std/regex: Replace standard headers with smaller
internal ones.
* testsuite/28_regex/traits/char/lookup_classname.cc: Include
<string.h> for strlen.
* testsuite/28_regex/traits/char/lookup_collatename.cc:
Likewise.
* include/bits/regex_automaton.h (_State_base, _State<>):
Remove _TraitsT dependency from _State<>; Make matcher member
into the union to reduce struct size.
* include/bits/regex_automaton.tcc (_State_base<>::_M_print,
_State_base<>::_M_dot, _StateSeq<>::_M_clone):
Adjust to fit the interface. Factor out common parts in
_M_clone as _State<>::_M_has_alt.
* include/bits/regex_executor.h (_Executer<>::_M_lookahead):
Only pass state id instead of the whole state.
* include/bits/regex_executor.tcc (_Executer<>::_M_dfs,
_Executer<>::_M_lookahead): Adjust to fit the interface.
* include/std/regex: Include <ext/aligned_buffer.h>
From-SVN: r226395
PR libstdc++/63920
* include/bits/regex_executor.h: Make _M_begin non const.
* include/bits/regex_executor.tcc (_Executor<>::_M_search): Increase
_M_begin in search algorithm, so that _M_begin is treated as
"current start position" for each search iteration.
* testsuite/28_regex/algorithms/regex_search/ecma/flags.cc: New
testcase.
From-SVN: r218037
PR libstdc++/61424
* include/bits/regex.tcc (__regex_algo_impl<>): Use DFS for ECMAScript,
not just regex containing back-references.
* include/bits/regex_compiler.tcc (_Compiler<>::_M_disjunction):
exchange _M_next and _M_alt for alternative operator,
making matching from left to right.
* include/bits/regex_executor.h (_State_info<>::_M_get_sol_pos):
Add position tracking fom DFS.
* include/bits/regex_executor.tcc (_Executor<>::_M_main_dispatch,
_Executor<>::_M_dfs): Likewise.
* include/bits/regex_scanner.h: Remove unused enum entry.
* testsuite/28_regex/algorithms/regex_search/61424.cc: New
testcase from PR.
From-SVN: r212184
* include/bits/regex_compiler.h (__detail::_BracketMatcher): Reorder
members to avoid wasted space when not using a cache.
(__detail::_BracketMatcher::_M_ready()): Sort and deduplicate set.
* include/bits/regex_compiler.tcc
(__detail::_BracketMatcher::_M_apply(_CharT, false_type)): Use binary
search on set.
* include/bits/regex_executor.h (__detail::_Executor::_Match_mode):
New enumeration type to indicate match mode.
(__detail::_Executor::_State_info): New type holding members only
needed in BFS-mode. Replace unique_ptr<vector<bool>> with
unique_ptr<bool[]>.
(__detail::_Executor::_M_rep_once_more, __detail::_Executor::_M_dfs):
Replace template parameter with run-time function parameter.
(__detail::_Executor::_M_main): Likewise. Dispatch to ...
(__detail::_Executor::_M_main_dispatch): New overloaded functions to
implement DFS and BFS mode.
* include/bits/regex_executor.tcc (__detail::_Executor::_M_main):
Split implementation into ...
(__detail::_Executor::_M_main_dispatch): New overloaded functions.
(__detail::_Executor::_M_lookahead): Create nested executor on stack.
(__detail::_Executor::_M_rep_once_more): Pass match mode as function
argument instead of template argument.
(__detail::_Executor::_M_dfs): Likewise.
* include/bits/regex_scanner.tcc: Fix typos in comments.
* testsuite/performance/28_regex/range.cc: New.
From-SVN: r211143
2014-04-27 Tim Shen <timshen91@gmail.com>
* include/bits/regex_automaton.h (_NFA<>::_M_insert_repeat):
Add _S_opcode_repeat support to distingush a loop from
_S_opcode_alternative.
* include/bits/regex_automaton.tcc (_State_base::_M_print,
_State_base::_M_dot, _NFA<>::_M_eliminate_dummy,
_StateSeq<>::_M_clone): Likewise.
* include/bits/regex_compiler.tcc (_Compiler<>::_M_quantifier):
Likewise.
* include/bits/regex_executor.tcc (_Executor<>::_M_dfs): Likewise.
* include/bits/regex_scanner.tcc (_Scanner<>::_M_eat_escape_ecma):
Uglify local variable __i.
* include/bits/regex_compiler.h (_BracketMatcher<>::_M_make_cache):
Use size_t instead of int to compare with vector::size().
2014-04-27 Tim Shen <timshen91@gmail.com>
* include/bits/regex_executor.h: Add _M_rep_count to track how
many times this repeat node are visited.
* include/bits/regex_executor.tcc (_Executor<>::_M_rep_once_more,
_Executor<>::_M_dfs): Use _M_rep_count to prevent entering
infinite loop.
2014-04-27 Tim Shen <timshen91@gmail.com>
* include/bits/regex.tcc (__regex_algo_impl<>): Remove
_GLIBCXX_REGEX_DFS_QUANTIFIERS_LIMIT and use
_GLIBCXX_REGEX_USE_THOMPSON_NFA instead.
* include/bits/regex_automaton.h: Remove quantifier counting variable.
* include/bits/regex_automaton.tcc (_State_base::_M_dot):
Adjust debug NFA dump.
From-SVN: r209844
2014-01-17 Tim Shen <timshen91@gmail.com>
* include/bits/regex_automaton.tcc (_StateSeq<>::_M_clone()): Do not
use std::map.
* include/bits/regex_automaton.h: Do not use std::set.
* include/bits/regex_compiler.h (_BracketMatcher<>::_M_add_char(),
_BracketMatcher<>::_M_add_collating_element(),
_BracketMatcher<>::_M_add_equivalence_class(),
_BracketMatcher<>::_M_make_range()): Likewise.
* include/bits/regex_compiler.tcc (_BracketMatcher<>::_M_apply()):
Likewise.
* include/bits/regex_executor.h: Do not use std::queue.
* include/bits/regex_executor.tcc (_Executor<>::_M_main(),
_Executor<>::_M_dfs()): Likewise.
* include/std/regex: Remove <map>, <set> and <queue>.
2014-01-17 Tim Shen <timshen91@gmail.com>
* include/bits/regex.h (__compile_nfa<>(), basic_regex<>::basic_regex(),
basic_regex<>::assign()): Change __compile_nfa to accept
const _CharT* only.
* include/bits/regex_compiler.h: Change _Compiler's template
argument from <_FwdIter, _TraitsT> to <_TraitsT>.
* include/bits/regex_compiler.tcc: Likewise.
2014-01-17 Tim Shen <timshen91@gmail.com>
* include/bits/regex_compiler.h: Change _ScannerT into char-type
templated.
* include/bits/regex_scanner.h (_Scanner<>::_Scanner()): Separate
_ScannerBase from _Scanner; Change _Scanner's template argument from
_FwdIter to _CharT. Avoid use of std::map and std::set by using arrays
instead.
* include/bits/regex_scanner.tcc (_Scanner<>::_Scanner(),
_Scanner<>::_M_scan_normal(), _Scanner<>::_M_eat_escape_ecma(),
_Scanner<>::_M_eat_escape_posix(), _Scanner<>::_M_eat_escape_awk()):
Likewise.
* include/std/regex: Add <cstring> for using strchr.
2014-01-17 Tim Shen <timshen91@gmail.com>
* bits/regex_automaton.tcc: Indentation fix.
* bits/regex_compiler.h (__compile_nfa<>(), _Compiler<>,
_RegexTranslator<> _AnyMatcher<>, _CharMatcher<>,
_BracketMatcher<>): Add bool option template parameters and
specializations to make matching more efficient and space saving.
* bits/regex_compiler.tcc: Likewise.
From-SVN: r206690
* include/bits/regex_automaton.h (__detail::_State): Split
non-dependent parts into new _State_base.
(__detail::_NFA): Likewise for _NFA_base. Use std::move() to avoid
copies when inserting _MatcherT and _StateT objects.
* include/bits/regex_automaton.tcc: Move member definitions to base
class. Qualify dependent names.
* include/bits/regex_compiler.h (__detail::_Compiler::_M_get_nfa): Make
non-const and use std::move to avoid copying.
* include/bits/regex_compiler.tcc: Likewise.
* include/bits/regex_executor.h (__detail::_Executor::_M_is_word): Use
array, so past-the-end iterator is valid.
From-SVN: r204571
2013-10-08 Tim Shen <timshen91@gmail.com>
* include/bits/regex_executor.h: Add _TodoList class.
* include/bits/regex_executor.tcc (_BFSExecutor<>::_M_main): Add
_M_match_stack and _M_stack to make everything faster. Break if
_M_stack is empty, to reduce unnecessary idling.
* testsuite/performance/28_regex/split.cc: New.
From-SVN: r203261
2013-09-24 Tim Shen <timshen91@gmail.com>
* include/Makefile.am: Add regex.tcc.
* include/Makefile.in: Regenerate.
* include/bits/regex.h: Remove definitions to regex.tcc.
* include/bits/regex.tcc: New.
(match_results::format, regex_replace): Implement;
* include/bits/regex_compiler.h: Move _M_flags to the top of class
member list, because other members' initialization depend on it.
* include/bits/regex_compiler.tcc
(_Compiler<>::_Compiler): Adjust member initializations.
(_Compiler<>::_M_quantifier): Fix ungreedy interval quantifier.
* include/bits/regex_executor.h: Remove _RegexT from _*Executor classes.
In the future, all regex classes may refactor to *Impl style.
* include/bits/regex_executor.tcc (_Executor::_M_set_results):
Merge identical code from _*Executor classes.
* testsuite/28_regex/algorithms/regex_match/extended/
string_dispatch_01.cc (fake_match<>): Adjust the hacking-style testcase
caller for new __get_executors interface.
* testsuite/28_regex/algorithms/regex_replace/char/basic_replace.cc:
New.
* testsuite/28_regex/match_results/format.cc: New.
* testsuite/28_regex/traits/char/lookup_collatename.cc: Remove digraph
testcase.
* testsuite/28_regex/traits/wchar_t/lookup_collatename.cc: Likewise.
From-SVN: r202858
2013-09-14 Tim Shen <timshen91@gmail.com>
* include/bits/regex.h (regex_match<>, regex_search<>):
Change regex_executor caller. Now use their return value instead
of checking __m[0].matched to find out if it's successful.
(regex_search<>): Move the search logic to regex_executor.
* include/bits/regex_automaton.h: Add some new _Opcode. Refactor
_NFA::_M_insert_*.
* include/bits/regex_automaton.tcc: Add DEBUG dump for new
_Opcode. Refactor _NFA::_M_insert_*.
* include/bits/regex_compiler.h (_Compiler<>::_M_get_nfa):
Use make_shared instead of construct by hand.
* include/bits/regex_compiler.tcc: Implement _Compiler<>::_M_assertion.
* include/bits/regex_constants.h: Fix indentation and line breaking.
* include/bits/regex_executor.h: Add _ResultsEntry to support
greedy/ungreedy mode. Move regex_search logic here.
* include/bits/regex_executor.tcc: Implement assertions and
greedy/ungreedy matching.
* include/bits/regex_scanner.h: Add a new token _S_token_ungreedy.
* include/bits/regex_scanner.tcc: Parse a new token _S_token_ungreedy.
* testsuite/28_regex/algorithms/regex_search/ecma/assertion.cc: New.
* testsuite/28_regex/algorithms/regex_search/ecma/greedy.cc: New.
* testsuite/28_regex/algorithms/regex_search/ecma/string_01.cc:
Fix comment.
From-SVN: r202591
2013-08-29 Tim Shen <timshen91@gmail.com>
* include/bits/regex.h (basic_regex<>::assign): Don't lose _M_traits.
(regex_iterator<>::regex_iterator): Return nullptr when regex_search
failed.
(regex_token_iterator<>::_M_end_of_seq): Should be defined true when
_M_result is(not isn't) nullptr.
* include/bits/regex_compiler.h: Store _Compiler::_M_traits by reference
instead of by value.
* include/bits/regex_executor.h (_DFSExecutor<>::_DFSExecutor): Add
_M_traits to _DFSExecutor.
* include/bits/regex_executor.tcc (__get_executor<>): Pass traits to
_DFSExecutor too.
* testsuite/28_regex/algorithms/regex_match/extended/wstring_locale.cc:
New.
* testsuite/28_regex/iterators/regex_token_iterator/wchar_t/
wstring_02.cc: New.
From-SVN: r202082
Patrick Palka