Summary:
In most cases user is not aware of ASAN and fiber stack problem, thus if the stackSize is specified we're not detecting ASAN and probably will crash.
And thus it doesn't seem like a good idea to make it a user responsibility to detect ASAN.
Test Plan: tests
Reviewed By: andrii@fb.com
Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2058741
Tasks: 7016680
Signature: t1:2058741:1431131082:9a41eb40d756c9c7af0632f7ecd55c17d10bb189
Summary:
I figured it would make sense to implement all the collect* functions using a shared_ptr<Context>, instead of doing our manual reference counting and all that. Fulfilling the promise in the destructor seemed like the icing on the cake. Also saves some line of code.
Test Plan: Run all the tests.
Reviewed By: hans@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2015320
Signature: t1:2015320:1431106133:ac3001b3696fc75230afe70908ed349102b02a45
Summary: Since the old one is weak and slow.
Test Plan: Unit tests
Reviewed By: ott@fb.com
Subscribers: trunkagent, maxime, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2052630
Tasks: 6998080
Signature: t1:2052630:1431015271:bc90ccf99941902cd4bd43a0980238c616e66abf
Summary:
In order to create an EventBase'd Suprocess class, I'd like to be able to manage the lifetime of pipes independently of the lifetime of the process. To this effect, I factored out basic Pipe handling, and provided a function that detaches the pipe vector from the Subprocess object.
#6996492 a push-blocking test is broken in trunk
Test Plan: added a unit test, fbconfig -r folly && fbmake runtests && fbmake runtests_opt
Reviewed By: dancol@fb.com
Subscribers: yfeldblum, chalfant, dancol, wez, anarayanan, trunkagent, net-systems@, njormrod, folly-diffs@
FB internal diff: D1699969
Signature: t1:1699969:1430975299:30d291ab7fcc555edddf098b33095a5b29500e76
Summary: This almost always indicates a double-close bug, or a similarly nasty logic error. I don't dare make it a check, we may have some code running in production which would be broken by the CHECK, but this should give us early warning of any such bugs.
Test Plan:
```
fbconfig folly/test:file_test folly/test:file_util_test
fbmake runtests && fbmake runtests_opt
```
This test used to pass until @tudorb made me remove it due to worries about death tests messing with (currently absent) multi-threading:
```
+
+void testDoubleClose() {
+ File f("/dev/null");
+ checkUnixError(close(f.fd())); // This feels so... wrong!
+ // The destructor will now try to double-close.
+}
+
+TEST(File, DCHECKDoubleClose) {
+#ifndef NDEBUG
+ // This test makes no sense otherwise, since this is a DCHECK.
+ EXPECT_DEATH(testDoubleClose(), "double-close-FD");
+#else
+ // That sound you hear is millions of lemmings falling to their doom.
+ testDoubleClose();
+#endif
+}
```
Reviewed By: tudorb@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2055610
Signature: t1:2055610:1431048270:a469d5c1f8182ffb74700908faa022e9613ed383
Summary:
Use std::chrono::steady_clock instead of clock_gettime(CLOCK_MONOTONIC).
In particular this fixes the build on Mac OS X, which doesn't have
CLOCK_MONOTONIC.
This also updates the code to use steady_clock::time_point correctly, instead
of using a raw milliseconds value for time since the epoch.
Test Plan:
Included unit tests, which were copied over from the legacy internal Facebook
(non-folly) version of this code.
Reviewed By: ldbrandy@fb.com
Subscribers: jwatzman, doug, fbcode-common-diffs@, net-systems@, exa, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2051557
Signature: t1:2051557:1431019654:ee76cfcf8318cc3d8a8d1522b3fc97f08831ecf4
Summary: Per discussion on D2040509 this is better.
Test Plan: Still builds.
Reviewed By: yfeldblum@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2051099
Signature: t1:2051099:1430949575:cc167b57f2d6ff42a73dee4e65d22d04932bb279
Blame Revision: rFBCODE9bdb427be1ef80b612e4f364db7809c6351cfe1c
Summary:
OS X doesn't support constructor init priorities, at all. AIUI,
it's a limitation of their actual binary format and loader, not just a
tooling/compiler limitation.
This particular usage appears to just be for for logging/bug-finding
purposes, so it looks like just removing the priority on OS X isn't the
end of the world?
Test Plan: g++-4.9 on OS X compiles this file now.
Reviewed By: njormrod@fb.com
Subscribers: ldbrandy, jdelong, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2040557
Signature: t1:2040557:1430975025:73f817b5d19a18dca6b19ba783dbea99192cbc41
Summary:
This is for OS X; there is apparently some difference between
its types and Linux's typical types, causing incomopatibilities between
a `long int` and a `long long int`. These two explicit template
instantiations fix the issue.
Test Plan: g++-4.9 on OS X compiles these files now.
Reviewed By: lucian@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2040509
Signature: t1:2040509:1430516624:9db8146b7824c0d09bac418a10a5f54451cdd5db
Summary:
I just had topaggr built from trunk crash with https://phabricator.fb.com/P19825084. It seems that
placing `FOLLY_DISABLE_ADDRESS_SANITIZER` next to the function definition has no effect. Though
what's strange, https://phabricator.fb.com/D2036280 test plan says the false-positive crashes
stopped after that diff. Have we run the tao script on the latest revisions of that diff? Anyway,
clang/gcc docs clearly say the attributes belong to declarations, not definitions, so let's do that.
Test Plan:
Compiled, unit tests.
Reviewed By: aap@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant, soren
FB internal diff: D2042149
Signature: t1:2042149:1430681824:aea21b9a1fcddfd5585180ba7f526c3a6ee0c8bf
Summary:
A client example to match telnet server.
Required a couple additions:
* future result when socket actually connects, similar to netty
* clients support IOThreadPoolExecutor groups
* a pipeline stage to make sure everything runs in the right eventbase thread.
Test Plan:
fbconfig follg/wangle/example/telnet && fbmake dbg
telnet_server
telnet_client
Reviewed By: hans@fb.com
Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2010289
Signature: t1:2010289:1430766232:65c6f946e454000f6ea9f41b49197ddbeea5ba3f
Summary: similar to https://github.com/netty/netty/tree/master/example/src/main/java/io/netty/example/telnet
Test Plan:
fbconfig folly/wangle/example/telnet; fbmake dbg
_bin/folly/wangle/example/telnet_server --port=8080
telnet localhost 8080
Still a couple sharp edges:
* No easy way to wait for ServerBootstrap termination.
* Pipelines always have to call attachReadCallback
* a bunch of missing methods in pipeline still, like channelActive
Reviewed By: hans@fb.com
Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D1959172
Signature: t1:1959172:1427993978:463f237036996451187e3ef3983cf2b4e89685ef
Summary:
Leave the important headers visible. Opens the door for decent inline docs.
Test Plan: unit
Reviewed By: hans@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2036799
Signature: t1:2036799:1430749004:db5e58655c27b96826549849722fe962b9ae3985
Summary:
Compiling folly/futures:futures-test, this saves 15% (dbg) and 7% (opt)
Compiling all of folly/futures, it's 7% ish for each
Main blocker right now is that this generates a spew of deprecated warnings from calls to core_->(de)activate() from Future::(de)activate(). Can the deprecations be moved up to the Future methods instead?
Also had to fix willEqual for Future<void> which was borked
Test Plan: compiles
Reviewed By: hans@fb.com
Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2021028
Signature: t1:2021028:1430749114:1dd78af47ea91aa5e67929a5884b66ca0c8ae2d8
Summary:
@markisaa, @ldbrandy: stdarg.h is for ellipses arguments, ##...##, which this file does contain.
The pull request indicates that this header is required for windows. Given that adding it doesn't hurt, and should be there anyways, I'd say go on this pull request.
Fixed a fatal error when compiling folly for HHVM
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: contbuild
Reviewed By: markisaa@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant, ldbrandy, markisaa
FB internal diff: D1999557
Tasks: 6783581
Signature: t1:1999557:1429220813:0580882b8a55d86f51cdd602d2d152920d368b75
Summary:
Fiber::init() method modifies all the bytes in the fiber's stack in
order to produce a high watermark of stack memory usage. This behavior
generates ASAN errors, as it modifies stack red zones. For that reason,
ASAN is disabled for that particular method.
Test Plan:
Unit tests
Reviewed By: alikhtarov@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2036280
Tasks: 6861962
Signature: t1:2036280:1430439550:e156e0e06fae3615d9d9204e99acaf3072bbfafd
Summary:
None of these functions should be templated with `class Executor`.
Except `then(Executor, Args...)` because otherwise the compiler gets
confused. This was the combination that worked for both Clang and GCC,
don't ask me why. I'm assuming this puts it on a low priority...
I think this is also OK, because `setExecutor` takes an actual
`folly::Executor`, so even `then(Executor, Args...)` won't just work for any
`Executor`.
Test Plan: Run all the tests.
Reviewed By: jsedgwick@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2036912
Tasks: 6838553
Signature: t1:2036912:1430493088:44f2ffe146298c3f978ac27a45b9b2e33b2b0422
Summary:
Oh, this is best of both worlds: D2015316
If your reduce is really cheap, just use `collect/CollectAll->reduce`.
Otherwise just use `reduce`. Great?
This one doesn't support returning Futures from `func`, but if you want
to do that, just use `reduce`.
Test Plan: Run all the tests.
Reviewed By: hans@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2017238
Tasks: 6025252
Signature: t1:2017238:1430422971:c3471afa53239681f835624becf3c4a238204b70
Summary:
I wanted to use `collectAll` for `reduce` if `func` does not return a
Future, because the overhead seemed smaller, but it has been suggested
that running the callback as soon as possible might be better. Not sure
which is. Note that this also makes n copies of the lambda and moves the
value at least n times.
I also plan to add a `streamingReduce` which calls `func` as soon as
results come in (but out of order).
Test Plan: Run all the tests.
Reviewed By: hans@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2015316
Tasks: 6025252
Signature: t1:2015316:1430349611:4f62a540ded85299a22670dd2add27cf1458e8f8
Summary: This is a temporary fix (until D2015320 is checked in) for race condition(s) in collect(..) method.
Test Plan:
Run unit tests
Run buffalo_aggregator canary
Reviewed By: jsedgwick@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2037406
Tasks: 6894157
Signature: t1:2037406:1430435227:ed9612d016cdbd708e2deba02dc4fe0b59632f5a
Summary: Lift void into the unit monad and pass other types through unscathed.
Test Plan: new unit tests
Reviewed By: yfeldblum@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2029785
Signature: t1:2029785:1430333928:ef2fbb2e3d94518a732f6818a06c32481120bd4f
Summary: The value constructor can be nice. But when it matches on Future<Something> it just confuses everybody.
Test Plan:
building and running tests
contbuild
Reviewed By: jsedgwick@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2036455
Tasks: 6925951
Signature: t1:2036455:1430423503:73906f748318c4ebec13f45ad3761f104e2ef888
Summary:
clang (>=3.6?) reports potential object slicing bug when MPMCQueue is used for
polymorphic class as the queue item, e.g. as in P19814469. This can be false
positive however, since the choice is based on the type trait already. This
diff uses tag dispatch to selectively compile the overload that will be
executed, therefore if there is no-throw move ctor supplied clang will not
examine the simulated relocation code.
This doesn't avoid object slicing bug however if the client insists to use
MPMCQueue to hold base class while enqueue and dequeue with subclassed item.
Test Plan: compile with --clang
Reviewed By: tudorb@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2029949
Signature: t1:2029949:1430264357:af479117adf90bc1915c071e7376a30aacb72f46
Summary:
Cleans up bootstrap a bit at the expense of a more complex Pipeline interface
This doesn't have to go in, lmk either way as I want to move on to reorganizing this code into inl headers etc
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2034634
Signature: t1:2034634:1430414670:c91712fb26353987cb471e35a54f55c869ae7cf1
Summary: Also, instead of throwing on finalize() if there's no inbound/outbound handler in the pipeline, log a warning and throw when the operations are attempted. This was necessary for CodecTest which doesn't use outbound handlers.
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2028176
Tasks: 6836580
Signature: t1:2028176:1430346333:fdd645a535e8158d780cfd1119e27803995b663f
Summary:
Much less copypasta this time around. I wonder if the getters and setters for write flags and read buffer settings are necessary in the new handler types, or even if they belong in the bidirectional handler
I'm all ears for more suggestions on reducing copypasta
I'm going to reorg the code (inl headers etc) in a subsequent diff once this is in - easier to review this way
Test Plan: existing unit, thinking about tests for these changes
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2026522
Tasks: 6836580
Signature: t1:2026522:1430346145:bd7f7770eddce0470e2ac72440fc001cf128df08
Summary:
I think this is the best I can do to minimize whjat must be copied for InboundHandler, InboundHandlerContext, InboundContextImpl, outbound equivalents, etc
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2025612
Tasks: 6836580
Signature: t1:2025612:1430346200:bc06162711bbc52b17574297a3569736f7973a7c
Summary:
This diff is based on https://github.com/facebook/folly/pull/95
by gitbug user mikekap.
It changes the struct/class naming of forward declarations to match
their definitions.
Test Plan:
fbconfig -r folly && fbmake runtests
Reviewed By: markisaa@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2032995
Tasks: 6783581
Signature: t1:2032995:1430339337:79193fd8409b73e8a9155133f326b7cee534244d
Summary: It looks like the ubuntu version of gtest complains about this?
Test Plan:
Will watch jenkins fbthrift and proxygen builds
previous jenkins break:
https://ci-builds.fb.com/job/folly/2178/console
Reviewed By: bmatheny@fb.com
Subscribers: doug, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2032761
Tasks: 6913742
Signature: t1:2032761:1430337624:ec65b26cf8da7d0d8c7e449f235de7147d4f17d6
Blame Revision: D2029802
Summary:
THeader.h went from 1.8s to 0.5s.
Total build of sample project doing thrift client and server stuff went
down 1.5-3% from
```
(run 1)
real 5m25.059s
user 130m30.853s
sys 10m18.642s
(run 2)
real 5m29.483s
user 129m12.209s
sys 10m14.762s
```
to
```
(run 1)
real 5m23.292s
user 125m44.318s
sys 10m7.103s
(run 2)
real 5m20.999s
user 127m4.969s
sys 10m10.118s
```
Test Plan:
fbmake --distcc off --ccache off dbg
Will also watch contbuild, since some files might be missing includes
Reviewed By: haijunz@fb.com
Subscribers: jmarg, anca, reachfrequency-eng@, jhunt, yanli, jgeller, chaoyc, search-fbcode-diffs@, net-systems@, zeus-diffs@, vikas, jcoens, unicorn-diffs@, ldbrandy, jteller, atlas2-eng@, everstore-dev@, leis, benj, laser-diffs@, zhguo, jying, wanghuan, jeremyfein, dbolcioni, jacekm, maxwellsayles, osmith, pallotron, fbcode-common-diffs@, davejwatson, andrewcox, marcelo, ckwalsh, mcduff, hitesh, mshneer, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, haijunz, chalfant
FB internal diff: D2029911
Signature: t1:2029911:1430261056:9ae688c50585a6454479f810d80b2c1c79b60e04
Summary:
Adds get_or_throw map helper (get a value, or throw with the specified
exception type) and get_optional (get an Optional<Value>). This is a folly
backport of some util helpers in experimental.
Test Plan: Unit tests
Reviewed By: wez@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2029802
Tasks: 4332480
Signature: t1:2029802:1430280249:3efbb2fb9394e31b3fbe6d5bd209d12ebb7ed587
Summary:
This should help executing sync thrift handlers on fibers. This allows the handler methods to achieve concurrency by awaiting.
Test Plan: Chimera unit tests (which also accesses running FacebookBase methods in fiber context!)
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, hannesr, andrii, folly-diffs@, jsedgwick, haijunz, yfeldblum, alandau, chalfant
FB internal diff: D2015886
Signature: t1:2015886:1430156870:a84499e0e781f5e6e77f48f1f3063f53d0968698
Summary:
add a free system memory limit to Load Shed Configuration for dynamic shedding
Test Plan:
tested on edge241.01.ams3, works as expected. Once the free memory is less than 23G (picked on purpose for testing), new connections are shedded. But the shedding rate is totally wrong. I will explain and address it in another diff.
https://fburl.com/103620501https://fburl.com/103620701
Reviewed By: afrind@fb.com
Subscribers: trunkagent, fugalh, bmatheny, nimishshah, folly-diffs@, jsedgwick, yfeldblum, chalfant, xning, alexkr
FB internal diff: D2026477
Tasks: 4604893
Signature: t1:2026477:1430176828:65eadd82efa3189a1bebcb8518efaf56cd36beed
Summary: Unit is a bit special because it's just something special to use instead of `Promise<void>`, so let's offer the same sugar that `Promise<void>` has (`p.setValue()` instead of `p.setValue(Unit())`)
Test Plan: New unit tests. Look, a pun!
Reviewed By: jsedgwick@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, davejwatson, hannesr
FB internal diff: D2014139
Tasks: 6847876
Signature: t1:2014139:1430159950:1484ee420c6d7f0f794a546b78ef1601c2eec45c
Summary: Calling `folly::gen::orderBy()` without arguments causes a compilation error because the first template argument cannot be deduced. This diff fixes it.
Test Plan:
fbconfig -r folly && fbmake runtests
Reviewed By: philipp@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D2025789
Signature: t1:2025789:1430167404:02fde7287b015d9dcbf398e8dc84cde7d74b4a5b
Summary:
[Folly] BenchmarkSuspender::dismissing.
Pass a lambda to it, and the lambda will be executed while the benchmark-suspender is dismissed. Just a bit of sugar around `BenchmarkSuspender::dismiss` and `BenchmarkSuspender::rehire`.
BENCHMARK(name_void, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
braces.dismissing([&] {
# benchmark timer is running
doSomething();
});
# benchmark timer is suspended
}
BENCHMARK(name_value, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
auto value = braces.dismissing([&] {
# benchmark timer is running
return doSomething();
});
# benchmark timer is suspended
}
Test Plan:
Unit tests:
* `folly/test/BenchmarkTest.cpp` (actually a benchmark)
Reviewed By: njormrod@fb.com
Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2024166
Signature: t1:2024166:1430163281:24df0ac98cbe36372f780372ee8f7dd3722b7868
Summary:
Only allow this if the handler is only ever attached to a single pipeline once. i.e. only ever associated with one Context
Test Plan: unit, thrift unit
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2024007
Tasks: 6836580
Signature: t1:2024007:1430157264:efcf70ca3531c10eec5d458c9e9d6cda60c507c3
Summary:
detachPipeline always goes bottom to top
attachPipeline always goes top to bottom
now we can attachReadCallback in AsyncSocketHandler::attachPipeline()
not sure of the implications for TAsyncTransportHandler... looks like Cpp2Channel still wants to attach/detach cb manually
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2023982
Tasks: 6836580
Signature: t1:2023982:1430157500:e09a4103550a3e5721effaa1b28ac5bed071fa67
Praveen Kumar Ramakrishnan