Summary:
(from @njormrod) This changes the READMEs to be .md,
appropriate for github. I viewed the docs on github, and they looked
WAY better as .md files.
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: view on github
Reviewed By: markisaa@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D1999531
Tasks: 6783581
Signature: t1:1999531:1429220432:68c6d8dc70806d993c83cacce6369cb7e6e964c8
Summary:
A few things:
- Eliminate HandlerPtr by managing all handlers with shared_ptrs instead of keeping them inline in the pipeline
- Kill recursively templated ChannelPipeline accordingly
- Introduce StaticPipeline to retain the flexibility of zero-alloc pipelines
- Introduce notion of an "owning handler" to avoid destruction order issues
Test Plan: unit (will add more), thrift unit
Reviewed By: davejwatson@fb.com
Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2023976
Tasks: 6836580
Signature: t1:2023976:1430159578:e50e8a149e549a40670d093fb65987a4843cdd8d
Summary: Move boring preamble stuff to `Future-pre.h` and `folly::futures` and `makeFuture` and pals to `helpers.h`.
Test Plan:
tests still build and pass
Reviewed By: jsedgwick@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, hannesr, davejwatson
FB internal diff: D2014330
Signature: t1:2014330:1429941589:1e2c336136f3375f9b96e5df8c06ca5820ba6aeb
Summary: as above. Only got a little messy when components within folly::wangle typedefed things to Pipeline
Test Plan: unit tests
Reviewed By: davejwatson@fb.com
Subscribers: wormhole-diffs@, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2022181
Tasks: 6836580
Signature: t1:2022181:1430157032:df0bdfb9ca0d76b86d52c55c4ad41ea953a18cb4
Summary:
[Folly] IPAddressV6::kToFullyQualifiedSize.
The scenario: One has a string known to be a fully-qualified IPv4 or fully-qualified IPv6 address, but one does not know which of the two it is.
The dirty but fast solution: just check its size.
Test Plan: Build. Unit tests.
Reviewed By: mshneer@fb.com
Subscribers: folly-diffs@, dougw, yfeldblum, chalfant, brettp
FB internal diff: D2016634
Signature: t1:2016634:1429831808:08ecb28eab3060988f9613748ed8dabf3724bf13
Summary: Similar to Promise::fulfil -> setWith change, this name is a lot clearer
Test Plan: tests
Reviewed By: hans@fb.com
Subscribers: netego-diffs@, fugalh, mwa, jgehring, fuegen, folly-diffs@, tingy, jsedgwick, yfeldblum, chalfant
FB internal diff: D2013124
Tasks: 6837405
Signature: t1:2013124:1429735106:e8861925dfeb6d7f0662c1057cbcf2ad8dcf008c
Summary:
Change ThreadManager interface to more generic Executor interface. Specific example of supporting fibers. This diff would replace D1967655
Test Plan: added several unittests
Reviewed By: alandau@fb.com
Subscribers: doug, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, alikhtarov, andrii, yitingli
FB internal diff: D1994663
Signature: t1:1994663:1429640509:d729ce6f020563b680d1d549f9aa273c739eb925
Summary:
We don't need to check for void after all, and with perfect forwarding we don't need separate const& and && versions.
Test Plan: tests still pass
Reviewed By: jsedgwick@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2014264
Tasks: 6847876
Signature: t1:2014264:1429735036:01ac166399ef8d0f2f34adb51e965809022c2b64
Summary: This covers the ubiquitous case. If the approach is approved (the task mentioned changing impl to use 'auto for' so this is a bit different) i'll codemod callsites
Test Plan: unit
Reviewed By: hans@fb.com
Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2013573
Tasks: 6779710
Signature: t1:2013573:1429735631:cfa0d3f6672a8966afc0ea18308307e2153793ce
Summary:
MIN_WRITE_SIZE is used to avoid small write calls to SSL_write. If there are
consecutive small buffers to write, then will be combined together(by being
copied to a local buffer) until total size exceeds MIN_WRITE_SIZE. This reduces
number of calls SSL_write, improving performance, and avoiding overhead in OpenSSL.
Currently, MIN_WRITE_SIZE is hard coded to be 1500 bytes.
Wormhole could benefit from this, as our average message size is several
hundreds of bytes. We could get even better throughput with larger
MIN_WRITE_SIZE.
As discussed with Adam and Alan, there is a good reason to make it
configurable, though default value is still 1500.
Test Plan: unit tests
Reviewed By: simpkins@fb.com
Subscribers: net-systems@, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, thomasf
FB internal diff: D1996570
Tasks: 6784543
Signature: t1:1996570:1429667035:a661ef30a715dafec3e134a7f6af6f56ada2e8e0
Summary:
Today it's hard to create SocketAddress from IPAddress
without converting to strings and back. This change adds this capability.
Test Plan: build
Reviewed By: yfeldblum@fb.com
Subscribers: ps, bmatheny, folly-diffs@, yfeldblum, chalfant
FB internal diff: D2011879
Signature: t1:2011879:1429733811:c05fc2d5ef5e9fdbbbb54ad26b4d1e3c0ad06dfa
Summary:
make future::wait() use fiber's baton, so wait works in threads or in fibers.
Much cleaner than making a new FiberRequest type in thrift
Test Plan: tests
Reviewed By: andrii@fb.com
Subscribers: doug, alandau, bmatheny, mshneer, andrii, folly-diffs@, yitingli, yfeldblum, jsedgwick, chalfant
FB internal diff: D1996283
Signature: t1:1996283:1429144165:da5dc6b1f2a053a45efd39877e79169e3fba810c
Summary:
as above. it never ends.
Test Plan: added unit
Reviewed By: hans@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2011569
Signature: t1:2011569:1429660210:930cb17682d5c86a11881a23efe0a91f4c6a36b1
Summary:
Pass an Executor to `then`, which applies only for that callback. This is on
the one hand just a convenience method, but it's a major convenience when it's
needed, because grabbing, storing, and restoring the old Executor in the
middle of a chain is very inconvenient indeed.
Test Plan:
new unit
docblock comments
Reviewed By: jsedgwick@fb.com
Subscribers: folly-diffs@, davejwatson, chalfant, yfeldblum, nkgupta, jsedgwick, exa, robbert
FB internal diff: D2011542
Tasks: 6771589, 6838553
Signature: t1:2011542:1429660204:f5959b1e0b3b36dfb8c3c7091302d19101dde93b
Summary:
this was a fun one. Add a specialized implementation that builds up the results in a map with their indices and aggregates them into a vector at the end
Test Plan: unit tests
Reviewed By: hans@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2002444
Signature: t1:2002444:1429642589:ee5aa5e8c461db97a28642b9887b3158df317813
Summary:
Rather than crashing spectacularly, if `Executor::add` throws (e.g. because the queue is full), then discard the result we got and assume the exception the executor threw instead.
Alternatively, we could pass this exceptional Try to the callback (without an executor, as it is here), but not perturb `result_`. This would mean two different world views in these two code snippets:
auto f1 = makeFuture(42).via(&crappyExecutor);
f1.value(); // 42 (no callback happened)
f1.then(...); // would see the executor's exception. Would also be ill-advised to do this after value()
auto f2 = makeFuture(42).via(&crappyExecutor)
.then([](int x) { return x * 2; }); // skipped
f2.value(); // throws executor's exception
It feels rude to throw away the result, but it feels too potentially dangerous to allow this split view of the world.
Test Plan: modified unit
Reviewed By: jsedgwick@fb.com
Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D2007729
Tasks: 5306911
Signature: t1:2007729:1429627114:b627ce758ce9231298f1b28e203ccc1ee415ed9a
Summary:
Pass an Executor to `then`, which applies only for that callback. This is on
the one hand just a convenience method, but it's a major convenience when it's
needed, because grabbing, storing, and restoring the old Executor in the
middle of a chain is very inconvenient indeed.
Test Plan:
new unit
Reviewed By: jsedgwick@fb.com
Subscribers: robbert, exa, folly-diffs@, jsedgwick, nkgupta, yfeldblum, chalfant, davejwatson
FB internal diff: D1985475
Signature: t1:1985475:1429148056:0450120263ba8110e8825420cbefe3b1887f7306
Summary:
(from @njormrod) This changes the READMEs to be .md,
appropriate for github. I viewed the docs on github, and they looked
WAY better as .md files.
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: view on github
Reviewed By: markisaa@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D1999531
Tasks: 6783581
Signature: t1:1999531:1429220432:68c6d8dc70806d993c83cacce6369cb7e6e964c8
Summary:
This broke the HHVM (on ubuntu at least)
Test Plan:
compiled it
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D1999341
Tasks: 6783581
Signature: t1:1999341:1429219016:235beef7522c4ada020eedf4f0a2ae200e94b7c8
Summary:
This is folly github pull request https://github.com/facebook/folly/pull/184
It seems we might avoid some temporaries in putMessagesImpl
function. To do so we pass arguments directly to constructors
with the help of emplace_back member.
Test Plan:
All folly/tests, make check for 37 tests, passed.
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D1998515
Tasks: 6783581
Signature: t1:1998515:1429207683:056db129405bed212f50a50821f095c8d3694ec8
Summary: Should also work if eventbase isn't looping
Test Plan: added new unittest
Reviewed By: hans@fb.com
Subscribers: doug, folly-diffs@, yfeldblum, chalfant
FB internal diff: D1984259
Signature: t1:1984259:1428691788:edb89424a48b8071a776d828300b7c1094f1c48f
Summary: We can access LocalData while currentFiber is set. We should make sure it's set to null when LocalData::reset is called.
Test Plan: unit test
Reviewed By: alikhtarov@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant, bwatling
FB internal diff: D1996071
Tasks: 6725667
Signature: t1:1996071:1429135408:d549d577e140ce2867aff4130e73be3884dbd2ed
Summary: let's not do this to ourselves :)
Test Plan: unit
Reviewed By: yfeldblum@fb.com
Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D1993491
Signature: t1:1993491:1429063112:b8664b365ad12fa4b51e9bd7fde9e4d197dbc648
Summary: Previously we'd call ensureLoopScheduled() but it'd be a no-op since the loop was already scheduled. Delaying the call to ensureLoopScheduled() fixes the issue
Test Plan: unit tests (FiberManager.yieldTest fails without the changes to FiberManager-inl.h)
Reviewed By: andrii@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D1993686
Signature: t1:1993686:1429070253:af933abbbbb33868a402f1d643e4e6f5fef1be83
Summary:
During destruction, we enqueue a callback to `eventBase_` that references `wheelTimer_`.
So long as that callback exists, we have a circular dependency as both reference the other and we -cannot- safely destroy this object (and its members). The fix is to break this dependency someway. In this case, I've chosen to simply `AndWait` until the eventBase has consumed the message referencing the timer. From here normal destruction of members can proceed safely (destorying HHWheelTimer first, EventBase second).
The problem in the current code is that the Eventbase will attempt to consume all messages during it's own destruction. If `wheelTimer_->cancelAll()` is still enqueued, it will attempt to use the now destroyed `wheelTimer_`.
Test Plan:
Running this test repeatedly was able to repro:
./folly/futures/futures-test --gtest_filter=Timekeeper.futureWithinVoidSpecialization
Reviewed By: yfeldblum@fb.com
Subscribers: enis, folly-diffs@, jsedgwick, yfeldblum, darshan, chalfant
FB internal diff: D1985967
Tasks: 6332729, 6741095
Signature: t1:1985967:1428726270:997ec277c6a73554e54b8cf673acd36ff62976e6
Summary: This code has useful logging when turned up high enough (e.g. --v=11). But we need to parse the flags to see it.
Test Plan: trial & error
Reviewed By: yfeldblum@fb.com
Subscribers: enis, folly-diffs@, jsedgwick, yfeldblum, darshan, chalfant
FB internal diff: D1985965
Signature: t1:1985965:1428713065:4607610ec438416fffb5fced1867efa13544eef0
Summary: We need to do forward declarations first, and types.h already does declarations, so just make sure it gets included first in each file.
Test Plan:
tests
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, exa, folly-diffs@, jsedgwick, yfeldblum
FB internal diff: D1861040
Tasks: 6258585
Signature: t1:1861040:1428604059:de6393e3c87e0694a6b04ad6d440cc9635a1008d
Summary:
SharedMutex used a saturating counter that records the number of
waiting lock() calls, but an ABA problem on futexWait could lead to a lost
wakeup when there was exactly 3 or 4 threads contending on the RW lock
in W mode. This diff changes the kWaitingE count to be heuristic (it is
possible that the count says 1 but there are two waiters), saturates at
2 instead of 3 (because there is no benefit from differentiating those
two), and doesn't decrement the count on a successful wakeup.
Also, I noticed while debugging this that boost::noncopyable was causing
SharedMutex to be 8 bytes when it should only be 4.
One way the wakeup could be lost in the old code:
1. A calls lock()
2. A updates state <- kHasE
3. A returns
4. B calls lock()
5. B spins
6. B updates state <- kHasE + 1 * kIncrWaitingE
7. A calls unlock()
8. A updates state <- 0
9. A calls futexWake(), which returns 0
10. A calls lock()
11. A updates state <- kHasE
12. A returns
13. C calls lock()
14. C spins
15. C updates state <- kHasE + 1 * kIncrWaitingE
16. C calls futexWait, expecting kHasE + 1 * kIncrWaitingE
17. B calls futexWait, expecting kHasE + 1 * kIncrWaitingE
18. A calls unlock()
19. A updates state <- 0
20. A calls futexWake(), which returns 1
21. C receives the wakeup
22. C updates state <- kHasE
23. C returns
24. C calls unlock()
25. C updates state <- 0
B missed the wakeup that was intended for it (sent at step 9, wait
started at step 17), but went to sleep anyway because it saw the write
state at step 17. Now there are two waiters but only 1 recorded in the
SharedMutex, at which point failure is inevitable.
Test Plan:
1. DeterministicSchedule test using uniformSubset that can repro the problem
2. Test in production scenario that produced occasional deadlocks under high stress
Reviewed By: yfeldblum@fb.com
Subscribers: folly-diffs@, yfeldblum, chalfant
FB internal diff: D1980210
Tasks: 6720328
Signature: t1:1980210:1428623932:ef1c00c3f88154578b2b253ac0cfdbadf9f31d8c
Summary: Move this from a test to its own file in codec folder
Test Plan:
already has a unittest
fbconfig -r folly/wangle; fbmake runtests
Reviewed By: hans@fb.com
Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
FB internal diff: D1959152
Signature: t1:1959152:1427935371:9d1338986df97fd94f98ad2bf08c731d229bc2b1
Summary: Part 2 of pipeline. Use OutputBufferingHandler, which is exactly the same output buffering thrift does currently.
Test Plan:
A couple canaries, unittests.
Specific issues of previous pipeline diffs I think are fixed already and should be fine with this one:
* Previously there were missing DestructorGuards. I don't think anything in OutputBUfferHandler needs a DG
* previously broke http GET handling. There is a unittest for it now.
Canary
Reviewed By: alandau@fb.com
Subscribers: doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, vloh, andreib
FB internal diff: D1970949
Signature: t1:1970949:1428360621:a0db142032894525f69c2b144cf946d63f790fe1
Alecs King