1660 Commits

Author SHA1 Message Date
Dimitri John Ledkov ca2e9c7f1b Fix forward declaration of basic_string and list, for dual abi libstdc++ with inline std::__cxx11 namespace.
Closes facebook/folly#213
Closes facebook/folly#208
2015-05-27 13:31:10 +01:00
woo ddfdba13bb Bump version to 40:0 v0.40.0 2015-05-26 11:32:31 -07:00
James Sedgwick b5a2084a1a remove spurious warning
Summary: see task

Test Plan: OK

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2099416

Tasks: 7166291

Signature: t1:2099416:1432658571:4ae632c218c4764f23e4b189167f6fdaff13dc2d
2015-05-26 11:32:19 -07:00
Anton Likhtarov 1f656e7d6b Some optimizations
Summary:
1. Eliminate some string -> StringPiece -> strings conversions
2. Mcrouter: eliminated unnecessary inlining by moving slow path logic into its own method.

Using a test setup with shadow sampling enabled and shadowing some requests,
(typical prod setup), this brings down the cost from ~1.4% cpu in standalone mcrouter to ~0.2%:

```
before:

+ 0.70%          3898  mcrouter_orig  mcrouter_orig  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.13%           864  mcrouter_orig  mcrouter_orig  [.] EventGroup<ScubaRow>::processExtraSamplers
+ 0.58%          3347  mcrouter_orig  mcrouter_orig  [.] DynamicScubaSampler::getSampler

~ 1.41% total

after:

+ 0.18%          1223  mcrouter_fix  mcrouter_fix  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.04%           205  mcrouter_fix  mcrouter_fix  [.] EventGroup<ScubaRow>::processSampler

~ 0.22% total
```

Fiber local optimization might have more of an effect.

Test Plan:
unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, alikhtarov, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2089133

Tasks: 5414865

Signature: t1:2089133:1432338487:4158dc6b720c04f43820193e73b98d4197afcffa
2015-05-26 11:32:15 -07:00
James Sedgwick da432bdcb8 move FiberIOExecutor to wangle/concurrent
Summary: Need to cut this dep on wangle out of folly as we're moving wangle to its own repo

Test Plan: chimera unit

Reviewed By: rushix@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2092122

Signature: t1:2092122:1432239179:9261c1b3a3a346b886b15899d25a4d0530d4e890
2015-05-26 11:32:09 -07:00
Chip Turner c947d451ab Revert "Make strlcpy available in folly" (D2062632)
Summary:
Reverting diff to fix the build.

Test Plan: contbuild rule that failed to build is okay now

Reviewed By: ldbrandy@fb.com
2015-05-26 11:32:03 -07:00
Chip Turner bc34a5a774 Make strlcpy available in folly
Summary:
strncpy is bad.  strlcpy is somewhat less bad.  We already had
this function, but let's move it somewhere more reasonable.

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: trunkagent, lins, anca, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2062632

Signature: t1:2062632:1431969926:cc7f7283073d0242fe8f361efac2557aa0b0a481
2015-05-26 11:32:02 -07:00
Marc Horowitz 3be65bf058 remove unused variable
Test Plan: search, and fbmake

Reviewed By: delyank@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093523

Signature: t1:2093523:1432249208:84a27e2328636c2aafe43412f61a693b5645b895
2015-05-26 11:31:59 -07:00
Jim Meyering 79e29a6c28 folly/experimental/fibers/EventBaseLoopController.h: include <atomic>
Summary: Without <atomic>, this fails to compile with clang:dev and newer glibc.

Test Plan:
fbconfig --clang --with-project-version=clang:dev --with-project-version=llvm_compiler_rt:dev --platform-all=gcc-4.9-glibc-2.20 --sanitize=address mcrouter/lib/network:client-test-sync && fbmake dbg

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093709

Signature: t1:2093709:1432250378:2b78c142598bf0a7b0869089c752f5ccdae9da22
2015-05-26 11:31:54 -07:00
Marc Horowitz b0131bea12 Refactor tests to verify that a single folly target can be used successfully
Summary:
This includes a change to Range to move operator<< into the
header, to achieve the goal.  Specifically, see format_test,
dynamic_test, json_test, demangle_test

Test Plan: fbconfig -r buck && fbmake runtests

Reviewed By: njormrod@fb.com

Subscribers: darshan, tjackson, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2063698

Signature: t1:2063698:1431467309:069da6d74bb5c384e7a21e6be19a4b20466bdd92
2015-05-26 11:31:47 -07:00
Marc Horowitz 3b4d7b6a8e folly refactorings to better support cross-platform
Summary:
I'm looking into pulling parts of folly (right now, dynamic,
json, and their dependencies) into fbandroid for use as part of xplat.
This diff includes a few kinds of changes: portability fixes for arm;
reduce the size of the code generated by moving non-templated
functions and methods into cpp files; refactoring header usages which
require extra compiler flags on android to cpp files; and slicing up
the libraries a bit differently to reduce dependencies.  This should
all be backward-compatible, and do no harm to fbcode.

Test Plan: runtests, sandcastle

Reviewed By: njormrod@fb.com

Subscribers: darshan, davejwatson, tudorb, dancol, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2057797

Tasks: 7005344

Signature: t1:2057797:1432145435:fa10f129fc669e682da5b4b207fc96986ca035fc
2015-05-26 11:31:44 -07:00
Dave Watson caf6cdc0cb fixup service filter interface
Summary:
Based on a more thourough reading of finagle's interface:
* adds close/isAvailable, which seem very close to thrift's interfaces
* ComposedServices are hardcoded to underlying services, to simplify the code (means extra allocs?)
* Made everything a shared_ptr
* Addd ServiceFactoryFilters

Test Plan: Updated the existing unittests and added some new ones

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037206

Signature: t1:2037206:1432147489:3464d4c12a9e434d4973febcabbf7b2b3a883257
2015-05-26 11:31:39 -07:00
Tudor Bosman d5550513ae tupleRange, tuplePrepend
Summary:
tupleRange<start, n>(tuple): return a tuple consisting of a range of elements
from the given tuple

tuplePrepend(x, tuple): return a tuple consisting of prepending x to the given
tuple.

For Lispies:

std::get<0>(tuple) is car.
tupleRange<1>(tuple) is cdr.
tuplePrepend(x, tuple) is cons.

Test Plan: test added

Reviewed By: lesha@fb.com

Subscribers: trunkagent, lesha, ilyam, folly-diffs@, yfeldblum, chalfant, jhj, alerer, kma, pamelavagata, tulloch

FB internal diff: D2087568

Signature: t1:2087568:1432164681:18795d0e8bb01f38ffc6949ac233f514ab098355
2015-05-26 11:31:32 -07:00
Hannes Roth 3651364b70 (Wangle) Fix typo
Summary:
This was supposed to be the `Result` type, since it's called on the
Future returned by the lambda.

Test Plan: Added tests for void and different types in vector/lambda.

Reviewed By: mhl@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2087819

Tasks: 7126300

Signature: t1:2087819:1432142435:72914fa64eff03454774b87a24c426379defab3b

Blame Revision: rFBCODEf229322bc273190a85b5e995dcd8209b1fbf0825
2015-05-26 11:31:15 -07:00
Viswanath Sivakumar eae7495c28 Bump version to 39:0 v0.39.0 2015-05-20 10:57:38 -07:00
James Sedgwick 48a8ecdb0d make AsyncSocket::WriteRequest an interface
Summary: This will allow a subsequent diff to implement file transfers as another type of write request

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant, fugalh, bmatheny

FB internal diff: D2080257

Signature: t1:2080257:1432044566:bcc0724d349879f46e3e58ee672aff7bf37fa5f6
2015-05-20 10:57:12 -07:00
Brian Watling e1c9764467 Make AtomicHashMap shadow declaration clean
Summary: Some of AtomicHashMap's locals shadow member functions - rename the locals to fix the warnings

Test Plan: unit tests

Reviewed By: chaoc@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@

FB internal diff: D2086270

Signature: t1:2086270:1432083900:fae1be39e55e4c30b47fdc7a069bb13d75292b0a
2015-05-20 10:57:12 -07:00
Alexey Spiridonov 0e7cb3feec Make Subprocess movable
Summary:
Subprocess doesn't have any non-movable members, and its implementation does not take addresses of the object, so I think it's safe. Move makes a bunch of code cleaner (you no longer have to wrap it in `std::unique_ptr` with associated clumsiness).

https://phabricator.fb.com/diffusion/FBCODE/browse/master/folly/Subprocess.h

Test Plan:
- unit test
- Searched for `this` in `Subprocess.{h,cpp}`.
- Inspected member variables: `pid_`, `returnCode_`, `pipes_`
- contbuild

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2079167

Signature: t1:2079167:1432048688:26f96e29310298f47a9a9a7abef22dc863f68942
2015-05-20 10:57:12 -07:00
Hans Fugal 461cac7784 Koans to the Future
Summary:
The first file of Koans. Some simple constructor and `makeFuture` stuff, but this diff is mostly about the framework (ie the `TARGETS` and `main.cpp` and `Koan.h`, and the layout of the Koan files).

Known Issues: I am not nearly enlightened enough to write these in a particularly zen style with lots of inside zen-jokes, so I'm not even trying.

Test Plan:
Work through the koans,
then `fbmake runtests`,
reach enlightenment

Reviewed By: davejwatson@fb.com

Subscribers: cgthayer, exa, folly-diffs@, jsedgwick, yfeldblum, bmatheny, chalfant

FB internal diff: D2082141

Tasks: 6973057

Signature: t1:2082141:1432057657:273708f566154cc54f726b85f05457388357ef4e
2015-05-20 10:57:12 -07:00
Hasnain Lakhani 2a919488e7 Add getTotalCount() to Histogram
Summary:
Add a getTotalCount() method to the Histogram class so that callers
can read out the number of values that were put into the histogram

Test Plan: Added a new test.

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, evanmao, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2078239

Tasks: 7097793

Signature: t1:2078239:1431739799:5de3a02df67ff535e06b4a1547690440cf594824
2015-05-20 10:57:11 -07:00
Brian Watling 3d381a3571 Add 'runnable' callback to ExecutionObserver
Summary: Add a callback when a fiber becomes runnable

Test Plan: unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2081306

Signature: t1:2081306:1432011152:0ee93cb2682eb2a289b99c403e91465e72dd4ee8
2015-05-20 10:57:11 -07:00
Dave Watson 1e2aee3a27 framing handler pipeline stage
Summary: last pipeline in the original diff (HeaderServer/Client channel can also become pipeline stages in the future...)

Test Plan:
fbconfig -r thrift/lib/cpp2; fbmake runtests

canary results will follow in a comment

Reviewed By: alandau@fb.com

Subscribers: fugalh, folly-diffs@, chalfant, trunkagent, doug, alandau, bmatheny, jsedgwick, yfeldblum

FB internal diff: D2033559

Signature: t1:2033559:1430417432:c6cf4ccbf9ef26d89e7d7c5955d103348205b365
2015-05-20 10:57:11 -07:00
James Sedgwick f51b046693 RFC: FutureDAG
Summary:
See task. Set up a DAG of Future-returning tasks (optionally with executors) and eventually kick them off.
One big question is ownership. Currently the user would be responsible for ensuring that the FutureDAG outlives its own completion. This requirement could go away with shared_from_this magic maybe

Test Plan: unit. I didn't bother to test via() functionality because it's too much work for now - the functionality is trivial. Same for "true-async" dags...

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2073481

Signature: t1:2073481:1431961131:82a8898502d5308f6ab3cc8cc5b84b016d3998fe
2015-05-20 10:57:11 -07:00
Tudor Bosman 4ecf6e3f6d Make folly::format no longer crash on invalid format strings
Summary:
Yes, ideally we'd detect this at compile time, but if we can't, causing a SEV1
is not the best way to do it.

format() now behaves like formatChecked(); the old API is maintained for
backwards compatibility, but deprecated.

Test Plan: format_test

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, dpittman, jdelong, sdmich, net-systems@, dmitri, folly-diffs@, yfeldblum, andrii, mssarang, chalfant

FB internal diff: D2075829

Tasks: 7095768

Signature: t1:2075829:1431713985:b3fdec4820104b4ddc4be0b6af999db174a692d9
2015-05-20 10:57:10 -07:00
Alan Frindell 1ae1c1ff3f Fix race issue in EventBase::runInEventBaseThreadAndWait (t6779663)
Summary: Huge thanks to @afrind for debugging this issue with me and found the root cause. As per the comment from @afrind for the diff D1823407, there is a tricky race issue. The main thread could have left and reset the condition_variable from its stack but the EventBase thread tries to access it afterwards due to race and could be blocked indefinitely. This caused the server-side IO threads not able to pick up the incoming connections for the proxygen case. The fix is to use a simpler struct barrier and get a hold of the shared_ptr instead of the same object in a safer way.

Test Plan:
The original issue reproes very easily in HDFS XDC encryption case. Servers easily enter into bad state and we got high volume of timeouts from the client. With the fix, this does not happen anymore after the fix being deployed at 11:15PM. Here is the Scuba log before and after the fix:
https://fburl.com/109969805
And here is the correspond Scuba diagram for successful calls in the same test:
https://fburl.com/109971729
The throughput improved a lot after the fix.

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant, afrind, brettp, dougw, fma

FB internal diff: D2071646

Signature: t1:2071646:1431709609:10fb033536f9e4fb428dea8ba68f6a9a051616c0
2015-05-20 10:57:10 -07:00
James Sedgwick d599ae3bef fix clang build
Summary: PipelineBase needs something virtual so we can dynamic_cast from it down to the real pipeline, as we do in Proxy

Test Plan: built proxy with clang

Reviewed By: hans@fb.com

Subscribers: fugalh, mathieubaudet, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2075855

Tasks: 7077419, 7077417

Signature: t1:2075855:1431708780:03ce8d8f40cdb56d24baf75f4dc29004e5ea4c7a
2015-05-20 10:57:10 -07:00
Hannes Roth e872b4400e (Wangle) unorderedReduce
Summary:
Use this if you don't need the order of the input, e.g. summing up
values. This constructs a separate Future chain to do the reducing,
because we don't want to add locking while reducing. The only lock
necessary is when adding a new Future to the chain, which should be
really quick.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015326

Tasks: 6025252

Signature: t1:2015326:1431557191:9ea2edccb0162dedf067b5b3300de2fe72a1a4c9
2015-05-20 10:57:10 -07:00
Hannes Roth 0593c2242f (Wangle) chain -> thenMulti + thenMultiWithExecutor
Summary:
If we make `chain` a member function we can avoid the type issues and infer everything. I also added thenMulti for symmetry. Sadly the compiler doesn't like having a thenMulti with an optional `Executor*` as the first argument, it fails after some deductions. Hence `thenMultiWithExecutor`.

itssobeautiful

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2021000

Signature: t1:2021000:1431557618:169447dc9d747b23a8a1ba830e78c43713d09a96
2015-05-20 10:57:10 -07:00
Hannes Roth ad0c254dbd (Wangle) window
Summary: `window` creates up to `n` Futures at a time and only starts new ones when previous ones complete. A sliding window.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: bmatheny, henryf, scottstraw, juliafu, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015310

Signature: t1:2015310:1431557556:1017006cc9c9c2562ebe2c3dabfc4dbf316ff408
2015-05-20 10:57:08 -07:00
James Sedgwick c8250894a8 via with priority
Summary:
I wish I could just have an add(Func, priority) but the damned overloaded virtual warnings become a nightmare, so it's addWithPriority.
I also switched priority to a uint8_t in the hopes of reducing Core size. Turns out std::atomic<uint8_t> is 8 bytes anyways :( I left it that way because come on you really shouldn't be using > 256 priorities.
Biggest problem is the data race with the two atomics executor_ and priority_. Should we just use a microspinlock to co-protect them? Could probably save some size from the atomics that way.

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: hannesr, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2039619

Tasks: 6928162

Signature: t1:2039619:1431551266:3b31ed2329301aaa9c32f0f41b6e61f3482d570e
2015-05-20 10:57:06 -07:00
James Sedgwick 1185881df4 transportActive and transportInactive
Summary:
These are equivalents to Netty's channelActive and channelInactive, but we've been calling channels transports so I'm staying consistent.
I skipped integrating this into TAsyncTransportHandler because thrift still does manual CB attachment/detachment and it's unclear how that fits into this model
If my suspicions are correct, it *should* be possible to make attachReadCallback and detachReadCallback private in AsyncSocketHandler, right? And perhaps get rid of the event base modifier methods? What's our use case for those?

Test Plan: unit, employ in telnet server

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2044520

Signature: t1:2044520:1431551998:af1de358b5dbefcca148814015d8e9f63f458d5d
2015-05-20 10:57:05 -07:00
James Sedgwick f639b4542e more restrictive implicit Future construction enabling
Summary:
Decay so we don't try to instantiate this for attempts to copy Futures
See https://www.facebook.com/groups/499316706783616/permalink/863260220389261/

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: hannesr, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2062442

Signature: t1:2062442:1431551169:d1ba61537c998067ee7e6f4819f7e0817cc2e700
2015-05-20 10:57:04 -07:00
Pavlo Kushnir 62dbf939bc Disable stack recording with ASAN
Summary: ASAN crashes on std::fill and doesn't respect no_sanitize_address attribute for some reason.

Test Plan: run fibers-test with ASAN

Reviewed By: stepan@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069437

Signature: t1:2069437:1431547972:7d2c7a6547f8d76b309a76ef69fd19a1de4ce261
2015-05-20 10:57:02 -07:00
James Sedgwick df7a2d078a makeTryFunction -> makeTryWith codemod
Summary: This is more consistent with setWith, makeFutureWith, etc

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2064667

Signature: t1:2064667:1431541614:a0e3f23d5effde13a93ce58ca3e21c7c3575215c
2015-05-20 10:57:01 -07:00
Igor Sugak ec66459bbd folly: fix mismatched-tags
Summary:
I tried to fix this for default glibc but there are many mismatched-tags inside glibc itself. gcc-4.9-glibc-2.20 does not have such issue, using it.

```lang=bash
% fbconfig --clang --with-project-version clang:dev --extra-cxxflags=-Wmismatched-tags --extra-cxxflags=-ferror-limit=0 --platform-all=gcc-4.9-glibc-2.20-fb -r folly
% fbmake dev -j55 2> err
% perl -n -E 'm/\[-Werror,-Wmismatched-tags\]$/ && print' err | sort -u
./folly/experimental/fibers/TimeoutController.h:52:3: error: 'TimeoutHandle' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
folly/experimental/fibers/TimeoutController.h:52:3: error: 'TimeoutHandle' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
folly/experimental/JSONSchema.cpp:65:10: error: class 'ValidationContext' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/futures/detail/Core.h:76:1: error: 'Core' defined as a class template here but previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Future.h:392:10: error: class 'Promise' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/futures/Future.h:45:1: error: 'Future' defined as a class template here but previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:137:1: error: struct 'Timekeeper' was previously declared as a class [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:23:18: error: struct template 'Promise' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:43:18: error: struct template 'Core' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Promise.h:26:20: error: class template 'Future' was previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Timekeeper.h:23:18: error: struct template 'Future' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Timekeeper.h:44:1: error: 'Timekeeper' defined as a class here but previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/Singleton.h:378:10: error: class template 'SingletonHolder' was previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLCacheOptions.h:17:1: error: 'SSLCacheOptions' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLContextManager.h:29:1: error: class 'SSLCacheOptions' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLContextManager.h:32:1: error: class 'TLSTicketKeySeeds' was previously declared as a struct [-Werror,-Wmismatched-tags]
% perl -n -E 'm/\[-Werror,-Wmismatched-tags\]$/ && print' err | sort -u | wc -l
16
```

Updated manually. In all cases preferred tag from definition.

Test Plan:
Compile with clang dev and gcc-4.9-glibc-2.20-fb and see fewer errors:
```lang=bash
% fbconfig --clang --with-project-version clang:dev --extra-cxxflags=-Wmismatched-tags --platform-all=gcc-4.9-glibc-2.20-fb -r folly
% fbmake dev -j55
```

Reviewed By: markisaa@fb.com, meyering@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2066327

Signature: t1:2066327:1431471232:c65c2827398ba29a4022cc6a5647fac2b3aad717
2015-05-20 10:57:00 -07:00
James Sedgwick 86d0e6a9d4 SharedPromise in OutputBufferingHandler
Summary:
as above. I'm torn on whether to sugar "*this = SharedPromise<T>" as SharedPromise<T>::reset()
If I see another use case I'll probably do it

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2064449

Signature: t1:2064449:1431476780:7113366b11feaf9e8a4ea1dc60fbafb36dd46ac5
2015-05-20 10:56:58 -07:00
James Sedgwick c5dabddc2f SharedPromise
Summary: I tried two "smart" ways (deriving from Promise, encapsulating a Promise) and got nothing but trouble. The KISS principle is applied with gusto in this diff.

Test Plan: unit, integrating in 3+ places in separate diffs

Reviewed By: hans@fb.com

Subscribers: craffert, trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2035528

Signature: t1:2035528:1431393438:4e554cd30fa531d75b9267dccaade6dc516f2b15
2015-05-20 10:56:55 -07:00
Adam Simpkins f099d2b48d expose FunctionScheduler::addFunction() with a custom distribution
Summary:
Make addFunctionInternal() publicly available, as an overloaded version of
addFunction().  This allows users to add functions with a specified poisson
distribution.

This allows us to deprecate our internal legacy version of FunctionScheduler,
and replace it with the folly version.

Test Plan: Confirmed all unit tests still pass.

Reviewed By: ldbrandy@fb.com

Subscribers: jwatzman, doug, net-systems@, exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051699

Signature: t1:2051699:1431379841:f3547d1ed371503b0bf91b509a4ef03e881aa991
2015-05-20 10:56:53 -07:00
Woo Xie 6d88461ace close idle HTTPDownstreamSessions before load shedding
Summary:
when any system resource limit is reached, proxygen reduces the number of idle downstream sessions to accomodate new ones.

Test Plan:
canarying on edge241.01.ams3.  Here is the number of idle connection closed during pre load shedding stage.

https://www.facebook.com/pxlcld/ml7J

Reviewed By: afrind@fb.com

Subscribers: alandau, noamler, fugalh, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant, xning, alexkr

FB internal diff: D2030988

Tasks: 5698711

Signature: t1:2030988:1431369559:ce7328d51c7fd0afa7e9e5c19b0c66736d01fee1
2015-05-20 10:56:41 -07:00
Praveen Kumar Ramakrishnan c923afa9c1 Bump version to 38:0 v0.38.0 2015-05-11 17:03:26 -07:00
Stepan Palamarchuk 8232c2cd35 Increase fibers stack size if we're running with ASAN
Summary:
In most cases user is not aware of ASAN and fiber stack problem, thus if the stackSize is specified we're not detecting ASAN and probably will crash.
And thus it doesn't seem like a good idea to make it a user responsibility to detect ASAN.

Test Plan: tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2058741

Tasks: 7016680

Signature: t1:2058741:1431131082:9a41eb40d756c9c7af0632f7ecd55c17d10bb189
2015-05-11 17:02:50 -07:00
Hannes Roth 4a92da5b87 (Wangle) Implement collect* using mapSetCallback and shared_ptrs
Summary:
I figured it would make sense to implement all the collect* functions using a shared_ptr<Context>, instead of doing our manual reference counting and all that. Fulfilling the promise in the destructor seemed like the icing on the cake. Also saves some line of code.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015320

Signature: t1:2015320:1431106133:ac3001b3696fc75230afe70908ed349102b02a45
2015-05-11 17:02:48 -07:00
Tom Jackson 5cc2f9945c Add SpookyHashV2-based hash to StringPiece
Summary: Since the old one is weak and slow.

Test Plan: Unit tests

Reviewed By: ott@fb.com

Subscribers: trunkagent, maxime, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2052630

Tasks: 6998080

Signature: t1:2052630:1431015271:bc90ccf99941902cd4bd43a0980238c616e66abf
2015-05-11 17:02:43 -07:00
Alexey Spiridonov 675a137da2 Factor out pipe methods, add takeOwnershipOfPipes
Summary:
In order to create an EventBase'd Suprocess class, I'd like to be able to manage the lifetime of pipes independently of the lifetime of the process. To this effect, I factored out basic Pipe handling, and provided a function that detaches the pipe vector from the Subprocess object.

#6996492 a push-blocking test is broken in trunk

Test Plan: added a unit test, fbconfig -r folly && fbmake runtests && fbmake runtests_opt

Reviewed By: dancol@fb.com

Subscribers: yfeldblum, chalfant, dancol, wez, anarayanan, trunkagent, net-systems@, njormrod, folly-diffs@

FB internal diff: D1699969

Signature: t1:1699969:1430975299:30d291ab7fcc555edddf098b33095a5b29500e76
2015-05-11 17:02:41 -07:00
Alexey Spiridonov ec3c8f790f Destructor DCHECKs for EBADF
Summary: This almost always indicates a double-close bug, or a similarly nasty logic error. I don't dare make it a check, we may have some code running in production which would be broken by the CHECK, but this should give us early warning of any such bugs.

Test Plan:
```
fbconfig folly/test:file_test folly/test:file_util_test
fbmake runtests && fbmake runtests_opt
```

This test used to pass until @tudorb made me remove it due to worries about death tests messing with (currently absent) multi-threading:

```
+
+void testDoubleClose() {
+  File f("/dev/null");
+  checkUnixError(close(f.fd()));  // This feels so... wrong!
+  // The destructor will now try to double-close.
+}
+
+TEST(File, DCHECKDoubleClose) {
+#ifndef NDEBUG
+  // This test makes no sense otherwise, since this is a DCHECK.
+  EXPECT_DEATH(testDoubleClose(), "double-close-FD");
+#else
+  // That sound you hear is millions of lemmings falling to their doom.
+  testDoubleClose();
+#endif
+}
```

Reviewed By: tudorb@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2055610

Signature: t1:2055610:1431048270:a469d5c1f8182ffb74700908faa022e9613ed383
2015-05-11 17:02:34 -07:00
Adam Simpkins be171a181d update FunctionScheduler to use std::chrono::steady_clock
Summary:
Use std::chrono::steady_clock instead of clock_gettime(CLOCK_MONOTONIC).
In particular this fixes the build on Mac OS X, which doesn't have
CLOCK_MONOTONIC.

This also updates the code to use steady_clock::time_point correctly, instead
of using a raw milliseconds value for time since the epoch.

Test Plan:
Included unit tests, which were copied over from the legacy internal Facebook
(non-folly) version of this code.

Reviewed By: ldbrandy@fb.com

Subscribers: jwatzman, doug, fbcode-common-diffs@, net-systems@, exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051557

Signature: t1:2051557:1431019654:ee76cfcf8318cc3d8a8d1522b3fc97f08831ecf4
2015-05-11 17:02:31 -07:00
Josh Watzman e3eecdae17 Tweak explicit std::max instantiation
Summary: Per discussion on D2040509 this is better.

Test Plan: Still builds.

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051099

Signature: t1:2051099:1430949575:cc167b57f2d6ff42a73dee4e65d22d04932bb279

Blame Revision: rFBCODE9bdb427be1ef80b612e4f364db7809c6351cfe1c
2015-05-11 17:02:29 -07:00
Josh Watzman c9f8c1dba0 OS X doesn't support constructor priorities
Summary:
OS X doesn't support constructor init priorities, at all. AIUI,
it's a limitation of their actual binary format and loader, not just a
tooling/compiler limitation.

This particular usage appears to just be for for logging/bug-finding
purposes, so it looks like just removing the priority on OS X isn't the
end of the world?

Test Plan: g++-4.9 on OS X compiles this file now.

Reviewed By: njormrod@fb.com

Subscribers: ldbrandy, jdelong, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2040557

Signature: t1:2040557:1430975025:73f817b5d19a18dca6b19ba783dbea99192cbc41
2015-05-11 17:02:26 -07:00
Ming Zhao c5fa8a9f88 Add missing #include <gflags/gflags.h>
Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2042952

Tasks: 6783581
2015-05-11 17:02:16 -07:00
YM CHEN 56e4be21c4 fix namespace v2
Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: yfeldblum@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2043151

Tasks: 6783581
2015-05-11 17:02:15 -07:00