Folly: An open-source C++ library developed and used at Facebook
github.com
github.com
folly is a community effort within Meta; it doesn't have a specific theme about what's included in it, but it is a set of core libraries that are likely to be depended on by most C++ software within the company, and are enough high-quality and self-contained that they can be useful externally, or showcase implementations undergoing standardization efforts (for example experimental/coro for coroutines).
Main focus is server applications, but it is also used in mobile, in particular Thrift (the serialization/RPC infrastructure) depends on it.
A very personal list of favorites:
- experimental/coro is the coroutines infrastructure, which is now widely adopted.
- SharedMutex is a heavily optimized mutex which supports shared/upgrade/exclusive semantics, and allows near-linear scalability in shared mode. It is used pretty much everywhere as it is the default mutex for Synchronized, a wrapper to safely synchronize access to objects. Its footprint is just 4 bytes.
- DistributedMutex, OTOH, is an exclusive-only mutex that supports flat combining, and performs extremely well under high contention.
- MPMCQueue and UnboundedQueue are lock-free MPMC queues, respectively bounded and unbounded, which are used in almost all our thread pools (see the executor/ directory)
- Userland implementations of RCU and hazard pointers, for deferred reclamation.
- LifoSem and Baton are other fundamental synchronization primitives that are not available in most core C++ libraries. They do a magic trick internally, where if they're waiting long enough, they unmap the unused suffix of the stack, so idle threads don't hold unused stack memory.
- Function is like std::function, but move-only, so it can hold non-copyable function objects (think lambda that captures unique_ptr). It is the vocabulary type for callbacks used in futures and executors.
- F14 is a SIMD-optimized hash table. Similar to Abseil's SwissTable, it was published around the same time.
- TDigest is a highly scalable quantile estimator, widely used for service counters (see fb303, also open source).
- (shameless plug, as I wrote this one) enumerate() is a replica of the Python function, so you can do
for (auto&& [i, element] : folly::enumerate(collection)) { ... }
Right now the title reads essentially: "Here's a library".
It takes about a minute to find any information at all about this library besides the fact that it's used at Facebook (by the way, did you miss that? it's used at Facebook! Facebook!!). And even that information is empty bromides about how it's "designed with practicality and efficiency in mind". Hmm, I prefer my software to be impractical and inefficient...
All I take away from this is that Facebook has an irrationally optimistic expectation of how the world views its engineering prowess.
Right in the intro section, second paragraph, it says "It complements (as opposed to competing against) offerings such as Boost and of course std". Come on, what more do people want?
Professionals often don't have that choice. Cue many thousands of comments on HN why working in c++ sucks.
Looking to implement a memory efficient search engine.
the opposite from practical is general. for example is std not practical but general. Its api is not optimized for 95% of the use cases but rather so that everything is possible and equaly verbose.
the opposite of efficient can for example be trade of for readability, portability and cost (as in development time).
For me as a possible user of the lib i found this information critical. I write always similar info in the readme for my libraries.
No, it's certainly not at all. Those are completely unrelated spectrums. Something can easily be practical and general, or impractical and specific.
> the opposite of efficient can for example be trade off for [...]
And if they stated what trade-offs they made, then that paragraph would actually start to say something useful. Unfortunately, they don't.
Also, if you're developing documentation for libraries, I really would recommend finding a better source of inspiration. Take Redis, in the very first paragraph:
> What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a server-client model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
That's all you need to do: before anything else, tell the user what the thing does. Then you can give some basic details about implementation: "written in Go", "runs on Linux & macOS", "uses GC, so not suitable for RT applications", etc. After that you can get into trade-offs, real narrow implementation details, setup, handwaving abstract adjectives, etc. But lead with what the hell it actually does.
Let me guess: English is not your native language, right?
That said, this is an issue for other languages as well. Another example: the "Google core libraries for Java": https://github.com/google/guava
If I was using java or python I'd definitely not use the standard things most of the time either because it's more than likely that there's a more efficient way in some other library, and because my main paycheck comes from "can you make this as fast as technically possible given this average consumer hardware"-problems
1. exception-safety guarantees
2. real-time safety
2 is caused by 1, because to be exception safe requires a heap allocation, and that's not RT-safe.Who would want to work in a language where you could not make these choices?
std::variant does not do this.
This comment makes no sense at all. I mean, would it make any sense to also criticize JavaScript/NodeJS for their relatively spartan standard modules and extensive use of vended modules from the likes of npm?
And have we learned nothing from the mistake of going with the "everything and the kitchen sink" that is java which failed to actually satisfy basic needs and resulted in projects like guava and apache commons and vavr stepping in to provide alternatives? And how about C's infamous standard library with it's security vulnerability-inducing standard functions which remain in the standard?
Perhaps we should learn lessons and avoid basic mistakes such as setting libraries in stone. C++ learned that lesson, and so did C#/.NET, and up to a point Rust. Why unlearn them?
Uh .. Yes? Yes, very much so.
I've never worked with a place using C++ that didn't make their own core libraries. As a low-level language, there's almost always a way to get some performance benefit by starting from scratch for your specific case.
(yes I've heard all of these "reasons")
1: Just calling it "the STL" is a key sign of living in the past--pretty much nobody uses the actual STL anymore: What we use is called the C++ Standard Library.
I won't deny your prior, but my prior is that people who complain about the use of the word "STL" to mean "C++ Standard Library" (and ignore that the latter is 10 times as many keystrokes) do so only to flaunt their knowledge, not because anyone actually experiences confusion nowadays.
Supporting evidence: https://github.com/microsoft/STL
For example iostream isn't part of Stepanov's idea, Stroustrup created that bundle of craziness in the 1980s and today it's part of the Standard Library, but it was not part of STL which only came into existence in the 1990s (although Stepanov's ideas about generic programming are older).
C++ is a massive beast but it works.
This has library contents: it’s a mix of various utilities - type conversion, data structures, memory management, random numbers, concurrency, etc.
https://github.com/facebook/folly/blob/main/folly/docs/Overv...
However, I did have decent success taking the classes I was interested in (CPUThreadPoolExecutor, and surrounding), stripping the library down to just those bits, and using them. Obviously a PITA to stay current with upstream, though.
If nothing else, it's worth reading for some of the highly-optimized stuff going on in there.
Which unfortunately means Folly is an extremely heavyweight dependency to pull in compared to Abseil, if you're not already using Boost
I mean, yes, Boost itself is big. But when I think a heavyweight dependency, I think of something that itself pulls in a shitton of dependencies Javascript-style. And AFAIK Boost doesn't have any dependencies.
At one point, I had a curated SVN repository of certain boost header-only includes that didn't require any sort of build integration, had no dependencies of their own (even other boost libraries), and didn't require any build system integration - and I know I wasn't the only one.
The world of native software development was just so unbelievably different from that of web development - it's really no surprise developers from that lived through that era have such a hard time accepting the modern-day age of PWAs in Electron sandboxes masquerading as native programs. People that cut their teeth developing desktop software with something like Python (just as an example) instead would have a completely different perspective on all this, of course.
Start using some of the heavier weight sub-libraries and you can quickly explode your compile times. I have boost beast/asio on one project and it takes over a minute to compile one of the source files…
I understand the FB rationale to have it, but what whould be a reason for other projects to adopt this lib?
I won't imagine maintaining such a lib as a dependency by ourselves. Maybe for one short-term purpose, but then it's still safer to go with standard or boost, or, God forbid, carve out some pieces and wrap them up.
More precisely, one that defines a datatype (no SDS-like trick) and has all goodies like SSO, CoW, views, thread-safety, etc..
I'm making such a lib but can't seem to find other examples to compare.
It's not c but almost c, it could be useful for you
[1] https://github.com/MoustaphaSaad/mn/blob/master/mn/include/m...
There are a few pure-C made STL alike containers for C that uses no c++ code at all, a random github search finds this: https://github.com/assyrianic/Harbol , there are quite a few of them just not recalling them now.
folly [https://www.wordnik.com/words/folly]
fŏl′ē
noun
Lack of good sense, understanding, or foresight.
An act or instance of foolishness.
A costly undertaking having an absurd or ruinous outcome.
At the risk of starting a meta conversation (pun intended) I wonder why the team chose this name for a set of core library components. The GitHub page indicates it is a loose play on an acronym. “acronymed loosely after Facebook Open Source Library”
Yet why accept a name with a negative connotation.. The name leads the product, so why not at least make it _neutral_?I am also partial to explicit error handling over exceptions, and there is a lot to like about absl::Status (and absl::StatusOr).
Disclaimer: I work for Google but the views expressed are my own and don't necessarily reflect those of my employer.
Conan
Vcpkg
Cpm.cmake
A lot of projects these days use conan.
Developing in c++ vs (for example) python does seem like more work. I think it is a consequence of offering more control, which is expected in the c/c++ community.
What the two options offer.. Is different. With a higher level language, you can get more work done for less cost [1]. With a lower level language, you get more control —- which is great, but you shoulder that responsibility yourself. And you recognize this already..
If you want to move with the crowd (which the higher level abstractions are likely tailored for), then you will be able to move fast and far. When you need to exert more control, however, things get tricky.
For example managing the OpenSSL library used in your project independently of the system library?
Writing thread-safe code isn't easy, and it isn't made easier by libraries that offer footguns like Synchronized. It's easier to get it right with compiler thread annotations than with Synchronized.
Synchronized cannot guarantee thread-safety, but in my experience wrappers that do not allow unsynchronized access to the object are better than the alternative. Annotations are just another way to guard access.
Thread annotations also effectively prevent unsynchronized access to the object, but they do it in a way that programmers make fewer mistakes with, and they do it with a less error prone mental model: mutexes are seen as guards of _class invariants_, not merely as guards of _data members_. Apart from encouraging a superior mental model, thread annotations also encourage a one-mutex-per-class model which I've found far more successful and error prone (both in code I've reviewed and in code I've written) over a one-mutex-per-member model which inevitably (in my experience) results in fewer bugs.
However, the API encourages practices that make it very hard to recursively acquire the lock (Synchronized is about access to objects, not critical sections).