coreytabaka

IO scheduling is a separate problem, though it shares the same fundamental properties. The same goes for network packet scheduling. All of these are ongoing efforts. Stay tuned! :)

A simple hierarchy: when there is deadline work to do that work takes precedence, fair work gets the rest of the time. This is effective because deadline work has bounded execution time, whereas fair work is elastic and can adjust to use the available bandwidth.

No, that's based on the deprecated multi-level round-robin scheduler, which we find tremendously amusing. :)

It does imply that. Multiple concurrent scheduling algorithms is nothing new, Linux and MacOS both support per-thread algorithm selection.

The library is transport agnostic. There is a Reader/Writer abstraction to adapt to any transport you like.

https://github.com/google/libnop/blob/master/docs/getting-st...

In most cases the context provides enough information:

  void Baz(Bar*);
  void Baz(const Bar&);

  void Foo(Bar* mutable_bar) {
    Baz(*mutable_bar); // Not mutated.
  }

  void Foo(Bar* mutable_bar) {
    Baz(mutable_bar); // Possibly mutated.
  }

  void Foo(const Bar& bar) {
    Baz(bar); // Not mutated.
  }

  void Foo(const Bar& bar) {
    Baz(&bar); // Compiler error.
  }

The rule doesn't perfectly eliminate ambiguity, but it does a pretty good job overall. The point is to address the general use cases with familiar constructs that work across multiple toolchains.

For the most part the optimization works well. The more subtle issue is that the union trick used by the endian utilities is not compatible with constexpr expressions. There are some interesting use cases for constexpr serialization that fail if the conversions are interposed in the Reader/Writer types. The alternative is to use reinterpret_cast, which is also incompatible with constexpr expressions.

Moreover, endian conversion templates are unusually frustrating in the current C++ standard. I wish there were a better way, but one does not currently exist.

The table approach is fairly flexible. Both Protos and FlatBuffers do more or less functionally similar things. What specific deficiency do you see?

Take a look at the binary format spec:

https://github.com/google/libnop/blob/master/docs/format.md

The format is designed specifically to provide enough structural information that the binary format can be parsed without knowing the original structure definitions. This makes it easy to write a binary-to-string converter than can grok any payload with minimal complexity. Message observability was a primary goal during development.

Yes! Hopefully C++20 or C++25-ish will provide better alternatives for compile-time reflection.

Don't forget that libnop also has NOP_EXTERNAL_STRUCTURE (and friends) to decouple the annotation from the structure definition. This is handy when you have a C ABI with C++ internal implementation. I don't recall seeing a similar facility in other libraries.

Supporting embedded platforms is one of the objectives of the library. Feel free to open a ticket on github if you have any further questions.

Thanks! MessagePack is one of the influences.

I tried to keep it C++11, but generalized lambdas are critical in important use cases (see nop::Variant). Besides, GCC and Clang have solid C++14 support. C++17 is another story... ;-)

Since you mention it, check out the experimental RPC support:

https://github.com/google/libnop/blob/master/examples/interf... https://github.com/google/libnop/blob/master/include/nop/rpc...

I haven't documented it yet since it's not fully baked, but it's quite functional nonetheless. I have a working prototype of RPC over USB between a host PC and a Cortex-M micro controller. The ability to define constexpr dispatch tables is very convenient in a micro controller environment.

This argument comes up a lot within Google. The style guide is revisited frequently (for example =delete in the public section is now the standard for disabling copy and/or move/assignment vs. the old macros in the private section).

Non-nullable types are helpful for implying pre-conditions. However, readability at the call site (e.g. Foo(&bar) might mutate bar, whereas Foo(bar) should not) is still considered more valuable in a large scale codebase. Passing nullptr as a pointer argument is generally assumed to not be okay unless explicitly documented as permitted -- this is opposite the assumption that you stated.

There are places exceptions are made, where the use of pointers is deemed more confusing than non-const references (e.g. move-maybe semantics in very specific cases). Ultimately, most code follows the default style guide.

Besides, nullptr dereferences are pretty easy to diagnose in a library like this. And more often than not everywhere else too.

Thanks for considering using libnop!

This library works well in an embedded context. I've personally used it on Cortex-M class micro controller firmware.

The serializer/deserializer does not require dynamic memory allocation. Whether or not dynamic allocation happens depends on how you use the library. If you avoid using data types that perform dynamic allocation (e.g. std::vector) and use (or write) Reader/Writer types that use static memory (e.g. nop::BufferReader/nop::BufferWriter) then you should be fine.

There are some nice tricks you can use to permit protocols that have convenient dynamic containers on the host and static versions on the embedded device:

https://github.com/google/libnop/blob/master/docs/getting-st... https://github.com/google/libnop/blob/master/docs/getting-st... https://github.com/google/libnop/blob/master/examples/shared...

Endianness is assumed to be little because the vast majority of hardware is little endian. There are utilities to convert here:

https://github.com/google/libnop/blob/master/include/nop/uti...

These are not currently used by the Reader and Writer types included with libnop for efficiency, but are available for you to use in your own Reader and Writer types if you really need it.

Floating point is a much stickier problem due to lack of standardization across hardware. The library does not address this automatically and just packs floating point types in machine order. However, the library provides tools to help address the issue. One approach that works well is to use a fixed point representation for the wire type -- a value wrapper type is convenient for automating this:

https://github.com/google/libnop/blob/master/docs/getting-st...

See the Fixed template in the example. This is especially handy if you have a micro controller without floating point support or you want to minimize the size of the payload at the cost of range and/or precision.

The library has a complete suite of tests and 97.9% line coverage according to GCOV, with particular attention to conditional and error paths.

We use the library for a few internal embedded prototypes. I have not tracked its usage outside of Google since its recent release.

Best, C

unique_ptr and nop::Optional<unique_ptr<T>> is on the way. shared_ptr opens up the ability to create cycles, which are not supported yet.

It means not currently an officially supported project. Most open source releases of internal code start out this way and may or may not become officially supported. It is different than deprecated, which was once supported but no longer.

Primary author and maintainer here! Happy to answer any questions anyone has.