4,390 karma · joined July 13, 2008
This account's password has been randomized, so if you see it commenting/updating in future.. that probably wasn't me :)
So long, and thanks for all the fish!
Would capturing and sending OpenTelemetry metrics from the PostgreSQL codebase using the Fluent Bit C library + the OT plugin require a separate process on the host at runtime?
(in other words, does the library ship those events to a collector/transmitter process, or is everything handled in-process?)
A quote from their v1.9.0 release notes[1]:
> We fully support Prometheus & OpenMetrics and we are also shipping experimental OpenTelemetry metrics support (spoiler: traces will come shortly!).
(I've no affiliation with Fluent nor particular knowledge of observability, so if anyone's interested in these findings, take them with a grain of salt :))
[1] - https://docs.fluentbit.io/manual/development/library_api
(note: there is a C++ client[1], although they mention that support for C is not currently[2] a goal of the project -- and I couldn't find any C implementations elsewhere)
[1] - https://github.com/open-telemetry/opentelemetry-cpp.git/
[2] - https://github.com/open-telemetry/opentelemetry-cpp/tree/db6...
https://dbdb.io/browse?programming=c&programming=cplusplus&s...
(note that this doesn't filter on license -- I couldn't see a way to add a negative-filter on proprietary licensing -- so you may have to inspect a few of the results to confirm)
> ...
> The other challenge with this kind of dealmaking is that the stakes are so high and the counterparties so formidable that if you put anyone except an equally formidable party in place to negotiate with them, you'd end up in a bloodbath that ended up unfavorably for the company and favorably for the counterparty.
I know you didn't connect these two paragraphs explicitly -- perhaps you had something more like {vendor/partner} establishment in mind for the latter -- but in practice I think they connect fairly neatly.
They ring true based on the way that strategic initiatives from large companies and startups alike often either succeed wildly or fail completely; the outcome depends on the ability to identify an opportunity and then have motivated and ambitious people in place to capitalize on it (both in terms of negotiations and implementation).
Personality-wise: an ability to quickly assess a business relationship, summon the relevant statistics and talking points, evaluate potential outcomes (a nod to your mention of BATNA) and then communicate and work towards the preferred ones: all of that is good. Persuasion, influence and manipulation are where things get a bit murkier, I think.
Environment-wise: is your sense that many of these negotiations are zero-sum? That kind of situation would seem to be a natural fit for extremely competitive mindsets, although my sense is that it could be better to wait for positive-sum opportunities, even if that takes longer and requires {business/market} development. Return-on-capital might not always reward that kind of delay, but I think that effective human advancement does (and, ideally, regulation should try to align those two vectors).
(rationale: positive-sum situations feel like they should be easier to agree upon (aided by the presence of honest and verifiable information), should be more sustainable growth-wise for both parties, and should reduce the risk of disputes occurring later -- something that employers might not necessarily have on their mind when allowing competitive negotiators to dealmake)
> PS: I'm also realizing another interesting petri dish where we could see parts of what you're suggesting play out are DAOs in crypto.
I share your skepticism on this - the code in cryptocurrency {contracts/organizations} as they exist today doesn't stop when a problem occurs; instead the stories we hear about are expensive mistakes and losses of funds.
Code as most developers write it also has limited ways to assess the real-world effect that it has on each contracted party (not to mention uninvolved parties, often equally or more important) beyond basic metrics.
To refer back to an earlier thought: perhaps the hosting environment for the code (not the contracts themselves, necessarily, to be clear, but the hosting environment within which they're evaluate) should look for and reward positive-sum results. And (human) governance of that hosting environment should confirm that those rewards are being distributed "correctly" -- in whatever senses that's possible to determine.
(by the way: thanks for the thought-provoking discussion - even if I've gone off on some unusual tangents)
> There's nothing fungible about any of these responsibilities, so no matter how many ordinary people you stack together, you'll never arrive at even a fraction of what a competent executive can do.
I think I'd disagree with that - you state it fairly strongly, but I think that each of the items you mention is possible to delegate and distribute successfully, to varying degrees. Much of that is aided by referring to existing patterns that have succeeded elsewhere.
> Executives rotate in and out of power subject to the ongoing favor of their stakeholders, but the hierarchical structure of retaining an executive to represent the will of those stakeholders remains in place because there's no efficient way to operate otherwise.
That I do find harder to challenge (and I don't necessarily think it's worth challenging). Conversation and consensus can arise, but even then it's useful to have someone to canvas that and say "ok, I've heard the commentary on this topic, here is our selected outcome".
No doubt that's not the entire job: there's also networking, communication and thought leadership involved. But similar approaches could apply. Liability could be a trickier one.
I think there are opportunities in PostgreSQL to improve the performance of multi-string pattern matching (essentially, queries that contain a LIKE comparison against a pattern that contains multiple wildcards/placeholders).
There's some work-in-progress[1] by another person to implement Boyer-Moore[2] string search, and that'll be a good improvement I expect. I did some digging into the state-of-the-art research-wise, and found a paper[3] that presents results that improve on Boyer-Moore, and that's what I'd like to try implementing.
I have the existing Boyer-Moore patch applied locally and compiling, and have begun thinking about how to systematically benchmark and performance-test changes (ideally I want to get to the point where I have a strong understanding of my own implementation and can rapidly iterate on that with support from performance tests). But it seems to be taking me a long time to achieve progress, and my guess is that I'll be distracted by other tasks before completing it.
(it's purely a personal objective; I think it'd be very cool to potentially increase the performance of PostgreSQL and many applications that depend on it)
[1] - https://www.postgresql.org/message-id/CAO2GH9Ekvcj19ihQyjk_S...
[2] - https://en.wikipedia.org/wiki/Boyer-Moore_string-search_algo...
I believe your debating partner is presenting false dichotomies regarding resource consumption and distribution.
So now you're an expert in that field - and it doesn't feel like work, because it's what you enjoy and want to see advance technologically. Now what you do is look around to see who benefits from experience in that area. If you're lucky, they've reached out to you already.
Worst case, you've learned about and perhaps developed and furthered a technology cause that you care about. Best case, you've found an employer (and team) who want to do the same.
What's your favourite example of a static analysis success story?
The reason I ask is that I think it'd be nice if third parties could also voluntarily sign releases.
For example: the author of a library -- or one of their users -- could fund a security audit of a specific version, and then the auditors could sign to attest their findings (issue and severity count, perhaps) in a way that's visible to package users via the command-line and web.
It'd add useful information for people trying to select appropriately-secure code, and also help foster a marketplace for open source security code review.
It looks like they are supportive and keen on the changes? From the article:
> Gross's proposal was greeted with a mix of excitement and robust questioning from the assembled core developers.
It's definitely a huge opportunity, no doubt. But it affects the C ABI[1] for Python itself, something that everyone in the conversation is aware of, and that would have implications for all distributions of Python.
Add in the importance of effective code review for a 20k LoC changeset, and it seems there are good reasons to be cautious despite the optimism and excitement.
[1] - https://docs.python.org/3/c-api/stable.html#stable-applicati...
Generally I think that the ability for individuals to upload packages and for those to be available immediately for consumption is an incredible benefit for developer productivity and pace.
I'd also agree that review steps are important, so perhaps the defaults for application-side package managers should err more towards pinning to specific versions, rather than automatic updates (with the possible exception of security-related updates).
To me there are two common modes of software distribution:
- "Building-out" phase distribution: this is while applications and the libraries that they rely on are in active development -- they're 'leaf nodes' if we consider them as a dependency tree. This is where pace and the ability to iterate quickly is beneficial. Code and dependencies are likely not installed from the OS distribution, because that may not evolve quickly enough to support development.
- "Baking-in" phase distribution: the applications and libraries are maturing and have proven their worth in multiple production environments. Stable releases exist, and may have support offerings associated with them. This is where distribution-packaging shines; it makes trusted, well-tested software immediately and easily available to a very large number of users in a way that is reliable and doesn't introduce unnecessary support burdens on the distribution packaging teams.
I'd love to see more code review at various stages of software development; we typically do it during changeset development (pull requests), but I think that pre-release code review would make sense, and I also believe that post-release 'diff' code reviews would be valuable (what has changed between v1.1 and v1.2? do reviewers agree that it is safe, correct and documented?).
The growth areas I'd expect to see in software development careers are in maintenance and review. Perhaps not hugely exciting compared to feature development, but (again, my opinion only) more important for stability and overall software quality long-term.
From the English-language translation: "We hope that the national steps still needed to make this decision will be taken rapidly within the next few days."
Human editors may cause the content within a CRDT datastructure to become inconsistent in the sense of "is this document understandable by another person", but they can't cause conflicts that block the editing process on-disk.
On the other hand, git merges can -- and frequently do -- involve conflict resolution that isn't effectively handled -- especially in a distributed system -- by automated measures.
Sounds like you've got a good, considered approach, so keep up the work to improve the environment and keep in mind other ways you can develop (and enjoy) your time is probably the best platitude I can offer (and one you probably already understand).
Don't forget that technology can be a useful fallback and/or way to confirm that you remembered details correctly (and you won't, always; it's always possible to improve, though). Good luck!
It's understandable (and not necessarily wrong) to vent here, and I'm not going to ask whether you have done this already, because it's not useful to draw out an answer, but: if you can, find the relevant anti-corruption agency and report the events to them.
There's a chance that doing so might increase the likelihood of harassment again in future -- I have no idea how these things show up on border entry systems -- so it's your choice, but the key idea is to get a paper trail in place that will show up if (and hopefully when) investigations are later taking place.
I haven't worked in law enforcement, but I imagine that the same way that good policework involves collecting statements and records of events to put together a case, internal (or independent) investigations into misconduct require similar records. The ability for individuals to get away with bad conduct is, I'd guess, based on people believing that it's not worthwhile to file reports against them. They do get caught when they are.
You could be correct, maybe this would work in practice, trading on the reputations of those companies. It doesn't feel particularly open or community-oriented, though. Why not present a trust graph built from a broad set of worldwide users instead?
(one of the benefits to a trust graph would be the volume of signers; perhaps you wouldn't want to weight each signing equally -- again referring back to something like the distance metric mentioned previously -- but for even mid-popularity packages, the detailed review possible by careful users of a package could, I expect, be more reliable than automated-and-manual review by one or two large companies)