sixdimensional

Are you familiar with the concepts of SDET (software development engineer in test) or software test/QA? I’m asking genuinely because in our industry, traditional quality control sometimes seems to have went out the window in the rush for speed.

Perhaps look for prior art on being a third party/contract SDET or test/QA engineer?

I suspect you’d need to look for a specific class of opportunities where you can work on bugs under contract for customers, or, look for things where you can work on bugs externally as a third-party on public products and services, things like chasing bug bounties.

With many companies trying to abolish more traditional quality controls for software, by having developers conduct more test driven development, self driven integration testing, etc., I am not sure if you can be as successful as an external third-party offering bug fixing services, unless it’s in the right circumstances.

Yes, that's true - those products are definitely more of trying to displace the open source tech by borrowing the interface. I still think it is a plain admission to the fact of the popularity of the original tech (e.g. MySQL and Postgres). For sure, credit due where it is due for implementing entirely new implementations - but it's such a common pattern to "borrow" the open source interfaces and reimplement the core engine.

That said, you didn't mention things like RDS or Google Cloud SQL, which are cloud-based versions of standard open source dbs.

This was both shocking and not surprising at all, all at the same time.

It’s worth noting how cloud vendors have fallen back on tried and true databases such as Postgres (mind you, often replacing the guts of those databases with new implementations.. but still).

Honest question about step #3, there's a theory of YAGNI (you aren't going to need it) I feel going on here. Information overload/hoarding is not good either.

One thing I cannot shake is, there are things that I know I learned, but I can no longer remember. Without a prompt, a shred of something from the past - a note, a book, something - I often cannot recall it anymore.

People with brains that suffer from memory loss/recall issues/Alzheimer's/dementia/long term retention - etc. - how does doing #3 address that? I feel like maybe you don't need to look at it all the time, but couldn't having that around as a reference still be a prompt for you at some point later in your life, where maybe you can't just remember everything?

This is strange. Lately, I've had a number of people in different situations (especially at work) tell me of similar efforts pushing back on organized process or.. organized anything, including note-taking, task tracking and more. Is there some kind of attempt at a mental revolution here or is this some kind of response to people being overloaded and not wanting to carry the mental burden/weight/let it go?

I almost feel like this should not even be a topic. Using tools to help organize thoughts - why do we care if it helps people? - not all brains work the same. What is wrong about organizing one's thoughts and spending time doing so??

I can see the argument that one can get lost in a meta-loop of sorts, but that's probably true of anything when you take into account procrastination - or certain brain types - that are prone to loops. For some people loops feel good, right? I say this as somebody who is lightly on the spectrum.

Even the pushback against knowledge management as a practice is perplexing - there was a time where Peter Drucker, for example, thought knowledge work would someday be all there was left to do.

That said, I can't help but wonder if there really is something bigger/some kind of trend going on here and I can't quite put my finger on it. Is this something evolutionary? Are we trying to evolve/seek a better way as an organism?

Is anybody else seeing/feeling the same trend?

This seems to be a kind of Petri net[1] almost.

[1] https://en.wikipedia.org/wiki/Petri_net

How about open sourcing and putting this on Github and letting anyone submit pull requests with suggestions/ideas?

I was like you once.

You have plenty of raw ability, and are capable of a lot, you've proven that. That can be hard for employers to digest when you apply for jobs "through the front door" (e.g. job boards, career sites, etc.).

A bigger question that might help guide you is, what are you hoping to do in your career next? Focus/clarity may help you here, especially to emphasize the skills that give you a better chance of being matched to and landing a job.

Also, I mean this with all sincerity, great for you for reaching out on HN. Do whatever you can to network, because the people who know you and know what you are capable of can be far more helpful in many cases than "going through the front door" applying for jobs. Going through the front door is notoriously difficult.

I don't promise to be able to solve anything, but reach out if you want to chat. I have 20+ years experience and started out much like you.

Email in profile.

Yes, but some municipalities figured that out and banned anything that appears to be an RV or trailer parked for longer than a certain period of time.

Zoning laws for land often also include clauses such as, if you purchase land X in zone Y, you must construct a home of N,000 sq ft by time T or you are fined.

After looking into achieving personal freedom through tiny home living (and which could take different forms than what we think of today with some creativity), I can say that one of the most challenging things standing in the way is zoning laws.

You'd be surprised how difficult it is to find any piece of land anywhere in the US where zoning allows relatively unfettered building of tiny or smaller dwellings. It's possible, but it's very difficult.

I too was also about to say the same.

I was literally just working on a project with TatSu[1] in Python which contains language elements that are conditional based on patterns in the syntax. I found that the added RegEx matching to EBNF-like syntax was quite powerful.

My only issue was the generated parser appeared to parse the entire content into memory rather than stream parsing.

I haven’t read enough on ixml yet, but, while it seems like while it would unlock many existing toolkits (I never hated XML but went away from it as the industry did), it seems like parsing through an XML format should be done in a stream mapping fashion, and not persisting data, for the XML to be truly “invisible”.

Adding the overhead of XML back into the processing chain… hmm, honestly have to think if the value of accessing the data and XML toolchain is worth it.

I’d almost rather see something that can read any input as a stream with a grammar and produce a stream (that can be materialized) of more optimized, yet open format, that can be compressed but handle complex types.. like protobufs, or flatbuffers. I’m not sure that humans need to read raw data files so long as we have great and open tooling to view data in binary formats (iirc, the biggest argument against XML and the added overhead originally).

[1] https://tatsu.readthedocs.io/en/stable/index.html

I worked in the data federation space for a number of years (it's actually quite an old term, I worked in it back in 2013, around the time there was an early wave of activity around this and the concept of a "data fabric").

When I saw GraphQL come out, I knew that what you are saying would happen.

In the data federation tool I worked in, SQL was the interface abstraction to join across heterogenous platforms (think of things like Presto/Trino or Dremio). GraphQL as an interface requires the same underlying infrastructure as that data federation tool I worked on in terms of query analysis, parsing, planning, optimization, execution, etc.

Those are "hard problems" due to lack of standardized interfaces, access patterns, direct data access, I/O, network bandwidth and infra related latency, costs, compatibility, data types, etc. These problems are distributed system problems coupled with often incompatible interface layers (e.g., even if you are using multiple SQL databases with GraphQL, you run into the same).

If your scale is such that you can build GraphQL on a handful of systems and for a handful of use cases, great! If you have to go to a certain larger scale, you're back into federation territory (which in the app layer might also be called API composition).

One potential option - when you reach the point where you need complex GraphQL query coordination, more than seems to make sense to implement, pair it with a data federation tool such as Presto/Trino, Dremio, Denodo, or research approaches such as caching/materialized views (engines like that are becoming decoupled from databases, such as Materialize.io) - and let those engines do the hard work.

In that case your work becomes more like GraphQL -> SQL or API -> a data federation, caching or materialization platform. CQRS and event sourcing plays a role here too.

Consider also, the possibility that if you are willing to accept a bit of delay in aggregated results from multiple systems, doing those compositions or aggregations in the data platform layer, and simple feeding those to the GraphQL interface. That could even be done in a single database/data platform if you really wanted without too much fancy federation tech.

Federation is powerful but complex. It seems like a fun hard problem, but for many tech teams, it can be a complexity and time suck. My recommendation would be try to avoid building that if you can.

This is not entirely true anymore. Many aerospace organizations attempted to move to less rigid/structured development process (e.g. wishing to become more agile) to attempt to "move fast and break things".

This trend actually has lead to an decrease in software quality and more quality escapes.

The larger aerospace manufacturers have mostly stuck with more structured process for safety/mission critical process, but around the edges (e.g. other systems running on aircraft), less structure and faster turnaround/more "testing in the field" is happening.

There was always the "skunk works" type of teams in aerospace organizations who worked with a lot less guardrails as well.

Source: worked for an avionics company previously.

Sounds like a federated query engine with a cost based optimizer. I worked for a company that went pretty far down this path using another database.

Definitely a lot of potential, and also a lot of potential gotchas.

Translating from one SQL syntax (e.g. Postgres) to others, while maintaining the full capability of the other system, for example, turned out to be quite complex (but doable).

Will be following your project and wish you all the best. I suspect if you keep things sharp/focused and don't go too crazy with promises or uses cases, this could be quite successful.

Is this going to stay an intelligent router and sort of proxy? Or do you have plans for federated, heterogeneous joins, for example?

That's where things get interesting :) I think there's a lot you can do without even having to go that far.

The fact that we work in an industry where 3-5 years is considered senior still boggles me sometimes.

Why position it as a database server if you could position it as something else, like an interface server? Just curious.

I personally feel Watson was an extremely clever marketing boondoggle. If you think of it, machine learning, neural networks and AI were just making a return into the public mind around the time they announced Watson.

I think somebody thought if they "humanized" AI by making it seem like it was a character, it would make AI seem all that much more closer to the dream.

On the face of it, not a horrible idea, but applied to what was essentially a bunch of separate algorithms.. pretty misleading, but that's just an opinion.

I'm glad to see this here. I found out about this from the employee side when I was asked to interact with the site, and when I dug deeper was shocked with what I saw.

I've been telling friends/colleagues but many didn't honestly believe me.

After I found out about this, I noticed a pattern in new jobs - they always knew exactly what offer to make me that was about 10% higher than my current salary. I mean, the offers were like clockwork. I strongly feel prospective employers were using this information to decide on what offer to make.

I can summarize my feelings about this pretty succinctly.... this is not ok.

I'm not qualified to help you in this area, but I wanted to add that I actually did try to do this once using a much simpler approach (it was kind of a REPL + conditional logic or fact-based rule engine). I called it "Fred".

One thing I thought really interesting and had noticed (and I think you have also) - current digital assistants don't actively/autonomously engage with the user much. Especially not in any way that would make me think they are "friendly". This is definitely something that can go sideways quickly, but on the face of it, I really do love the idea of a benevolent AI that actively tries to interact with me in a safe way (with the ability to shut it off when you don't want it to!).

Wish you luck and if you have anything I can use to follow your work, would be interested!

It's a good suggestion, and we need volunteers more than ever (if you can dodge COVID while volunteering). Taking care of others helps you take care of yourself?

I actually understand this and wonder how many others feel this way.

Why don't we?

Thank you for this very interesting share, I had not heard of this before!

This, to me, is a general problem with a purely quantified, metrics driven approach to management, not just limited to NPS. I'm not demonizing these approaches, just saying that you have to balance humanity with data.

I like to look at this kind of thing as the one of the dark sides/dark patterns of using data for decisionmaking.

Qualitative measures are important, as is maintaining as much humanity as possible, to have a balanced and healthy culture. Being solely metrics/data driven can lead to cold, heartless, damaging culture (might be efficient or make profit, but very dehumanizing).

This seems like a case for an axiom I hear infrequently, but I think comes up a lot - things that seem like they should be simple and easy, but are in fact difficult.

To answer my own question, I suppose both roles (CIO and CTO) benefit from having coding experience, maybe CTO more than CIO assuming a CTO is supposed to be "closer to the technology engineering".

I am a little confused why we are talking about CTO and not CIO (chief information officer). I thought the latter is the role responsible for information systems.

Although for years we have been told that CIO means "career is over" and was a dead role. Which I never fully understood.

"Essentially the government has built a surveillance state by outsourcing it to private enterprise."

There's a rather innocuous sounding name for this - "public private partnership" [1]. If you've ever experienced this scenario first hand, you'd truly be surprised how much government is run in partnership with private enterprise.

[1] https://en.wikipedia.org/wiki/Public%E2%80%93private_partner...

I am following you here, but I also took this another way.

I think what this comment was about was a person who recognizes his own mortality but could put that against the backdrop of the bigger picture of our own shared human mortality.

I believe Shatner was recognizing the pure inhospitability of space to human life and the fact that we are propelling ourselves into it. If anything I think this was precisely a commentary on the very nature of the risk and adventure that space represents. Also a recognition of how much we thread the needle even just living on the Earth.

For if it were not that true exploration and adventure came at risk of life and limb, if the very risk of living did not include the risk of death, then what is it really? Thinking of the early explorers of the Earth, many who died doing so, and who risked it all in the name of exploration and adventure.

If I ever heard a more realistic characterization of what it truly means "to boldly go" where no one has gone before... I can't think of it.

I am eager to see further debriefing from Shatner, I think he will have much to say. In so far as they wanted to achieve sending a different kind of personality to space, to learn what human experience would be like, I think this was an incredible success.. putting all the commercial aspects aside.

For what it's worth, I actually do understand your sentiment. I am also a pragmatist and practitioner and find that academic theory sometimes has to "bend" for lack of a better word.

That said, I was brought up to find what is useful and good regardless of whether it comes from academia or practice. It's a beautiful thing to be able to find a balance of building an operational world view that borrows from both theory and practice.

It is my personal observation that much theory is based on observation of practice + theoretical projection, followed by more observation - i.e. the scientific method.

I loved Adam Savage's comment that sometimes science boils down to doing stuff and writing it down. I think as long as we find a good balance in learning from the past and building on it, while not letting the past limit our future endeavors for no reason, we can have the best of both worlds.

Actually, there is a entire field dedicated to this (disclaimer: I studied it at uni, plus CS and info systems) - organizational studies [1]. It has subdomains such as organizational structure/models, behavior, communication, etc. Much of it backed by theories, studies and more, ranging in disciplines.

Time and motion studies (for example) were part of the scientific management revolution for industrial management [2]. There have been both qualitative and quantitative studies of all kinds of things - organizational forms, people networks (things like, Dunbar's number[3]), power distribution (ex. work of Pfeffer), etc. I could go on.

I do think we are reliving an era of interest in management by data and metrics, much like that of the industrial revolution and scientific management. Nothing wrong with using science and quantitative measures to optimize, but any human who has been subject to purely management by quantitative objective will likely tell you it often becomes.. rather, inhumane. This is often what led to automation, I feel - to remove the human element that was crushed by industrial efficiency.

I suspect this is why the qualitative balance is important (and no less scientific - science can be logic not just metrics right?).

My two cents...

[1] https://en.wikipedia.org/wiki/Organization_studies

[2] https://en.wikipedia.org/wiki/Time_and_motion_study

[3] https://en.wikipedia.org/wiki/Dunbar%27s_number