mjw_byrne

They're pretty different in British English, I struggled to figure it out until I started thinking about how it would sound with an American accent.

I've done a from-scratch billing system build. As well as the complexities in the article, some of the logic required can result in rather exotic SQL.

Answering questions like "what was the maximum number of concurrent sessions per account between these two dates" with a SQL query is interesting. Making it perform properly adds a layer of fun.

It's 40% above a threshold; the trouble is that the threshold is laughably tiny when you compare it to house prices.

It's also relatively easily defeated by transferring assets to others well in advance of your death, which is the kind of thing very wealthy people are more likely to be able to arrange.

It's good for a delimiter to be uncommon in the data, so that you don't have to use your escaping mechanism too much.

This is a different thing altogether from using "disallowed" control characters, which is an attempt to avoid escaping altogether - an attempt which I was arguing is doomed to fail.

> I'm not clear why quotes prevent parallel processing?

Because of the "non-local" effect of quotes, you can't just jump into the middle of a file and start reading it, because you can't tell whether you're inside a quoted section or not. If (big if) you know something about the structure of the data, you might be able to guess. So that's why I said "tricky" instead of "impossible".

Contrast to my escaping-only strategy, where you can jump into the middle of a file and fully understand your context by looking one char on either side.

> Do you really have a use case where reading itself is the performance bottleneck and you need to parallelize reading by starting at different file offsets? I know that multiple processes can read faster from certain high-end SSD's than just one process, but that's a level of performance optimization that is pretty extraordinary. I'm kind of curious what it is!

I used to be a data analyst at a management consultancy. A very common scenario would be that I'm handed a multi-gigabyte CSV and told to "import the data". No spec, no schema, no nothing. Data loss or corruption is totally unacceptable, because we were highly risk-sensitive. So step 1 is to go through the whole thing trying to determine field types by testing them. Does column 3 always parse as a timestamp? Great, we'll call it a timestamp. That kind of thing. In that case, it's great to be able to parallelise reading.

> And data corruption is data corruption

Agreed, but I prefer data corruption which messes up one field, not data corruption which makes my importer sit there for 5 minutes thinking the whole file is a 10GB string value and then throw "EOF in quoted field".

I agree that it's best to pick "unlikely" delimiters so that you don't have to pepper your data with escape chars.

But some people (plenty in this thread) really do think "pick a delimiter that won't be in the data" - and then forget quoting and/or escaping - is a viable solution.

Yep, we had a constant tug of war between techies who wanted to use open-source tools that actually work (Linux, Postgres, Python, Go etc.) and bigwigs who wanted impressive-sounding things in Powerpoint decks and were trying to force "enterprise" platforms like Palantir and IBM BigInsights on us.

Any time we were allowed to actually test one of the "enterprise" platforms, we'd break it in a few minutes. And I don't mean by being pathologically abusive, I mean stuff like "let's see if it can correctly handle a UTF-8 BOM...oh no, it can't".

Right, but the original point I was responding to is that control characters are disallowed in the data and therefore don't need to be escaped. If you're going to have an escaping mechanism then you can use "normal" characters like comma as delimiters, which is better because they can be read and written normally.

It is true for everything that uses quoting, I didn't mean to imply otherwise.

If you disallow control characters so that you can use them as delimiters, then CSV itself becomes a "binary" data format - or to put it another way, you lose the ability to nest CSV.

It isn't good enough to say "but people don't/won't/shouldn't do that", because it will just happen regardless. I've seen nested CSV in real-life data.

Compare to the zero-terminated strings used by C, one legacy of which is that PostgreSQL doesn't quite support UTF-8 properly, because it can't handle a 0 byte in a string, because 0 is "special" in C.

The idea that binary data doesn't go in CSVs is debatable; people do all sorts of weird stuff. Part of the robustness of a format is coping with abuse.

But putting that aside, if the control chars are not text, then you sacrifice human-readability and human-writability. In which case, you may as well just use a binary format.

There's just no such thing as a delimiter which won't find its way into the data. Quoting and escaping really are the only robust way.

I used to be a data analyst at a Big 4 management consultancy, so I've seen an awful lot of this kind of thing. One thing I never understood is the inverse correlation between "cost of product" and "ability to do serialisation properly".

Free database like Postgres? Perfect every time.

Big complex 6-figure e-discovery system? Apparently written by someone who has never heard of quoting, escaping or the difference between \n and \r and who thinks it's clever to use 0xFF as a delimiter, because in the Windows-1252 code page it looks like a weird rune and therefore "it won't be in the data".

I considered this but then went the other way - a \ before anything other than a \, newline or comma is treated as an error. This leaves room for adding features, e.g. \N to signify a SQL NULL.

Regarding quoting and escaping, there are two options that make sense to me - either use quoting, in which case quotes are self-escaped and that's that; or use escaping, in which case quotes aren't necessary at all.

Exactly. "Use a delimiter that's not in the data" is not real serialisation, it's fingers-crossed-hope-for-the-best stuff.

I have in the past does data extractions from systems which really can't serialise properly, where the only option is to concat all the fields with some "unlikely" string like @#~!$ as a separator, then pick it apart later. Ugh.

CSV is ever so elegant but it has one fatal flaw - quoting has "non-local" effects, i.e. an extra or missing quote at byte 1 can change the meaning of a comma at byte 1000000. This has (at least) two annoying consequences:

1. It's tricky to parallelise processing of CSV. 2. A small amount of data corruption can have a big impact on the readability of a file (one missing or extra quote can bugger the whole thing up).

So these days for serialisation of simple tabular data I prefer plain escaping, e.g. comma, newline and \ are all \-escaped. It's as easy to serialise and deserialise as CSV but without the above drawbacks.

As with GDPR, the spirit is admirable but the fundamental definitions have been hand-waved. So for the foreseeable future, the expensive lawyers you hire are going to answer the important questions with "well, we don't have much case law yet..."

Also, the definition of AI seems to exclude anything that doesn't "exhibit adaptiveness after deployment". So, a big neural network doing racist facial recognition crime prediction isn't AI as long as it can't learn on-the-fly? Is my naive HTTP request rate limiter "exhibiting adaptiveness" by keeping track of each customer's typical request rate in a float32?

Laws that regulate tech need to get into the weeds of exactly what is meant by the various terms up-front, even if that means loads of examples, clarification etc.

"No one wants a preboot password though" - really? Doesn't strike me as particularly inconvenient, especially given the relative rarity of actual bootups these days.

I've been using bog-standard FDE for as long as I can remember. One extra password entry per bootup for almost-perfect security seems like great value to me.

Just adding another comment to say how brilliant this film is. So atmospheric, such great music, such a grand presentation of the wild west and it's demise. It makes other westerns feel half-baked.

The map iteration order change helps to avoid breaking changes in future, by preventing reliance on any specific ordering, but when the change was made it was breaking for anything that was relying on the previous ordering behaviour.

IMO this is a worthwhile tradeoff. I use Go a lot and love the strong backwards compatibility, but I would happily accept a (slightly) higher rate of breaking changes if it meant greater freedom for the Go devs to improve performance, add features etc.

Based on the kind of hell users of other ecosystems seem willing to tolerate (cough Python cough), I believe I am not alone in this viewpoint.

NAL, but I think GDPR has exceptions for remote access, i.e. if a worker in India is viewing data held in the US, that is not necessarily formally considered a transfer from the US to India, even though the data clearly has made it to India if it's being displayed on a screen there.

4. Stability and backwards-compatibility. I have never seen a Go version upgrade break anything. Meanwhile I have colleagues who do a thousand-yard-stare if you so much as mention upgrading the version of Python we're using.

That doesn't strike me as a problem. Definitions are often highly abstract and counterintuitive, with much study required to understand at an intuitive level what motivates them. Rigour and intuition are often competing concerns, and I think definitions should favour the former. The definition of compactness in topology, or indeed just the definition of a topological space, are examples of this - at face value, they're bizarre. You have to muck around a fair bit to understand why they cut so brilliantly to the heart of the thing.

These statements always feel a bit circular to me. Sufficiently bad for what? Sufficiently bad to be indistinguishable from a cyberattack. So, "a software update bad enough to look like a cyberattack looks like a cyberattack". Well, yes.

Silly clickbait title, which the author even admits up front.

UDP and TCP have different behaviour and different tradeoffs, you have to understand them before choosing one for your use case. That's basically it. No need for "Never do X" gatekeeping.

I think statements of the form "X is immoral" are just secular versions of "X is against God's will", in the sense that they (not necessarily intentionally) use linguistic trickery to present a personal preference as a universal rule.

If I am against something I can say "I am against X"; making statements about oneself is generally safe and not an overstep. There's no obvious path from "I am against X" to "I will stop you doing X".

If I say "X is wrong", that sounds like an absolute, and is more easily (ab)used as a justification for stopping others doing it.

The exact same linguistic distinction applies to offence. "I am offended by X" is accurate and works fine in a society. "X is offensive" is a precursor to censorship.

What exactly are you saying nobody believes? Moral absolutists, moral relativists and moral nihilists all exist.

Interpreting the statement charitably, I think they mean "morality and ethics are arbitrary constructs of humankind, not the deep universal principles they are sometimes taken to be".

I tend to agree. "All nontrivial abstractions are leaky" reminds me of other slightly-too-cute rules, such as "full rewrites are a mistake" and "never parse JSON manually".

I wouldn't call TCP leaky because it can't deliver data across a broken network cable, for example. It's abstracting away certain unreliable features of the network, like out of order delivery of packets. It's not abstracting away the fact that networking requires a network.

Yep, one of the big wins for Go was preventing this by just not supporting complicated, "clever" code. It doesn't have a ternary expression, "i++" is not an rvalue, you cannot pack a huge amount of work into a single line, you can't overload operators or even functions and so on.

The opposite extreme is arguably C++, which I personally quite like (probably because I use it only for solo projects and don't try to collaborate with anyone), but I can't deny that it's an awkward, gnarly monster of a language. It'd be awful to see Go end up like that.