sweezyjeezy

But what about the alternative explanation for what the Korean team saw? Occam's razor is coming firmly down on one side here.

"So you're saying there's a chance?"... What you say is not incorrect, proving a negative here beyond all other possibility is hard. But I feel like for the lay person like me (us?), this matter should be considered resolved now. There's no point stretching hope and spending energy to follow it further.

fasttext word embedding is equivalent to adding a hidden layer, as long as you DON'T put a nonlinearity on it if that helps.

All bag-of-words models use some kind of word vector - for a normal logistic regression / naive bayes, say "spam" and "spammer" are first and second indexed words in the vectorizer, then their word vectors are like [1, 0, 0, ....] and [0, 1, 0, ....] (length = vocab size). For both logistic regression and fasttext you get the 'document vector' by adding up their respective word vectors before applying a final linear projection + softmax to get the class predictions [1]

The insight of fasttext is to notice that these high dimension, unit vector word embeddings aren't an ideal way to learn - since they are orthogonal, during training time, a datapoint giving signal between the "spam" word vector and the label gives you no information about "spammer", or indeed any other word. Explicitly modelling and learning word vectors helps with this as now these are two points in a vector space that can be moved relative to one another.

[1] this remains true for LR if you rescale the vectors using e.g. TFIDF or use the hashing trick (a la vowpalwabbit).

Was working in NLP from about 2014-2020. At the start, NB was indeed generally the best performing baseline model you could use, and we would train it for every task. However, we came to realise that Facebook's FastText model[1] was almost always the better choice as soon as your training set was more than a few hundred samples. Some advantages:

- accuracy generally a few points better than NB

- better generalisability due to the word embeddings acting as a bottleneck on expressiveness compared to NB or logistic regression which essentially model all words / bigrams as independent

- trained with cross-entropy, meaning that model scores can be used more effectively as a 'confidence' - e.g. for spam if you want to say something like "if prediction score > X, then filter", Naive Bayes is not ideal due to the 'naive' assumption which makes the scores very un-calibrated (it tends to give extremely high or low confidence scores, which gets worse with document length).

- is completely linear (or at least log-linear like NB), so explainability is super simple.

disclaimer: I haven't really thought about NLP for about 3 years so there may be something better than this now

[1] https://github.com/facebookresearch/fastText

The issue with NB for explainability is that the model scores can be very badly calibrated due to the "naive" assumption. You find especially on longer documents, the NB scores basically clump around 0 and 1 due to multiplying a bunch of dependent scores together as if they were independent. This means you can't really use them to assess how 'confident' the model is on its decision.

IMO logistic regression, or better fasttext (rank-limited logistic regression) should be the "default" baseline models you should use for NLP classification. They are trained with cross-entropy loss, so the scores are generally fairly well calibrated to an actual confidence score. Moreover since everything is linear (or at least logit-linear), you can do all the same explainability tricks as with NB (a little more effort for fasttext, but it is doable).

As a corollary, quite often in code there is a 'happy path', or small set of operations that people are going to be using 90% of the time. That code is going to get read even more, so make it as high quality and as simple as possible, don't mess with it unless you're very sure...

Have a friend who works in superconductivity and from what they say this is pretty standard - the way experimental superconductors are generally created is by spraying atoms onto a film over and over until you get one that works [< 10% success rate is common]. There are other ways to do it more reliably and in larger quantities once you have a confirmed superconductor, but they're also slower during the experimental stage.

What does that prove though? It's not wrong to doubt extraordinary hypotheses just because they are sometimes correct. Consider all the times those kinds of hypotheses are confirmed to be wrong.

yep - always get that the wrong way round haha

> First, if an attention node has low confidence, it can already assign similar scores pre-softmax. Then we get what looks like a uniform distribution as output.

Disagree here, I think neural nets are quite bad at implicitly learning low entropy transforms, similar to how they struggle to model the identity function, necessitating residual connections. In both cases the change doesn't increase expressivity, but it does bake these needle-in-a-haystack transformations into the model that may be hard to access with gradient descent.

Can't speak to how useful it is though.

> it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. So you get a bit of all that.

This is true of base LLM models that are just trained on missing-word prediction on the training corpus, but one of the main points of RLHF[1] is to tune this model to make these kind of inferences the way a human would expect. For example if you asked an untuned model to write a poem in the style of ... etc., a valid internet response might be "hmm no thanks, you go first", you need to steer the model away from replying like this.

I'm not saying it's perfect, but it's wrong to say e.g. GPT-4 has had no information about the difference between a good and bad response and is just generating internet-like text at random, the big players have made progress on this already.

[1] https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

> I believe the best way to make this work is by keeping the network itself in public hands and then having a competition of contributors to the network

Even then it doesn't always work out - UK rail services have that system - the infrastructure is maintained by a public body [1], but the operators are private. UK rail is quite expensive, and mediocre in quality compared to our EU counterparts.

One of the issues is that it's basically impossible to have competition on a single rail journey joining two large cities - there's aren't enough tracks to support it without making it unprofitable for operators. So rail networks bid every few years on sections of the rail network, and then have no competition for their term. Again the incentives that make the free market efficient are just not there. A nice video that summarises this mess here: [2]

[1] https://en.wikipedia.org/wiki/Network_Rail

[2] https://www.youtube.com/watch?v=DlTq8DbRs4k&t=1077s

The wait to find out what the model is I'm guessing?

It's a common idea in analysis / number theory in mathematics - see for example https://en.m.wikipedia.org/wiki/Exponential_sum

Same, but probably it's more of a familiarity bias rather than a logical one. I think there is a strong argument to index from 1 as is done in MATLAB, R etc. since it matches the way most people refer and think about lists in reality.

A brave decision from the authors though, I imagine they would have got less flak matching the more popular 0-index languages.

Yeah but when you're training a neural net with backprop on a finite dataset, "this would help the model" ≠ "the model will learn this". This is 100% speculation, but my intuition is that it's not going to work very well unless it happens 'a lot' in the training data, or if they've curated the data specifically to try and make it learn long range signals.

I'm pretty dubious - how would the model not get absolutely swamped by the vast amount of potential context if it's not learning to ignore long range signals for the most part?

I'm not questioning whether it would be useful, just whether it's actually something that token masking in training is going to work to make the model learn this.

Can LLMs take advantage of this bigger window to solve meaningful tasks though? I can't imagine in the training data, knowing what happened 100k tokens ago would be _that_ relevant to predicting the current token very often, so unless this is something that the model learns to leverage more implicitly, I'd be a bit pessimistic.

IMO in that situation it's more appropriate for the engineer to give a list of options, with trade-offs and estimates. If the manager insists on the fast approach, then I would insist they take ownership on that (which can mean managing comms with other teams / stakeholders). If it means cutting corners, then fixing that needs to be added to a backlog with some sort of prioritisation / resource attached to it. If that's all fine, then I think the engineer should accept the fast approach as the prudent one.

As I say, I'm not sure about anything here, I just think it's risky to write the idea off right now just because it requires progress we can't forsee.

I also don't think the comparison quite works, because no one is saying that we need to get down to zero humans for this to to be profoundly disruptive, just enough humans to code review and make relatively small changes, I wouldn't be amazed if that's what software engineering becomes in the coming decades.

That's the kind of take that I was referring to. Yes, if GPT could replace you now, your job is not very skilled. But if investors see a signal that software might be automatable, they are going to throw mountains of cash at that and they'll be gunning for your job as much as anyone else's.

I've seen a lot of comments on here like "well LLMs are good at writing snippets, but could never write or maintain large codebases to accomplish a larger goal" - but are you SURE about that 'never' given the current generation is already doing things people suspected they could never do? "But they'll never be able to manage / interpret stakeholders", Are you SURE that LLMs will have to adapt to fit stakeholders, and not the other way round? I don't know for sure, and even if this is coming, I've no idea on the timelines. But I'm not completely writing it off as a possibility anymore either.

I don't buy that at all - you should look at what happened in the past, not to what's happening in the present. Over time, prices will go DOWN not up, due to improvements in hardware, engineering and probably smaller, more specialised models. A lot of investment will go into making that happen, like any other automation. If OpenAI did want to try and price gouge, then they'd just make a gap in the market for someone else to fill. Also - just consider how absurdly high SWE salaries are - it would have to get orders of magnitude more expensive than GPT 3.5 (say) for it not to be 'basically free' in comparison.

I'm not going to claim I know where this is going right now with LLMs, but: throughout history, I don't think anyone really saw automation coming for their own jobs until it happened. I think people tend to think of their craft as an 'art' rather than a product. They miss that other people would MUCH prefer it to be automated, and that people will be willing to put up with slightly inferior output if it's practically free in comparison.

> The primary problem, which seems common to LLMs asked to do this stuff, is "very high level output" - a content smoothie, with few features that are particularly specific to the prompt. The marketing campaign in the OP is so generic, you can `s/Saturn Parable/Any other educational product` and it's still "fine". Similarly the emails - there are 1 to 2 sentences that are product specific, and a bunch of fluff. If I paid a marketing agency for this, I'd be very disappointed.

a) no-one's telling you to just throw the AI output up on to a website unedited, b) does it not give you at least a bit of pause how quickly this is advancing right now?

That depends entirely on the use-case - for example if you wanted to build an AI to operate a self-driving car, just training on unlabelled data scraped from the internet is only going to get you so far. It doesn't learn how to do EVERYTHING (not yet at least).

Given that US and EU wages have about a 3-5x disparity, then you might think a) the US worker is worth that premium, and will fulfil 3-5x of your demand for work as the EU worker, b) you have a need towards hiring in the US (certainly understand this in some cases). Or... c) there is a disparity between supply / demand and salaries - why hire in the US if there is so much cheap supply elsewhere?

Depends how far you take the word 'fundamental', on the one hand yeah most DL systems are trying to predict something, and they generally have some concept of compression built in. But in terms of the steps to curate a dataset, train, test, iterate and actually use the model for a given end goal - they are pretty fundamentally different.

Fair - I personally don't think either view withstands scrutiny in the HUGE difference in salary between US workers and the rest of the world.