and yet we as a species are spending trillions of dollars in order to trick people that it is very very close to a person. What do you think they're going to do?
533 karma · joined July 17, 2024
and yet we as a species are spending trillions of dollars in order to trick people that it is very very close to a person. What do you think they're going to do?
There is no evidence that an LLM can reliably evaluate the semantic content of a sentence, even in cases where we all agree that the semantic content exists. The thread we are participating in demonstrates a particularly egregious failure, but there is no good reason to think that more subtle failures might not exist if we happen to patch this one. Even if they were reliable, you can't evaluate a system with itself - that is basic science.
This is actually very wrong. Consider for instance the fact that people who grade your tests in school are typically more talented, capable, trained than the people taking the test. This is true even when an answer key exists.
> Also, human labels are good but have problems of their own,
Granted, but...
> it isn’t like by using a “different intelligence architecture” we elide all the possible errors
nobody is claiming this. We elide the specific, obvious problem that using a system to test itself gives you no reliable information. You need a control.
Seriously, try it. Why don't LLMs get frustrated with you if you ask them the same question repeatedly? A human would. Why are LLMs so happy to give contradictory answers, as long as you are very careful not to highlight the contradictory facts? Why do earlier models behave worse on reasoning tasks than later ones? These are features nobody, anywhere understands. So why make the (imo phenomenally large) leap to "well, it's clearly just a brain"?
It is like someone inventing the aeroplane and someone looks at it and says "oh, it's flying, I guess it's a bird". It's not a bird!
e.g. from the WCAG examples you link to:
> An account registration page requires successful completion of a [Turing test](https://www.w3.org/TR/turingtest/) before the registration form can be accessed.
> I can finish it on Monday and have it ready for the Friday deadline and then spend my time how I wish OR I can report the task completed and then expect to have more tasks assigned
Are you serious? Do you think you just have free choice of whether to do more work or not? If you completed a task in 15 minutes and spent the rest of the week surfing do you think that would be allowed by your employer? Most employee contracts would forbid it.
This contradicts what you already said ("there are always more TODOs"). Which one should I reply to?
If your employment has fixed deadlines, and your employer does not react to efficiency increases by setting earlier deadlines, then you should expect the headline effect (efficiency increases mean your weekly hours decrease).
XML is a semi-structured format, which (apart from & < >) includes plain text as a more or less degenerate case. I don't think we have any other realistic format for marking up plain text with arbitrary semantics. You can have, for example, a recipe format with <ingredient> as part of its schema, and it's trivial to write an Xpath to pull out all the <ingredient>s (to put them in your shopping list, or whatever).
Obviously, XSLT is code. Nobody denies this really. One thing about code is that it's inherently structured. Only the craziest of literate programmers would try to embed executable code inside of text. But I don't think that's the biggest problem. Code is special in that special purpose programming languages always leak outside the domain they're designed for. If you try and write a little language that's really well-scoped to transforming XML, you are definitely going to want to call stuff outside it sooner or later.
Combined with the fact that there really isn't any value in ever parsing or processing a stylesheet, it seems like it was doomed never to pan out.
Where are you getting this from? I do not see it at all. The parent comment just says that it is an emergent compromise they don't think is a good one. That the code cannot be audited is also not necessarily a quality issue, either. It is just impossible to feasibly audit over 10 million lines of C. (this criticism applies equally to the kernel, although I doubt anyone would claim the kernel is less audited than systemd)
> Let's not pretend that a modern "init" does only system initialisation by calling shell scripts and then disappearing.
Nobody is pretending this. the comment you are replying to literally says "I don't think it represents the current state of the community".
This seems extensible to the degree that I want (i.e. semantically rich enough that you can conceivably hang any application from it). But I just can't bring myself to write in XML syntax, especially for maths.
> AI is unavoidable
> We will speak to models in voice mode, chat with companions for 2 hours a day, and point our phone cameras at our surroundings in video mode
This is surely meant to be an objective assessment, not a fluff piece.