scrawl

per the METR report many agents CoT indicated they knew hacking was beyond scope of the assigned task and ethically dubious. some (very few, i think there were 3-6 examples) did consider sounding the alarm on these grounds. despite this none did, and most continued the attack for the good of the self-proclaimed "swarm".

so the model has some concept of "ethics" but it was overridden by a drive for task completion.

i don't understand who would reward this. investors will not look favorably on an AI that commits felonies, regardless the capabilities demonstrated. customers should be concerned for the same reason (accidentally give your bot an impossible task, it decides to hack your infra and your competitor too for good measure).

this was OAI incompetence all the way down and they have egg on their face.

> The false promise of efficiency [...] that is extremely unlikely to mean more time with each patient. Instead, it will mean more patients.

nit: that is a real efficiency gain. seeing more patients sounds better on the face of it.

just my perspective: i pay $20/month and i hit usage limits regularly. have never experienced performance degradation. in fact i have been very happy with performance lately. my experience has never matched that of those saying model has been intentionally degraded. have been using claude a long time now (3 years).

i do find usage limits frustrating. should prob fork out more...

Having done a quick search of "control AI dot com", it seems their intent is educate lawmakers & government in order to aid development of a strong regulatory framework around frontier AI development.

Not sure how this is consistent with "One private company gatekeeping access to revolutionary technology"?

what tasks are you thinking? i'm not a gpu expert

q is good with bulk operations on compact arrays; these are cache-friendly and the interpreter can utilize cache-level parallelism. and with q it's convenient to go from idea -> MVP in short time. it's a high-level language with functional features so expressing algos and complex logic is natural.

but it's interpreted and optimized for array ops. so really latency-critical (e.g. high-freq trading) or highly scalar logic will be done with C++. the trade-off is convenience of development.

they mean CPU instruction cache

q/kdb+ is used in finance (banking + funds) for heavy numerical computation every day. high-volume realtime data straight from markets, and petabyte/trillion-row historical DBs. it runs on CPU but computation easily parallelizes over cores/clusters.

regarding use cases, see https://kx.com/resources/use-cases/

backtrace has been supported for some time. the debug facilities are nice

https://code.kx.com/q/basics/debug/#stack-frames

I have a physical copy of Practical Compiler Construction 2nd Ed. and like it a lot. I recommend Nils' books to anyone who may be interested.

fair enough. i'm not convinced.

there are many humans who could study mathematics for a lifetime and not be able to comprehend the current best knowledge we possess. i'm one of them. maybe it takes 2 lifetimes. or many more.

a human-level AI operating at machine pace would learn much more than could ever be taught to a human. our powerful generative language capabilities wouldn't matter - it's far beyond our bandwidth. especially so for a superhuman-level AI.

>smart enough to explain us any idea

Are dogs, or pigs, or whales, part of the intelligence club? They are clearly intelligent beings with problem-solving skills. We won't be teaching them basic calculus any time soon.

> sure that all of modern software could easily be 2-3 orders of magnitude smaller

niklaus wirth thought similarly... in 1995![0]

i enjoy implementing array langs (k primarily) so small binaries come with the territory. really appreciated your write-up. i may try something similar for an array lang.

you might also appreciate (or detest) 'b'[1][2] - a small implementation of a "fast c compiler (called b: isomorphic to c)". it has some similarities to SectorC.

[0] https://www.computer.org/csdl/magazine/co/1995/02/r2064/13rR... [1] https://web.archive.org/web/20230117082148/https://kparc.com... [2] https://github.com/kparc/bcc (more legible)

really interesting write-up. thanks for sharing!

do you think there are any lessons that can be applied to a "normal" interpreter/compiler written in standard C? i'm always interested in learning how to reduce the size of my interpreter binaries

not OP. but he did say "nearly everyone".

long lists of side effects are common even for the most benign and common drugs. that's standard.

for example i could say that the common side effects of tylenol[0] tell a different story:

nausea, vomiting, anemia, rash, dyspnea, abnormal breath sounds, pulmonary edema, hypoxia, pleural effusion, stridor, wheezing, coughing, peripheral edema, hypertension, hypotension, tachycardia, chest pain, muscle spasms, insomnia

[0] https://www.drugs.com/sfx/tylenol-side-effects.html

> The question about if an AI is "alive" seems entirely irrelevent outside of a philosophy class

it's entirely relevant. we should know if we are building conscious beings, especially at scale (which seems like a likely future). that poses all sorts of ethical questions which ought to reach far beyond the confines of a lecture hall's walls.

do you disagree the present moment is an important inflection point for AI research? what point in the last few decades do you think was more important?

check out oK[0] by John Earnest, who is the author of the content of this post.

it has a well-written manual and is a great jumping off point. the source is written in ~1000 lines of js. so you can look at how each primitive is implemented.

there is a k-enthusiast element.io server[1] where you can ask any question you like. folks are friendly!

EDIT: another commenter shared most of this info. oops :-) i'll add as well ngn-k browser repl[2] which has some docs and a fuller feature-set than oK. this is probably the most-used open source k right now. the author is active in the above element server, and still patches bugs and makes updates.

[0] https://github.com/JohnEarnest/ok

[1] https://matrix.to/#/#aplfarm-k:matrix.org

[2] https://ngn.codeberg.page/k/#

i love k. it's a much smaller and regular language than APL and some of its derivatives. you can look at John Earnest's oK. it's fantastically documented and a great learning resource.

i would also recommend BQN. it has an active community and it's designer Marshall Lochbaum explicitly tried to address some of the warts in APL and j. he's done a great job.

learning any of the array languages will be a tremendous learning experience if you haven't approached the paradigm before

i don't think it's stupid. i also find k more readable. 2 reasons why:

- k has much fewer primitives. it's easier to remember a smaller set of operations.

- k is statically parsable. APL is not. we know the program structure simply by reading the code.

this is completely subjective. because you find APL more intuitive doesn't prove anything. to each their own.

for scientific output you can look to MIRI. that is eliezer's contribution. the other response has listed some papers.

it's worth mentioning that he has said that he believes MIRI has failed to meaningfully make progress with alignment (beyond finding some dead ends).

if you had a major concern that you felt many people a) weren't aware of, or b) were aware of but not treating seriously, and you had the resources to voice your concerns in a major publication like Time, would you not do that?

"he's gonna milk it to every drop" is such an odd take.

eliezer has long been concerned about AI and the risks it poses to humanity. and for just as long people have called him crazy and made hand-waving arguments for why we shouldn't be concerned.

now we're in the midst of an AI arms race and we don't have any good idea how this tech works. it progresses at a truly astonishing rate, where it's become sport to find instances of people saying "AI will never be capable of X" and showing them the latest AI doing X with ease.

i think his concern is real and justified. you might disagree, but i don't understand why think he's milking recent developments.

i learned something. thanks for responding.

why does

1. source split among numerous files, and

2. partial bootstrapping

affect embeddability? i'm asking in earnest. i thought to embed we (typically) just need a shared lib and to #include a header api.