peter_d_sherman

It shows to us just how skewed our (human) perception of light really is.

It also shows to us just how much is going on, in terms of combinations of wavelengths present, with various light sources, and just how different, again, in terms of permutations of constituent waves, any two of them are...

Anyway, a great page that should be included in any future Physics textbook!

1) Capitalism - Adam Smith, John Maynard Keynes (Keynesian Economics), etc., etc. in most places in the world...

2) Huge validated existing international market...

3) Multi-jurisdictional World... laws/statutory codes applicable to businesses in specific circumstances in one place may not be applicable to businesses in specific circumstances in another...

4) AI Tokens are a commodity; i.e., there is no chokepoint or monopoly controlled by one AI company in one jurisdiction, i.e., if one AI company makes rules unacceptable to a token consumer, that consumer can simply switch providers to another provider in another jurisdiction somewhere else in the world.

5) Tokens can be bought, sold, and resold at profit just like any other good or service.

6) Tokens can be bought from anywhere in the world and sold to anywhere in the world. Easily.

7) Tokens are a digital good, easy to scale, and do not require supply chains, lead times, labor, manufacturing, warehousing, shipping, going through geographic chokepoints, customs, etc., etc. -- all of the things that manufactured goods do.

8) Many people around the world want to make money or make more money... i.e., "economic incentive" (aka Capitalism's "profit motive")...

Well... add all of those together and what do you get?

You get buy/sell/trade forums/auctions/individuals/brokers/businesspeople -- around that market...

Just like you get those same things around every other market.

In this large multi-jurisdictional world, if one government makes all of that illegal in their country, then another government is going to be happily collecting all of the taxes from making all of that legal, in theirs!

If a given government makes trade illegal -- then they correspondingly lose the tax revenue...

Taxes and trade are intricately, intricately intertwined...

Could this business model be used for money laundering or other illegal activities?

Yes -- but any other business model could as well!

And, on the flip side, this business model could be accomplished legally/lawfully/morally/ethically -- just like any other business where there is an actual underlying value being exchanged.

Because, AI Tokens, if legally/lawfully/morally/ethically traded, do have underlying value...

In conclusion, at this point in time, I am neither for this business model nor against it...

But I think it'll be highly interesting to watch this space for the next couple of years, to see what happens, to see who does what, to see what plays out on the legal front, on the government front (foreign + domestic), on the media front, and on the technology front surrounding it...

Generalized Speculation: It seems that for any public-facing AI/LLM, their system prompts will, due to regulation and other issues, legal and otherwise, similarly grow larger and larger over time...

Now, I'm all for responsible, well-tailored guardrails on public AI's/LLM's, but consider the following:

Every time a system prompt is expanded, the LLM's context window is commensurately reduced.

Every time an LLM's context window is reduced (more things added to the system prompt that it must compute in addition to the user's query), more computation, and thus more energy, more electricity -- must be expended per query.

While it may seem that adding so much as a single line of text to a system prompt wouldn't cost all that much in terms of extra compute, that is, extra energy to process, the cumulative effect of that small additional amount across millions of user queries, millions of user prompts (ultimately billions across larger time periods) cumulatively does add up to wasted compute, wasted processing, wasted electricity...

Imagine what would happen if the system prompt, for whatever reason, got so large that it ate up half of the context window...

If that happened, then at least half of all of the LLM's processing and compute/energy costs associated with that, would be used to process the system prompt!

Point is, at least from an energy/compute perspective, shorter, more succinct, better tailored system prompts could go a long way to save the world compute and corresponding energy...

Anyway, great link, and a very interesting web page!

Observation: I never saw the link between Wave Equation Solvers, Finite Element Analysis (Eigenvalues, Eigenvectors, etc.) and Cymatics -- until I visited the above web page!

Related:

Cymatics: Chladni Plate - Sound, Vibration and Sand:

https://www.youtube.com/watch?v=tFAcYruShow

Cymatics:

https://en.wikipedia.org/wiki/Cymatics

Open Hardware takes another step forward!

Well done!

damping function

, and it needs to be perfectly tailor-made to work. “This is a challenging thing to construct,” Cohen said. But he knew that if he could do it in higher dimensions, he could unlock the entire proof."

Brilliant, absolutely brilliant!

An absolutely spot-on observation about the true nature of Silicon Valley, Venture Capital, Angel Investors, Tech Incubators and the whole surrounding early-investment (high risk, high financial reward if successful) ecosystem...

Every task, your coding agent starts blind. Before it changes anything, it re-explores the repo: grep a term, open a file, follow an import, back out, try again. It is rebuilding a picture of a codebase it mapped an hour ago and threw away.

That rediscovery burns most of a run's tool calls, tokens, and latency, and it is pure overhead"

The author of this article brings up a very interesting problem -- that, at least as far as using LLM's as coders/coding assistants go, eventually context runs out and context related to the underlying codebase does too. This in turn burns tokens and in turn, wastes energy resources.

Historically (well, in the past couple of years!), a bunch of solutions have been proposed to address this problem (i.e., take abstracts/subsets/maps of code, write them to different databases and persistent storage methods, bring them back in when the LLM requires it, etc., etc.)...

But there's no really good solution to this problem (although, arguably Graft goes a lot farther than past tools and should be commended for that!) because the problem seems to lie in separate parts, across several problem domains:

1) LLM context window size -- limited. Anything that future LLM's do to make context windows larger will help ameliorate this problem.

2) Lack of a good way to represent a codebase to an LLM for training other than text.

In other words, first we need some kind of way to map codebases into Tensors rather than text (i.e., a higher-level "map" of the code) then train future LLM's on those code-specific Tensors.

3) Arguably, programming languages themselves share some of the blame...

Programming languages have historically been written so that an arbitrary corpus of text represents and can be interpreted and/or compiled into a computer program.

That is, while tools for mapping codebases exist, tools for directly training LLM's on those specific created "code maps" as Tensors, do not, do not seem to, or at least I'm currently unaware of any!

(Anyway, just thinking aloud...)

Graft looks good, and looks like it has made some serious inroads to solving the problem...

Absolutely brilliant!

The vendor locked regions (on hardware that you already own!) that this could unlock (or help future security researchers to unlock, in the case of later model CPU's) has the potential to solve many auditability/transparency/defensive security/repair (cf. "Right to Repair") problems in the future.

Christopher Domas has earned the right to be called a 'Legend' -- again!

(For probably like what, the 3rd or 4th time now? :-))

Anyway, upvoted and favorited!

We wanted a way to restore service that didn’t involve rolling back to the last known-good backup (which would lose a lot of data) or repairing the known-corrupted database (which was potentially risky). To do this, we built a

transaction logging pipeline.

We streamed every SQL statement that modified the database to a separate log file. Because SQLite is a single-writer database with serializable transactions, our transaction history was completely linear and

deterministic.

(This wouldn’t be true in a multi-writer database like Postgres or MySQL.) Replaying those transactions against the latest known-good backup should restore the database to its most recent state, safely bypassing the corruption.

[...] This pipeline worked, but then it did something even better: it gave us a clue.

[...] To understand what was happening during these faulty checkpoints, the SQLite developers created a new debugging tool for the virtual filesystem layer.

[...] To help diagnose our problem, the SQLite developers created a wrapper around the virtual filesystem that

writes additional tracing information and logs

about changes to the database.

[...] After our next corruption incident, the additional logs from the new tmstmpvfs shim allowed the SQLite developers to find and fix the bug:

a rare data race

in the SQLite source code between a checkpoint and a write transaction."

Great article!

Software Engineering lessons (that repeat in this article!): So called "Heisenbugs" (bugs that make it past developer test harnesses and a company's Quality Assurance (QA) team) that show up post-deployment intermittently and can't be reproduced locally, occur because one or more of the following factors:

1) The lack of Determinism in a software process or processes.

2) The lack of appropriate logging.

3) The lack of the ability to replay a software process, step by exact step, state by exact state, as it has occurred in the field (occurs as an effect of #1 and/or #2).

4) Multi-threaded code; i.e., multiple threads giving rise to race conditions or other very specific intermittent combinatorial/permutational conditions caused by multiple threads and specific sections of code, which due to very large numbers of permutational timing possibilities, were not or could not be exactly tested for in development...

Anyway, great article! A must-read for any Sr. Software Engineer, or any developer that wrestles with hard-to-find-and-fix bugs in the field...

My intention was not to misunderstand you personally, only to express the general trend of what I see happening on HN, other message forums, and the mass media in general...

In fairness to you, you make some excellent points, and you long kept going into the discussion when other people would have simply given up and left.

So, to that I say, well done, and thank you for the excellent conversation!

I am not sure however, that a global public ban on the use of smart glasses, whether enacted by one (or more) countr(y/ies) law and/or by social conscience (of which Meta's smart glasses are but one in a sea of many that are/will be produced by many manufacturers in many countries, present and future) is the right answer...

The argument you present is not persuasive.

Your argument is the equivalent of saying that chainsaws (or any other inanimate tool, neither good, nor evil, merely neutral inanimate tools) are "bad" because occasionally they cut off limbs of people, both in cases of accident and by means of a few isolated cases of bad faith actors using them.

It happens from time to time, that's true...

But it's the exception, it's not the rule.

It's the same argument against self-driving cars...

Do they kill people?

Yes, from time to time they do!

But in such relatively low numbers (fractions of a fraction of a percent) relative to the amount of people using them...

It's the exception, it's not the rule.

The same argument could be applied to elevators...

Do they kill people from time to time?

Sure they do!

But again, it's the exception, it's not the rule.

Your argument is an exception, it's a corner-case, it is not the norm, and it is not how the majority of most lawful, responsible individuals in society act.

It is not how you would act, and it is not how I would act.

It is not how most responsible people would act.

In short, it is not persuasive.

Now, I don't mean to offend you in any way by saying that. Certainly you are free to attempt to persuade as many people as you wish of your viewpoint. You could seek political office and with enough political support, perhaps you could get your viewpoint enacted into Law.

And certainly I and millions of others would respect such Laws if you did get them enacted, even though I currently do not own a pair of smartglasses, produced by Meta or any other company, and I certainly would not take them into a Men's locker room with me if I did own a pair!

But you see, as far as your central argument goes, I can't (and it may be just me!) distinguish it from any other argument of the form

Neutral inanimate object or tool X is "bad" (we need laws/bans/broad social disapproval, etc.), because it enables bad faith actors Y (who are exceptions, corner-cases, outliers, not the norm of society) to engage in illegal/immoral/anti-social/unethical/unlawful action Z.

You see I've heard that same argument over and over and over again.

Your argument is an example, a specific case of that pattern.

And that pattern, because I have heard it over and over (and over!) again, is simply not persuasive.

This being said, you're entitled to your viewpoint, and you're entitled to seek any political or social action you feel may help that viewpoint!

I will not stand in your way!

But, at least at the present moment, I individually am not persuaded...

For the longest time, I conceptualized LLM's as Text Input -> Text Output transformers, then later as Text Input -> Video Output transformers. Later still I conceptualized them (if they were general purpose) as Any Format Input -> Any Format Output transformers...

The idea of a smaller parameter model runable on smaller/slower/less complex hardware (computers with no GPU, slower CPU's, less memory, aka "Edge Devices") trained for Text Input -> JSON Output (used for tool calls, etc.) I could honestly not conceptualize before seeing the demo on the web page...

But now that I've seen it and conceptualized it -- I'd have to say: "Yes, there's definitely a huge niche, a huge market for this, directly between the non-LLM driven tools and software and SaaS's of yesteryear, and the latest, cutting edge Frontier AI models of today!"

So, I like Needle a lot!

I like Needle a lot, and I love the idea of any tiny resource-thrifty LLM that can run on older hardware, that outputs only JSON!

I can see a huge market for it!

Explain to me how recording device A "when caught" (i.e., the glasses) differ from recording device B (say, a smartphone) "when caught", with respect to "plausible deniability"?

In both cases, do you not have the person responsible for making the recording (because they've been "caught" as you say), regardless of the device used to make it?

If you do, then that person has, as you say, no plausible deniability at that point, do they not?

And if you don't, then isn't the device type used to make the recording in question, irrelevant?

Because you haven't "caught" them?

Also, why do you care so much about plausible deniability as relates to a specific type of device?

Are we talking about legal accountability, or are we talking about something else entirely?

What do the glasses enable that any other video camera does not?

I suppose that someone could argue that the glasses enable covert, non-consensual recording, and that this behavior might be abused by certain individuals, and that would be a valid argument, but I don't think it could be applied to only Meta's video glasses (i.e., Meta is not the culprit) -- it would have to be applied universally to all companies present and future (including all of the ones in China) who manufacture smart glasses...

If I may suggest...

If the central issue with these smart glasses (made by whatever manufacturer, present or future!) is that they enable covert non-consensual recording, then how would it be if the manufacturers put a big, red light on the front of the glasses, that would automatically turn on when the glasses are recording...

If these glasses were created with big red recording lights that turned on once the device started recording, then that might serve as adequate notification that the counterparty is being recorded, would it not? (You'd rather someone, if they were going to use video recording glasses have them with a big bright red light, then no such light, right?)

So, to recap, I don't understand.

You're looking for a way to prosecute unethical individuals who use covert (or at least, somewhat more covert than a cell phone, let's say, at least until large swaths of society accept the existence of video recording glasses like we already do with smartphones) video recording technologies, such as those found in smartglasses?

Or is there some other issue or issues?

I don't get it; help me out!

Any article, even the really good ones on HN, while they get positive comments, for whatever reason, always get a lot of negative ones, too...

That is, the negative comments are absolutely unavoidable, even for people accomplishing great things!

I personally think that what you've done is brilliant, absolutely brilliant!

I can't wait to see more in this space...

Brilliant, absolutely brilliant!

>"Here is something even more interesting. As you approach α = -1, the time domain approaches a shape of t-1, and the frequency domain approaches a flat magnitude with a zero phase. However, a flat magnitude and zero phase corresponds to a delta function, δ(t), in the time domain."

Related:

https://en.wikipedia.org/wiki/Dirac_delta_function

>"Indeed, Heaviside introduced the δ-function in his work on electromagnetism and electrical engineering.[14] In a 1963 interview, Dirac stated, "All electrical engineers are familiar with the idea of a pulse, and the δ-function is just a way of expressing a pulse mathematically."[15]"

Exactly!

People could be "covertly recorded at the gym" (as the article states!) by other pre-existing recording devices (any other kind of video camera, mounted in the gym or on a gym member's smartphone or other electronic device) -- not just Meta's smart glasses!

In other words, the central issue of the article seems to stem more from consent (more specifically, the lack of consent!) than it does from a specific type of video recording device...

The users secretly videotape and share what happens when they try to pick up women. Now, a growing number of people are worried they could be covertly recorded at the gym..."

Observation: The same technology could be used to record business deals, job interviews and other negotiations with consent of the counterparty... In fact, both parties in such transactions could record this way (legally and ethically!) if each has consent of the other...

In his book, The Most Important Thing, Howard Marks explains the concept of second-order thinking, which he calls second-level thinking:

"First-level thinking is simplistic and superficial, and just about everyone can do it (a bad sign for anything involving an attempt at superiority). All the first-level thinker needs is an opinion about the future, as in “The outlook for the company is favorable, meaning the stock will go up.” Second-level thinking is deep, complex and convoluted.*"

Related:

https://jamesclear.com/book-summaries/the-most-important-thi...

https://en.wikipedia.org/wiki/Howard_Marks_(investor)

My First Million Podcast - Howard Marks:

https://www.youtube.com/watch?v=Kp__G0eUyIc

Although being familiar with Coreboot and Libreboot, I never knew about about Canoeboot until I found the web page a few minutes ago...

Looks like it could be worth exploring!

Not so much for the hacking (White Hat, Black hat, other-color-hat) aspects (although they're certainly there too), but for the

visualization of higher-dimensional mathematics aspect...

In other words, have a look at the following URL's, then come back here:

https://gods.art/articles/equation_shadows.html

https://kettenreihen.wordpress.com/

See, there's Math (which typically generates graphs, graphics, other visuals), and then there's Higher-dimensional Math (you could almost call it 'Meta-math') -- which generates graphs about graphs, graphics about graphics, visuals about previous visuals...

That is, take a math equation that generates a graph. OK, so a simple example is that we could take the derivative... That generates a second graph which gives us information about the first graph... a "graph about a graph", so to speak, a "signal about a signal", information about the original information...

Point is, Cantor Dust looks like another great mathematical tool in any Mathematician's and/or Computer Scientist's and/or Engineer's visualization/understanding toolbox!

Oh sure, bad faith actors could use it for hacking (bad faith actors could use aspects of Isaac Newton's Calculus for hacking in various contexts, heck, any mathematical tool could be exploited in specific contexts!) -- but those people I'm sure, would not have an appreciation of the sheer mathematical beauty of such things! (Why use it to hack, when you can admire the mathematical beauty?)

Also, I should point out that humanity as a whole is far from discovering every single possible method, every single equation, every single way to visualize higher dimensional mathematics...

In other words, Cantor Dust is one such method... there will no doubt be many more in the future (I'd love to see fractal visualizations of higher dimensions!), and of course, we still have yet to understand all of the "old" previously discovered math in terms of all of the possible ways it can be used to visualize higher dimensions...

Anyway, great link!

The problem, to recap the title, is that "99% of My Website Traffic Is Bots".

How to fight all of those bots, all of those web-scrapers...

If the sole underlying issue is bandwidth for one's website (someone doesn't have enough of it and/or they pay too much for it), then one possible solution to this might be for a single tech company to create an online cache of web pages, accessible to all bots as an alternate route to fetch the desired web pages.

If a tech company (or consortium thereof) decided to do this, they could certainly charge money to each AI / bot utilizing the service...

Which means that it might be an investable idea as a for-profit service...

Now, if the underlying issue is the privacy of individual web pages (or relative privacy, as the case may be!) from the public internet, then perhaps the solution is to simply put those behind user logins and/or a gauntlet of tests (which could also be tied to login!) designed to exclude AI/bot web traffic from human web traffic.

Consider what some BBS'es and the online services of yesteryear used to do (Compuserve, The Source, Prodigy, QuantumLink, AOL, etc.)... basically they'd get a user's mailing address, and then physically snail mail them their login and password to their physical home address...

Oh sure, user registration wasn't as fast back then as it is today... it might take several days to receive your username and password via postal mail -- but as a system operator you were 99.999999999% guaranteed that when you saw such a login on your BBS, that it was an actual live human user...

There may be a market for a service like that, too...

That is, validate that actual users are actual users, and give them some kind of credential that can be checked by an actual web site... find a way to do this at scale...

Anyway, just thinking aloud... a very interesting article, and a very interesting problem to have!

That is a downside to be sure, but from a pure business perspective, "that's not a bug -- it's a feature!"... from a pure business perspective it's the ability to sell and resell, to purchasing and re-purchasing customers, way into the future -- that is, recurring revenue from the perspective of the company being able to make those future recurring sales...

In the above case, that company is AMD...

(Also, on a related note, it would be interesting to see what open source / open hardware work has currently been done to offload LLM weights (and/or anything else that could be offloaded to silicon ASIC's) to FPGA's...)

matrix multiplication can be effectively replaced with additions

, lowering the total arithmetic workload needed to infer through a model."

Interesting! I never knew that before... I will have to do some more research on this, but yes, it definitely sounds possible!

(Also (and this is just a wild guess/hunch here!)

in theory, matrix multiplication could be replaced with pre-computed cached lookups from a lookup table in memory

if, if and only if the resultant pre-computed lookup table of result matrices could fit into memory available!

In other words, think of a Matrix not as a 2D array, but as a 1D string. Think of the Matrix multiply operation as first appending the 1D string of the first matrix to the 1D string of the second matrix, then using that string as a Key to look-up a precomputed result in Memory...

Of course, the limitation to this approach is the availability of memory relative to the permutations of input values... It could work -- but only for smaller matrices... which leads to another strange idea... in that set of pre-computed matrix permutations, there might be duplicates... figure out a way to put duplicates in memory only once (could use pointers or some other scheme to do this) and you could theoretically store more matrices in the same memory and/or slightly larger matrices!

Anyway, just rambling out loud! :-)

Remember, "almost all programming can be viewed as an exercise in caching" to quote the legendary programmer Terje Mathisen! :-) )

Anyway, great article!

Wikipedia:

(https://en.wikipedia.org/wiki/Bending_Spoons)

>"Based in Milan, the company acquires products with existing product‑market fit[4][5] (such as AOL, Eventbrite, Evernote, komoot, Meetup, Tractive, Vimeo, and WeTransfer) and manages them for long-term ownership, often increasing revenue and lowering expenses (including by reducing headcount).[6][7] The company employs full-stack developers to rewrite acquired codebases using artificial intelligence, proprietary platform tools, and open-source software (often Python, FastAPI, and TypeScript).[8][9]"

Ah!

So they're kind of the equivalent of a

special kind of Private Equity firm

that specifically purchases/manages/improves: Commerical websites, Apps, SaaS'es and other Digital/Software products that already have many users/customers (aka "Product-Market Fit" in business parlance...)

Although, arguably, Microsoft, while never called a "Private Equity" firm, sort of did all of that first, with other tech companies, in earlier decades!

Well, wait a second now...

If memory serves me, then actually Computer Associates did all of that (for software mostly for the Mainframe / Minicomputer markets and related!) even earlier than Microsoft, probably about a decade or so earlier than Microsoft!

Anyway, interesting article!

Interesting!

Induction Vs. Deduction Vs. Abduction!

(You know, if you like Logic, Philosophy, Law, or... just plain different ways to think/reason about something! :-) )

For any given Lisp-like language (in the above case, Clojure), implement code which allows the programmer to create statements in that language which look very similar to SQL, that is, as code, read very similar to SQL (rather than directly send SQL commands as strings to the SQL database back-end), which allows further processing in that language, if desired (one such ability might be the ability to check for, find & remove SQL injection attacks automatically, but there could be an infinite amount of others...).

AFAIK, only Lisp-like (list based, AST available in memory at all times) languages, of which Clojure is one, are capable of accomplishing this...

Basically re-define a custom subset of the language to support an external language, in this case, SQL, and be able to do further automatic processing on that, because ultimately such a definition is only a list (or list of lists), and other code in a Lisp-like language can automatically process any list or list of lists...

Way more flexibility than traditional SQL database components, database interface libraries, ORM's, etc. in non-Lisp-like languages!

Anyway, great idea!

I don't know, how about:

make peace with them?

?

(I'm guessing that that idea is either too creative and/or too unconventional to be considered! :-) (And yes, I am bracing for the disingenuous one or two line bot comments and downvotes!))