impure-aqua

Documentation on features your SQL dialect supports and key requirements for your query are very important for incentivizing it to generate the output you want.

As a recent example, I am working on a Rust app with integrated DuckDB, and asked it to implement a scoring algorithm query (after chatting with it to generate a Markdown file "RFC" describing how the algorithm works.) It started the implementation with an absolute minimal SQL query that pulled all metrics for a given time window.

I questioned this rather than accepting the change, and it said its plan was to implement the more complex aggregation logic in Rust because 1) it's easier to interpret Rust branching logic than SQL statements (true) and 2) because not all SQL dialects include EXP(), STDDEV(), VAR() support which would be necessary to compute the metrics.

The former point actually seems like quite a reasonable bias to me, personally I find it harder to review complex aggregations in SQL than mentally traversing the path of data through a bunch of branches. But if you are familiar with DuckDB you know that 1) it does support these features and 2) the OLAP efficiency of DuckDB makes it a better choice for doing these aggregations in a performant way than iterating through the results in Rust, so the initial generated output is suboptimal.

I informed it of DuckDB's support for these operations and pointed out the performance consideration and it gladly generated the (long and certainly harder to interpret) SQL query, so it is clearly quite capable, just needs some prodding to go in the right direction.

Certainly in my version of LM Studio (0.3.15) it has a branch button at the end of every message [0]

[0] https://i.imgur.com/xZ2Fkn7.png

Reversing the brain drain will certainly be difficult, but stemming the flow of graduates is perhaps a different story.

For Canadians considering working in the US, the recent politically-motivated detentions and deportations against green card holders - a group that has significantly stronger rights of abode than TN visa holders under USMCA - certainly factors into the calculus of whether a US job is worth it.

Further, the appeal of the American political environment is inversely correlated with the distance between one's personal characteristics and the feature vector of able-bodied, white, male, cisgender, and heterosexual. This is nothing new, but the importance of these characteristics, especially the last three or four, have dramatically increased with the rightward shift of the US over the past few years - and there are also more individuals in Canadian STEM (i.e. TN-eligible) degrees than ever before who don't have these characteristics.

Just to put some HN-relevant ballpark numbers to it, the University of Waterloo, a notorious Silicon Valley hiring pool, is reporting 38.6% women [0] in their 2024 engineering admissions. This was 21.2% in 2014 [1], the earliest year with statistics available.

I don't think it is much of a stretch to say the 2014 first years were more likely to aspire to intern at, say, Tesla or Meta than the 2024 first years (who will be entering their first co-op internships in a month's time). That is a function of both the American tech companies having cozied up to the political right, and an increased proportion of the students being both more directly affected [2] and morally repulsed [3] by this state of affairs.

Add an additional nationalism multiplier for Canadians being turned off by the annexation rhetoric coming out of the US, and I think we may see a change in this trend towards Canada retaining more of its local talent.

[0] https://uwaterloo.ca/engineering/about/faculty-engineering-s...

[1] https://web.archive.org/web/20140804070109/https://uwaterloo...

[2] https://www.theguardian.com/commentisfree/2022/jun/27/roe-wa...

[3] https://www.ft.com/content/29fd9b5c-2f35-41bf-9d4c-994db4e12...

I think that is much too hand-wavy regarding the performance differences.

Both Passmark and Geekbench are aggregates of a variety of tasks. If you dig into the individual tests that constitute this aggregate score, you will find different platforms perform better, or worse, on certain tests than others. I would wager that, for many applications, only a subset of these tasks are relevant to the performance of the application, yet such benchmark suites distil out all nuance into a single value.

Here is a personal anecdote. I have tried running CASTEP (built from source), a density functional theory calculator, on both an M1 Max MacBook Pro [0], and on a Ryzen 7840HS Lenovo laptop [1]. A cursory glance at those Geekbench results linked might make you expect that the performance is roughly equivalent, but the Ryzen outperforms the Mac by about 4x, a huge difference.

What happens if we try and dig into any particular benchmark to explain this? If you click on any particular benchmark in the Geekbench search lists, you will see they test things like "File Compression", "HTML5 Browser", "Clang". Which of these maps most closely to the sorts of instructions used in CASTEP? Your guess is as good as mine.

If anything, I would say Passmark is quite a bit less abstract about this. Looking at the Mac [2] and Ryzen [3] Passmark results, you can see the Ryzen outperforms the Mac by about 2x on "extended instructions", which appear to involve some matrix math, and also about 2x on "integer math". The Mac, meanwhile, appears to be extremely good at finding prime numbers, at over 3x the speed of the Ryzen. Presumably the Ryzen's balance of instruction performance is more useful for DFT calculations than the Mac's, which perhaps is weaker in areas that might matter for this application, but stronger in areas that might matter for others.

Of course, optimization is likely a component of this. How much effort is put into the OpenBLAS, MPI, etc, implementations on aarch64 darwin vs. x86-64 linux? This is a good question. It is, however, mostly irrelevant to the end consumer, who wishes to consume this software for use in their further research, rather than dig into high-performance computing library optimization.

[0] https://browser.geekbench.com/search?q=7840hs

[1] https://browser.geekbench.com/search?q=m1+max

[2] https://www.cpubenchmark.net/cpu.php?cpu=Apple+M1+Max+10+Cor...

[3] https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+7+PRO+784...

I am curious, have you used a code generation tool that is linked to your IDE?

If I ask it, "why is the structure I am feeding into the `do_cool_stuff()` function not working", it won't say "sorry, I can't figure that out because I don't know the library that implements `do_cool_stuff()`. It will:

1. read my code where I construct the data to feed into the function,

2. see that the function is imported,

3. then read the file containing the linked `do_cool_stuff()` function to understand exactly what data structure it expects.

Crucially, the linked function can be anything from my own personal super secret UI framework for building widgets, to the `lib` folder containing my proprietary driver code for my unreleased hardware. This is fundamentally the LLM figuring out how to do "development [] against unreleased APIs or hardware".

I don't really know what "private knowledge" you are referring to, given you talk about Apple developing widgets against their new unreleased framework. The code is not really "private"; certainly to you or I it might be, but Apple owns the code on both sides of the interface, no? What precludes their internal generative AI tooling from doing the equivalent of Ctrl-Clicking through to the linked `do_cool_stuff()`?

If anything, this is perhaps a benefit of genAI; for security reasons, I am sure your average programmer at Apple is expected to work solely against the internal documentation regarding how `do_cool_stuff()` works, and has no access to the actual implementation of it. Yet, an in-house inference server could certainly have access to both sides without the risk of code leakage you have directly sharing the other side of an interface with the consuming developers.

To me, in this context, "closed source" implies something the LLM does not have access to the implementation details for, e.g. source unavailable for the user invoking the LLM; in this case, I think my original analysis holds re: genAI further encouraging the development on top of open source technologies.

>What about closed source tooling? How do you expect an AI to ever help you with something it doesn't have a license to know about? Not everything in the world can be anonymously scraped into the yearly revision.

This is not a novel problem. Proprietary toolchains already suffer from decreased resources on public forums like Stack Overflow; AI did not create this knowledge segmentation, it is scraping this public information after all.

>It's pretty interesting and mildly shocking that everyone is just making the same 'who needs a new JS library' joke.

Surely the proprietary toolchain is itself the 'new JS library'?

Most developers I know don't enjoy working with esoteric commercial solutions that suffer from poor documentation, when there exists an open-source solution that is widely understood and freely documented.

I do not see why AI code generation further incentivizing the use of the open-source solution is a problem.