Hacker News Reader

RockyMcNutson What Is Social Status?

people hate the word 'privilege' but privilege is literally what social status buys and what people spend their life chasing

RockyMcNutson Setris – Tetris with Sand Physics

Sandstorm

RockyMcNutson How to Finetune GPT-Like Large Language Models on a Custom Dataset

not strictly true I think

- you could add new units throughout and train those while freezing existing units (adapter-based fine-tuning)

- you could train all units and use e.g. low-rank adaptation to limit how much they can change

- you could do prefix tuning and train an input to add at every layer

see e.g. - https://lightning.ai/pages/community/article/understanding-l...

RockyMcNutson Show HN: I made an iOS HN app to navigate large threads without getting lost

LLMs might offer some creative ways around the trilemma of navigating threads chronologically, by topic/contextually, and by score.

RockyMcNutson Optimization Without Derivatives: Prima Fortran Version and Inclusion in SciPy

I think there are a few derivative-free global optimizers, curious what sets this one apart? see e.g. https://www.microprediction.com/blog/optimize . If the current scipy-native one is unmaintained and this one is superior it certainly makes sense to update.

RockyMcNutson Run Llama 13B with a 6GB graphics card

hmmh, if we have the reinforcement learning part of reinforcement learning with human feedback, isn't that a model that takes a question/answer pair and rates the quality of the answer? it's sort of grading itself, it's like a training loss but it still tells us something?

RockyMcNutson America Forgot About IBM Watson. Is ChatGPT Next?

ChatGPT lets normies use natural language to do complicated stuff that used to be only accessible to engineers ... so, no.

The winner might not be something called ChatGPT though.

The AutoGPT stuff is pretty ridiculous...

- ask GPT for the steps to perform a task

- then for each task ask GPT, hey you're good at coding, can you code this? or if not can you break it down into more detailed steps? and add those to the queue

as GPT gets smarter this type of agent workflow seems like it might work quite well for a lot of high-level tasks.

RockyMcNutson Mercedes EVs can go faster for $60 a month

not sure if the current airline industry is a great model for all other industries.

at some point unbundling becomes extreme lack of transparency and customer-hostility if not a downright scam.

there are such things as externalities and transaction costs. they are manufacturing externalities and transaction costs to their benefit.

some companies put out the initial product with high-quality parts and as soon as it gets good reviews and traction they sub out cheaper parts. you could say if they didn't do that they'd have to charge more, they are giving early adopters an incentive or something, people should research which parts they are getting and let the market sort it out. if you value your time and expect reliable information from reviewers it's your problem.

today everything is a SaaS and you are permanently in a hostage situation where your car could get remotely downgraded, they could stop supporting it or discontinue a feature.

a market is a set of legal and social conventions and this is a pretty bad one. if you like markets you should want them to not be hellish, which means reasonable rules and conventions.

RockyMcNutson Geoffrey Hinton leaves Google and warns of danger ahead

The real-time voice cloning seems ready for prime time, the video is getting there.

https://www.wsj.com/articles/i-cloned-myself-with-ai-she-foo...

(very not impressed by certain financial institutions who seem to be relying on the voice fingerprinting and SMS for 2-factor)

RockyMcNutson Geoffrey Hinton leaves Google and warns of danger ahead

The real problem is the bad actors - trolls, mental and financial strip miners, and geopolitical adversaries.

We are just not psychologically adapted or intellectually prepared or availing of a legal framework for the deluge of human-like manipulative, misleading, fraudulent generative fake reality that is about to be unleashed.

Free speech, psychopathic robots, adversaries who want to tear it all down, and gullible humans, are a very bad mix.

RockyMcNutson Ask HN: Most interesting tech you built for just yourself?

I made a python script / notebook to scrape a reddit thread on music, send all the comments to ChatGPT with a prompt like 'extract all the songs to CSV' and then upload to a Spotify playlist. probably high school level these days but it was amusing. https://github.com/druce/reddit_prettiest_songs

RockyMcNutson Finetuning Large Language Models

Which OpenAI fine-tuning API? I don't think fine-tuning is currently available for GPT-4

https://help.openai.com/en/articles/7127982-can-i-fine-tune-...

RockyMcNutson Why some researchers think I’m wrong about social media and mental illness

If Pascal's wager teaches us anything, it's that we want to start from a maximum-entropy prior and model the process generating the two hypotheses.

If a guy ends up living in a gold palace with Michelangelos on the ceiling if those are the only two hypotheses to consider, that might skew his analysis just a bit.

RockyMcNutson Amazon Titan

where's the paper on Titan LOL ... can't be good if no paper or metrics saying it's better then GPT-4 or even GPT-3.

but it's amazing that Google has all the papers and models and the TPU and BERT is pervasive in their products and they can't even stake a better claim to leadership than AWS does here, Amazon stock probably goes up, Google announces anything their stock goes down.

RockyMcNutson Ask HN: Open source LLM for commercial use?

not sure what 'mixing LLM models' entails but these are maybe some good starting points

- karpathy - https://www.youtube.com/watch?v=kCc8FmEb1nY

- https://towardsdatascience.com/beautifully-illustrated-nlp-m...

- https://dzone.com/articles/a-deep-dive-into-the-transformer-...

- https://peterbloem.nl/blog/transformers

- http://nlp.seas.harvard.edu/2018/04/03/attention.html

- https://lilianweng.github.io/posts/2023-01-27-the-transforme...

- https://blog.quickchat.ai/post/tokens-entropy-question/

- https://dugas.ch/artificial_curiosity/GPT_architecture.html

- https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

- https://d4mucfpksywv.cloudfront.net/better-language-models/l...

- https://arxiv.org/pdf/2005.14165.pdf

- https://arxiv.org/pdf/2303.08774.pdf

- https://arxiv.org/pdf/2303.17564.pdf

RockyMcNutson Hetzner Introduces ARM64 Cloud Servers

in other words less competition, more market power tax

RockyMcNutson EditAnything: Segment Anything + ControlNet + BLIP2 + Stable Diffusion

can someone ELI5 what it does and why it's magical?

RockyMcNutson Bob Lee, former CTO of Square, has died after being stabbed in San Francisco

point taken, there's definitely racism, anti-Asian violence, militias and Proud Boys and idiots in the North. The totality of circumstances and history and socioeconomics and state-sponsored nonsense and fundamentalist nonsense and gun culture just hits a little different in the South.

RockyMcNutson A trick to reaching flow: Leave your work broken

Never stop until you know what you are going to pick up with next.

https://www.fastcompany.com/3021905/hemingways-secret-to-mai...

RockyMcNutson Bob Lee, former CTO of Square, has died after being stabbed in San Francisco

I hear you, I don't live in South but I know people who do, and I see a lot of the mentality of, shoot first, ask questions later, and it goes exponential where someone who looks different is involved

https://www.mysanantonio.com/news/local/article/san-antonio-...

https://nypost.com/2022/02/01/dashcam-shows-florida-man-open...

RockyMcNutson Bob Lee, former CTO of Square, has died after being stabbed in San Francisco

fair enough ... or Miami ... you're still talking about states where there's lively debate about banning books, banning abortion, putting the 10 commandments and creationism in every classroom (and guns), making it effectively legal to shoot certain people or run them down with your car ... places with not-great schools and health care and culture, with cops and 'community orgs' dedicated to keeping certain people in their place. if you're a tech company and want to attract the best and brightest without regard for color or creed, women, lgbtq, people who think and look different, it's an issue, even if it's in a comparative island of receptivity.

RockyMcNutson Bob Lee, former CTO of Square, has died after being stabbed in San Francisco

every socioeconomic map of the US looks like this tho https://twitter.com/amazingmap/status/1642644670160175105

RockyMcNutson What if ChatGPT was trained on decades of financial news and data?

coming full circle, Peter Brown and Robert Mercer of Renaissance started off doing statistical language modeling at IBM and their 1992 paper in Computational Linguistics is cited in the BloombergGPT paper.

RockyMcNutson What if ChatGPT was trained on decades of financial news and data?

https://twitter.com/abacaj/status/1635837820270002178

the bloomberg paper does talk about how they sized the model based on their compute budget and research on time to train ... megatron has 1t parameters

i'm intrigued by some of the knowledge graph performance benchmarks and wonder about training it alongside an explicit knowledge graph, instead of building its own implicit knowledge graph, despite the 'bitter lesson' about just letting simple models and massive computation do their thing.

RockyMcNutson What if ChatGPT was trained on decades of financial news and data?

a fun fact is Bing AI already gives pretty good answers on the tasks Bloomberg cites as motivation, like who is the CEO of various companies, "write a bloomberg excel blp formula to return industry subgroup of ibm apple microsoft google"

another fun fact from the paper, it trained for 1.3m GPU hours and didn't get through the whole corpus, did 80% of 1 epoch (p. 16).

another fun fact, according to Twitter rumors GPT-5 is already training on 25,000 A100s w/8 GPUs each, so 4.8m GPU/hours per day.

it's a good paper with a lot more detail than OpenAI is releasing. also confirms OpenAI is quite a bit ahead. They compare with benchmarks they ran with Bloom and open source GPT-NeoX which they were able to run on same data, and reported benchmarks for GPT-3 since you can't run it yourself (well I guess you could sample the API), and BloombergGPT was competitive with GPT-3 which is a couple of years old now.

my takeaway was, this stuff is hard and OpenAI is crushing it.

I wonder if they will just use it for terminal features, or make the API available, and at what cost, and if they will license the model and weights for people who want to run it internally and maybe train incrementally on their own data. (not sure how feasible that is)

RockyMcNutson Pandas 2.0

TFA: Pandas 2.0 is also backed by Arrow as an option, yielding a large speed improvement, although not all the way to Polars

RockyMcNutson For the first time, the Fed is losing money

like an army of MIT PhDs didn't model obvious 1st-order effects

literally everyone who knew anything expected it, it's built into every model

what is supposed to happen if the Fed reports book losses or negative equity. hard to go bankrupt if you can literally print money. stupidest FUD and agitprop and unfortunately typical of the WSJ opinion section

RockyMcNutson For the first time, the Fed is losing money

following this statement to its logical implications, if they lower interest rates even further below a Taylor Rule, then that would be deflationary since less money would find its way back into the economy and fuel inflation?

by not printing money they are printing money or something..cough...tell me you don't understand monetary policy without telling me you don't understand monetary policy.

the money they lose when rates go up is of no more significance than the profits they make when they create high-powered reserves. everything on the Fed's balance sheet is outside the real economy. what matters to the real economy is the current nominal and real interest rates and the expectations for where they are going.

net income (as opposed to cash flow) is mostly a fiction in general but all the more so when it's the Fed.

these stories amount to "boo! OMG big numbers! Zurg say stop doing economics, they have played us for absolute fools" https://ifunny.co/picture/stop-doing-economics-eyears-of-yet...

RockyMcNutson Block's Response to Inaccurate Short Seller Report

https://en.wikipedia.org/wiki/Non-denial_denial

RockyMcNutson Block: Inflated user metrics and “frictionless” fraud facilitation

so, you can't get a physical credit card in the mail in the name of Donald Trump, like the picture in the report?

seems pretty bad if true. gotta wonder if Block was really that lax that anyone could do that, or if the authors found an exploit and gamed the system, and if the vulnerabilities have been fixed.