Quanta Magazine consistently explains mathematics/physics for an advanced lay audience in ways that don't terribly oversimplify / still expose you to the true ideas. It's really nice! I don't know of any other sources like this.
1,507 karma · joined January 21, 2009
Re: favorite systems for migrations — not really; I've always just kind of not used one, or rolled my own. Desiderata:
* fully atomic (all goes through or none goes through)
* low-boilerplate
* can include execution of arbitrary application code — data-query-only only migrations feel kind of limiting.
* painless to use with multiple developers multiple of which might be writing migrations
(2) When a schema is provided, is it fully enforced? Is there a way to do migrations?
Migrations are the only remaining challenge I can think of that could screw up this tool long-term unless a good approach gets baked in early. (They're critically important + very often done poorly or not supported.) When you're dealing with a lot of data in a production app, definitely want some means of making schema changes in a safe way. Also important for devex when working on a project with multiple people — need a way to sync migrations across developers.
Stuff like scalability — not worried about that — this tool seems fundamentally possible to scale and your team is smart :) Migrations though... hope you focus on it early if you haven't yet!
Forms just inherently can have partially-finished/invalid states, and it feels wrong to try and kraal model objects into carrying intermediary/invalid data for them (and in some cases won’t work at all, eg if a single form field is parsed into structured data in the model)
Wait, what? Where did they get human fat from…?
Naively, it seems reasonable to believe that if you adjust all the weights of a neural net towards the behavior you want via SFT and RLHF, that it would compete with/mute/obscure undesired behavior like a back door. But it seems not to be so… Indeed the cute mask does not cover the entire shoggoth— it may still have tentacles (https://images.app.goo.gl/YW9g3BvwGqGwYTgd6)
AutoGPT itself is pretty weak, but it’s possible to write wrapper code that leads to stronger agency. Also, agents formed this way with GPT4 are way stronger than with GPT3.5… so expect this trend to continue.
This is often said, but it isn't so.
The task of predicting the next token in human speech really well requires immense intelligence — potentially far more intelligence than possessed by the original speaker! Imagine yourself engaging in the task of listening to someone who isn't that smart speak and then trying to figure out what they'll say next — in doing so, you might make all sorts of extrapolations about the person, their motivations, their manner, their dialect, etc — calling on all sorts of internal models that you've built up about people over time. This is what models are being trained to do when we train them on predicting tokens.
There are concrete examples of models inventing new ways of thinking that are not described in their training set. For example, when training a transformer from scratch to perform addition mod P (and having no training data other than examples of addition mod P), the transformer was able to discover the use of discrete fourier transforms and trigonometric identities [1]. As we can see, neural nets can build all sorts of internal mental models that no one explained to them beforehand. These internal mental models can then be elicited and used for other purposes by e.g. fine-tuning.
I think a good mental model for transformers/neural nets is that they're automatic scientists. They figure out ways of modeling things in order to predict the output from the input — which is what scientists do! As part of this, they can de-facto discover new theories, and come to rely on the theories that prove useful in their prediction task.
Also, not all tokens in the training set are from human speech, so models are being trained to model all manner of data-generating processes.
Do you think https://github.com/Significant-Gravitas/Auto-GPT et al will become more performant as models improve?
```
You are a super intelligent honest question-answering system.
Q: What's 2+2?
A: 4
Q: What color is the sky?
A:
```
Caro’s works have spoiled me — other biographies feel out-of-focus and surface-level and speculative.
(And all this despite the fact that Caro isn’t even really focusing directly on his subjects, because actually his books are about power and its dynamics.)
I wonder if we’ll see them used for large scale applications whose correctness is critical.
In the limit, all this effort brings about an artifact that could be thought of as a "magic" computer where:
- Everyone trusts that the computer operates to spec,
- Anyone (or any user interface) can type whatever they want into the terminal and press "enter",
- Anyone (or any other computer program) can read state out,
- The computer has immense amounts of storage and compute,
- Anyone who wants to change the computer's state in some way that cannot be achieved by using the terminal can freely make a copy of the computer with whatever changes they want, and encourage others to use their version instead.
What, one might wonder, would the use be? There are many, many potential uses — it almost boggles the mind — it's kind of like trying to imagine the uses of the internet when it was first invented.
To name one example: you could use this magic computer to operate a VR metaverse, instead of relying on a centralized Facebook one. Failing to do that would lock meaningful sections of our lives into a single vendor whose motivation is profit. Using the magic computer instead would mean that the metaverse would operate according to its predefined rules (which could leave plenty of space for human elections or other forms of collective decision-making), and that if things went awry anyone could try to set us back on track by making a modified copy of the machine and raising awareness about using it instead.
Just like regular computers, this magic computer could also have plenty of negative consequences. It depends on how people opt to use it. I think we'll probably have more luck adopting technology and trying to drive it in the right direction, though — the alternative is widespread Luddism, which might actually be a good idea, but I don't see a realistic path to achieving it.
A lot of folks who are very skeptical of web3 due to its dark sides are probably people who probably have great values and, if they adopted web3, could probably do a lot to improve its trajectory.
- Bitcoin offers a transferrable store of value which cannot be inflated by governments,
- Stablecoins, thanks to being cross-border, are often used in e.g. Argentina where the local currency is unstable and it's not legal to buy dollars,
- Proof of Humanity + universal basic income has provided extra income to Argentinian people (e.g. heard of someone who was able to purchase a ticket to visit their family for Christmas thanks to crypto UBI),
- Crypto has been used to send remittances to economically unstable places (Lebanon, Turkey, Venezuela)
- Gitcoin has provided public goods funding and advanced our conception of mechanism design,
- Helium has created a new 5G network that people can actually roam onto,
- NFTs have provided a new funding model for artists (who create public goods),
- Zcash and Monero have allowed for fully private digital transfers,
- Dark Forest (https://zkga.me/) has been an amusing game,
- Snapshot has helped create a delegative voting system that governs a $3B treasury,
These might not be problems that you face or believe are important, but all these are examples of intended problems being solved.
P.S. the tone of your comment made me a little sad :(
There simply are numerous hard problems to solve to make this all work at greater scale. In a healthy ecosystem like Ethereum's, there are frequent research discoveries (discovery of the concept of data availability, the application of BLS signature aggregation, proposer-builder separation, zkevm, data availability sampling, ...). The software engineering effort required to implement such research is colossal as well. Eventually we'll even see e.g. specialized hardware for efficiently producing or verifying zero-knowledge proofs.
It would be easy to look at the very early computers, which were perhaps not all that useful, and shrug — but that take wouldn't have extended well into the future as the technology scaled.
(a) that they build robotaxis ~5 years ahead of competitors
(b) they continue to sell ~50% more cars each year (they have a stated goal of 20m cars/yr by 2030)
How much is a robotaxi worth? A regular taxi can drive ~100k miles/yr @ ~$2/mile. Let's say that Tesla robotaxis undercut on price by a factor of two, thus making ~$100k/yr in revenue. Let's say 50k goes to the buyer, and 50k goes to Tesla. Five years of that would be worth $250,000. Let's divide by two to be a tad more conservative. If you're selling 20m cars a year, that's $125,000*20m = 5T of value in cars in a single year. If Tesla were doing that in a single year, their market cap would be in excess of $10T]. If you think Tesla has a 1 in 10 chance of pulling off the above, then you'd expect a fair market valuation for them today of 1T — which is roughly what the market puts them at.
Plus add Tesla Energy, which could be as big as the car business.
Plus add Tesla Bots (labor), which is probably the largest market in the world.
This priority fee is free money for the miner; the only reason they would not include your tx is because it would make their block slightly larger, which would make it propagate over the network slightly more slowly, and slightly increases the chance that the race is won by a different miner who discovered a block nearly simultaneously. So, the priority fee just needs to be big enough to make up for this tiny extra risk.