1,399 karma · joined November 12, 2010
@shawnup on X
Founder/CTO @ Weights & Biases
When refactoring is easier, it happens more often, which means code quality and readability are higher, which reduces the likelihood of every other kind of error.
This is akin to teaching an adult human about a specific domain. Better to just do that than make a whole new human from scratch!
I see this repeated a lot. I think you're saying bitcoin is no better than normal banking for moving value.
Do you really believe this? Yes, in the US I can immediately wire money somewhere as long as:
1) I do it within east coast business hours. Weekend? Sorry, wait til Monday 2) I have the correct information for the receiving institution, some of which require a small transfer up front to confirm the setup. 3) My bank actually supports wiring (Wealthfront, for example, only supports ACH transfers which take a few days) 4) I'm sending within the US, otherwise there are a bunch of other complications and requirements
I also need to trust that the bank actually has my money.
With cryptocurrency you can confirm that your stored value exists and is stilled owned by you, and transfer it to anywhere else within 30 minutes, in a provably secure and verifiable way.
Pachyderm makes different tradeoffs and we're excited to see their launch. Seriously congrats to you all! This is an invigorating space to say the least ;).
As life goes on novelty of new moments goes down.
You could say this explains or underlies the Janet model.
But one can seek out novelty to slow down time. In the extreme you could “reset”, move away from everything you know, and start from scratch.
I have no evidence for any of this other than my own perception. Does anyone know where to find more discussion on this topic?
Trump is incredibly vindictive. Remember a month ago when he hosted a way under-attended rally, because of a fake registration movement that spread on, yup, TikTok? Trump remembers.
But really what holds it back still is sluggishness. I'd take a few extra clicks in exchange for instantaneous responses to each click.
Think of two agents that run for awhile and end up in different parts of a virtual world, that require very different code to execute. This might be pretty difficult to parallelize on a GPU.
After collecting a bunch of experience, you update whatever function you’re optimizing. This could be something like a neural network that is the brain of your agents. The update step can happen on a GPU.
But then we need to go collect a lot more experience again, so we’re back in CPU land.
Experience collection often dominates overall compute time in reinforcement learning. Which means you want a lot of CPUs.
We've considered benchmarks that proceed in phases: a closed competitive phase for 3 months, then award a prize to the top result, and another prize for best user writeup. Follow that by a collaborative phase where it's more about sharing, teamwork etc. Rinse and repeat.
The question of attribution is really interesting. Who made the largest contribution to the development of a model? It could have actually happened in a hallway conversation, or something equally as untrackable. We'd love to hear other peoples' thoughts on this.
Stacey on our team has put a lot of thought into these topics and may have more to say here!
We've seen huge advances in human language modeling and translation due to the success of deep learning. Often new directions start with a really motivated team producing a new kind of dataset. Who better to do that for code as language than Github!
This started as a grassroots effort inside of Github, and went through many iterations. When it was presented to Github's CEO six months ago, he correctly pointed out that we needed to go back and include Github's most popular language (javascript). As the project went on many smart people chipped in, and we produced something that we think is truly useful.
Check out the paper here: https://arxiv.org/abs/1909.09436
We overcame plenty of challenges to pull this off. For example: how do you clean this data? how do you label it? We've got folks from Github, Microsoft Research and Weights & Biases here to answer any and all questions you might have. Can't wait to see where this goes!