Amusing that Anthropic's approach to migrating context is asking their competitor's product to hand over the data it's stored about you.
Must be some of the lowest switching costs I've seen which doesn't bode well for OpenAI's consumer revenues...
270 karma · joined July 15, 2021
Must be some of the lowest switching costs I've seen which doesn't bode well for OpenAI's consumer revenues...
Really well made and enjoyed the audio explanation.
It's a shame that it includes the now-mandatory discussion of how this shipping is actually bad because of carbon emissions. Seems to me the widespread availability of cheaper goods has been a great thing for humanity on balance!
I disagree - this is how the internet can strengthen democracy.
Upvoting and commenting makes this post hit the top of HN and stay there. This makes it visible to many EU citizens who can reach out to their MEP's to ask them to vote against it. Seems a pretty effective strategy to me as someone living in a non-EU country.
Although agree that we should also be discussing the questions you raised.
> Instead of explaining the technical background first so listeners understand the solution to a problem, start with the problem. Then explain the context/technical background second
This isn't necessarily going to limit it though. It's possible there are clever approaches to leverage much more data. This could either be through AI-generated data, other modalities (e.g. video) or another approach altogether.
This is quite a good accessible post on both sides of this discussion: https://www.dwarkeshpatel.com/p/will-scaling-work
The dice roll animation is :chefkiss:
Taking selfless actions like these, that have major personal costs, require serious courage.
Particularly anyOf, allOf and oneOf (especially when nested) lead to really confusing nested specifications in OpenAPI. Really like how TypeSpec handles unions & intersections.
Playground is great for getting a feel for it fast too
This tokenizer is correct for the 7B open model and 8x7B MoE model. It'll probably be the closest to the ones their proprietary API-only models use
Token count for inputs allows comparison of different data formats (YAML, JSON, TS) and is a crude measure of prompt importance weighting. For outputs it is a relative measure of output speed between prompts (tok/s varies by time of day) and a crude measure of compute used in outputs (why “Think step-by-step” works). Token count also determines the cost of a prompt.
Since there’s no equivalent for other providers, I built one for Mistral & Anthropic. If it’s useful, I can add other providers too - let me know which you’d like.
Sadly it errored with the classic NextJS:
Application error: a client-side exception has occurred (see the browser console for more information).
The error in the console was: 601-ce9691b65ce5066e.js:4 Error: An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.
It's a little hamfisted in it's allegorizing and felt it sometimes spent a long time making simple points. Even still, it's much more memorable than this speech, so would recommend it if you have the time and inclination!
Aren't their core users developers? Why release a sheets extension?
Hey, this isn't _my_ guide. Shared it because I found it interesting. :)
Access to accurate emissions data is useful but comparative research often involves sifting through disparate sources. Combining the chatbot with the Carbon Interface API gives a quick and easy overview of CO2 emissions from a number of activities.
There are still a number of limitations. Like any LLM, it can make factual errors and mistakes when calling the API.
I wonder what other natural formations are man-made, but we don't realise it yet
Deterministic systems are clearly not creative. But randomness is a necessary, but not sufficient, requirement for creativity.
Adding random noise to an image makes it grainy (like TV static from back in the day), it doesn't lead to new masterpieces.
Base Llama 2's reliability on our prompt isn't good enough right now so working on fine-tuning it to make sure it outputs the right format - we'll release this at the same time as we make it easier to self-host
On trying us out - ping me an email at henry@superflows.ai
I remember reading that most optical illusions don't work on people raised in remote tribes in the Amazon, as their visual perception has been 'fine-tuned' for jungle contours, instead of the straight lines in the west.
Is it possible that we _learn_ how to perceive line drawings in our early years?
You'd either need access to the model weights or a fine-tuning API.
Then depending on which fine-tuning approach you want to use, the user data you need to collect will be different: RLHF requires multiple outputs to a single query vs instruction fine-tuning where you need great input-output pairs to train on. You could ask the user's feedback after running the LLM to pick out good training data.
Does it just spin up the right number of probes up so the experiments don't include this extra cost of spinning up these probes?
(apologies if this is a dumb question - I don't know enough about vector databases to know if this is a dumb question or not!)
What can you make predictions about if the matching hypothesis isn't true? What does this actually tell you about the world? Nothing.
Really interesting reading that Norvig's piece. I agree with almost all of it and think what we're doing at Delta Academy lines up with most of it!
Particularly:
> The key is deliberative practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again.
This is exactly what we do - providing weekly challenges that stretch your ability (that also happen to be fun), discussing the approaches taken by teams & giving expert feedback on code.
And his recipe for programming success:
> Get interested in programming, and do some because it is fun. Make sure that it keeps being enough fun so that you will be willing to put in your ten years/10,000 hours.
> Program. The best kind of learning is learning by doing.
> Talk with other programmers; read other programs. This is more important than any book or training course.
> Work on projects with other programmers.
Again - team projects which are fun sound right up his street.
We suggest ~10 hours per week, although except for the 30 min live competition each week, all of these hours can be done at times that suit you.
It's more a cohort-based online class that's punctuated by competitions (once per week). The competitions serve to motivate you to learn in a fun finale each week, rather than all being about winning. :)
The price tag includes 12 tutorials with exercises, 4 competitions (incl live discussion of solutions with the cohort) and expert code review from instructors on all the exercises.
Our competitions are designed to teach. Each is a progression in difficulty over the previous one. Also there are a set of tutorials preceding each competition which get you up to speed on what you'll learn in the competition.
Plus we're organised into a cohort so you're not competing with the whole world - rather you're competing with your peers who are also learning. You work in a pair on the competition. Then we discuss the solutions teams came up with & what an 'ideal' solution would look like (if one exists - sometimes it doesn't!).