Escapado

Thanks for that link, I genuinely laughed out loud while reading some of those points! Love the presentation and what a wonderful reality check I can’t agree with more.

Interestingly that’s less energy than the mass energy equivalent of one gram of matter or roughly 5 seconds worth of the worlds average energy consumption (according to wolfram alpha). Still an absolute insane amount of energy, as in about 5 million dollars at household electricity rates. Absolutely wild how much compute goes into this.

The devil is in the details and a very careful process and very careful wording of said law could mitigate that to some degree. I mean representative democracy could also be considered a system conflicting in principle with values underlying a direct democracy, which, to be clear, I think would be an even worse system but my point is that regardless of which rules of the game you make up, there is almost certainly upsides and downsides to them. E.g. You want a functioning economy? Well, listening to the corporations that employ a vast majority of the workforce might be a good idea in principle - until they start corrupting politicians and the lobbyism presents a detriment to society at large.

I think there is a lot of nuance when it comes to determining truth too. In germany for example the AFD is full of climate change deniers and anti-vaxxers. And even when there is a public discourse and plenty of information to discredit their lies, they keep growing in popularity - so even if truth presents itself, it is apparently of little consequence to their political power. And to turn that problem around: I think we are kind of in trouble when truth is primarily determined by democratic consensus and history has a lot of examples to show that. Furthermore for different types of statements truth has fairly different forms. If some politican made up that eating 4000kcal of raw sugar every day is the healthiest way to live and coincidentally his family runs a sugar plant then it's incredibly easy to discredit that based on scientific evidence from mechanistic studies to randomized human control trials. Now this is a case that currently is not handled in the political system and apparently impresses a significant amount of voters. At least from my perspective the lies used in european politics are in their quality not very far from the example given. Now what kind of punishment there should be for lying exactly and in which case I can not say. But leaving it up to the voters is showing to be increasingly fragile imho.

> "For the largest model size of 13B parameters, the MatMul-free LM uses only 4.19 GB of GPU memory and has a latency of 695.48 ms, whereas Transformer++ requires 48.50 GB of memory and exhibits a latency of 3183.10 ms"

That's a really, _really_ big difference in memory usage and since this scales sub-linear (300M param model uses 0.21GB, 13B model uses 4.19B) a 70B model would fit on an RTX 4090. I think currently people often run 34B Models with 4bit quants on that so I would like to see some larger models trained on more tokens with this approach.

Also their 2.7B Model took 173hours on 8 NVIDIA H100 GPUs and that also seems to roughly scale linearly with the parameter size, so a company with access to a small cluster of those DGX pods (say 8) could train such a model in about 30 days - though the 100B token training set might be lackluster for SotA but maybe someone else could chime in on that.

I moved from Hamburg, Germany to Tokyo in October of last year. I would agree with their observation and have a few points to add that I found remarkable (some of them are probably a little specific to Tokyo):

- In larger cities there are Konbinis _everywhere_. As in: You can barely walk a block without encountering one in the more busy areas. Sometimes there are three within a 30m Radius.

- Most fruits (except bananas or Kaki when in season) are pretty expensive.

- Package sizes in supermarkets are often smaller that in europe, so if you are used to buying 1 or 2kg bags of frozen veggies you'll have to go to Costco.

- People make due with much less space here than in europe. Tiny apartments everywhere.

- Most single detached family homes in residential areas of Tokyo usually have no garden.

- Even though a lot of places look like they have not changed since the 80s, they are suprisingly well maintained and clean.

- There are basically no public trashbins except at some train stations or in konbinis (which often makes me feel like I need to buy something else to justify throwing stuff away).

- Lots of people wear masks casually (not judging whether thats good or bad)

- Food is not only cheap but usually tastes amazing. I have not even once gone anywhere (from cheap chain to upscale sushi place) an was disappointed.

- You do get 日本語上手 (Nihongo Jouzu = your japanese is good) a lot even if you just say Hello in japanese.

- Trains here are usually really punctual, go frequently and during rush hour are unbearably crowded. It's kind of a miracle that people don't start fighting constantly and everyone is just somehow accepting the fact that they are forcefully pushed around on the train.

- The japanese people I met are are almost all incredibly nice, once you break the ice. They are so mindful of others feelings and needs and really good at reading people.

I wondered about the same thing but at the same time only about 5% of the training data is non-English and I would be surprised if the total amount of published text from all non English languages combined was also just 5% of all published text. So my intuition tells me there is still heaps of data but what might be tricky is to properly access and asses it’s quality.

Also the redpyjama v2 dataset has 30T tokens and is based on common crawl. Now I don’t know too much about common crawl but I doubt it has in it all published scientific books in all the different languages as these are often not freely crawlable. I remember when I studied physics there were at least 20 different 200-800 page long books on particle physics in German alone in our campus library. That must amount to 5million token by itself, from just one niche of physics. The Hamburg public library hosts about 5 million books and 90 million scientific articles mostly in English and German. If the average length of a scientific article is 6000 tokens and the average book about 100000 then that alone is already 1 trillion token. I bet there are significantly larger libraries and this is before even crawling the internet and looking at other languages or even generating training data.

German reporting in. Disclaimer: I am biased as I think immigration is great and I wish the process was easier and faster for those seeking to live in Germany. I think your observation is sadly very correct and this minority is very well measurable and concerningly large and growing in a lot of places. Just look at how many people vote for the AfD. Those are the ones having that concern. Luckily some places(e.g. Hamburg, Berlin) are on average more open to foreigners than others (e.g. Sachsen).

Super stoked to see this. The loss curves look like there is still some gains to be made from further training. Maybe I’m too stupid to read but I could not find anything on the amount of tokens this has been trained on except for some of the ablation runs.

I hate all that tracking and marketing bs as much as the next guy but if the marketing team is the main stakeholder and is responsible for the budget that won't work. I also might be a bit biased as a freelancer but every team I worked in so far had other freelancers on it and if we strongly recommend aginst a practice but the client insisted then we basically had the choice to either abandon the project (and therefore our current source of income) or simply do what they say. I would love to be on a position where refusing is an option that would not cost me my gig.

Which is still about twice of what I paid for my entire bachelors and masters education in germany and it included a ticket for the entire public transport system in my city for the entire 5 years and full access to _very_ comprehensive library (of both physical books and e-books so there was no need to buy even a single book during my studies).

I may speak from a perspective of ignorance here but I believe 6000$ a year is still prohibitively expensive for a lot of people, especially if they don't want to accrue debt.

I just want to feed an LLM hunter x hunter episodes and get out new ones.

But on a more serious note, I vividly remember when GANs were the next big thing when I was in university and the output quality and variability was laughable compared to what midjourney and the likes can produce today (my mind was still blown back then). So I would be in no way suprised if we got to a point in the next decade where we have "midjourney" for video generation. So I wholeheartedly agree.

I also think the computational problem is tackled from so many angles in the field of ML. You have nvidia releasing absolute beasts of GPUs, some promising start ups pushing for specialized hardware, a new paper on more optimized training methods every week, mamba bursting on the scene, higher quality data sets, merging of models, framework optimizations here and there. Just the other day I think I saw a post here about locally running larger LLMs. Stable Diffusion is already available for iPhones at acceptable qualities and speed (given the devices power).

What I wonder about the most though is whether we will get more robust orchestration of different models or multi modal models. It's one thing to have a model which given a text prompt generates a short video snippet. But what if I instruct my model(s) to come up with a new ad for a sports drink and they/it does research, consolidates relevant data about the target group, comes up with a proper script for an ad, creates the ad, figures out an evaluation strategy for the ad, applies it and eventually gives me back a "well thought out" video. And all I had to do was provide a little bit of an intro and then let the thing do its magic for an hour. I know we have lang chain and baby AGI but they are not as robust as they would need to be to displace a bunch of jobs just yet (but I assume they will soon enough).

Was at Onsen in Fukushima (Koriyama) when it happened. For two seconds I thought I was dizzy from the hot water until I realised what was happening. Luckily it was not much worse than the feeling you get when driving a car on the highway. I hope however everyone is safe and the damage is not going to be devastating. And I hope everyone can get to safety in time.

Having moved from Hamburg Germany to Shinjuku a few months ago I would like to add that I am still astonished by how clean the streets are here despite there not being trash bins except for the ones for cans and the ones inside of konbinis. Of course some people litter and if you go to Kabukicho or the along the street to Shin Okubo you’ll see more trash than let’s say in Nishi Shinjuku but given the density of these areas it’s still so surprisingly clean here. I also regularly see people (often elderly) pick up trash from the streets and put it into the trash bags just to clean the streets for everyone apparently. Maybe it is just the collectivist society showing here but I think this is rad!

I should have been more clear on what I mean: I am well aware that the job market is not optimizing for fairness but that is what people often judge situations on.

I do understand how it came to all of this and how the situation is justified but just posed that if we turned our back to completely free markets and imagined a structure based on "fairness" then things should probably be different.

Furthermore I disagree with the argument that nobody would want to work an intense job while being "underpaid". Plenty of labor is really intense and pays minimum wage or barely over.

We don’t adjust salary for what proportions people manage or are responsible for in a lot of other fields, at least not to the same extend. And especially if we would go by responsibility. I earn about 3 times as much as the average school teacher in my home town and about 4 times as much as the people driving the buses there while I sit in front of my screen and fix irrelevant css bugs or spin up the next semi-useful microservice for a cross financed project that nobody really cares about except and that doesn’t really help anybody. Their responsibility and utility is much greater than mine. If they fuck up it has much larger and potentially grave consequences. If I fuck up there will be a Jira ticket and a semi-interested PM who will assign it to me 3 weeks down the line. Furthermore this relation does not automatically imply fairness. And on the outside it also appears very asymmetric in personal risk. A megacorp CEO might tank the entire company and can often still walk away with millions. So if we wanted a “fairness” optimized solution then maybe it should take into account the public utility, difficulty of the job, amount of work required and personal risk. And maybe, (just maybe) we don’t need anyone being arbitrarily wealthy and having more assets than some developing countries.

Whenever I read something like this I sometimes wonder if there is a viable model for a consumer electronics company which works akin to a non-profit that has irrevocable principles it has to follow where something like this won't happen and where shareholder value, buy outs and angel investors can't force the company to become like that. A company that makes use of open standards and hardware, respects right to repair, user privacy, local/offline first, open sources its hardware and software, works with sustainable ecofriendly materials and processes whereever possible, has a transparent and fair salary structure and open inclusive company culture and won't engage in toxic or anti-competitive behaviour.

Now I realise that a lot of those points are not as clear cut and products would probably cost a bunch more but somehow I would prefer that. What do you guys think?

Very valid points. But I would argue that guiding the user through a set of basal questions to exclude a bunch of fields and pages would do wonders for the whole thing. I get by alright but most of my friends and family find it utterly complicated and have to consult a tax advisor because they don't even know which fields are even relevant to them and the explanations in them often leave them with more questions than they had when they started.

For anyone thinking this is satire: It's suprisingly close to the truth. We have Elster for electronically submitting taxes. Apparently the Elster Project started in 1996. And all it is, is a digital version of the manual tax forms. It's entirely stupid. When I file my taxes I still have to leave out 12 random pages (instead of Elster figuring out that I don't need to file them and simply not show them to me). And last time I called the tax office the woman on the phone told me she can't answer a specific question I had because she would need to get my printed out file for that from the cabinet down the hall so I will have to call again later. 27 years. This is where we're at.

I am very uneducated in this so please forgive my ignorance. These videos always look so insanely cool which I really like and from what I understand the scenes/radiance fields are always static and have lighting baked in. Is there a chance this might be turned into something that can be dynamically lit and support motion?

I was in japan for the first time earlier this year for 6 weeks. I travelled around with my japanese girlfriend and the first night we spent at a traditional inn in Hakone were we booked a private onsen later at night.

In short: It was amazing. It smelled a little funny at first but it felt incredibly relaxing, my skin felt really good afterwards and it was perfect to recover from the 16 hours of flight earlier. At home I also regularly take hot baths in the bathtub but it can't compare at all with the night sky view and the whole ritual around it. I'll be back in japan this october for an extended amount of time and I will absolutely go to onsen every chance I get.

One of the comments on this site pointed out performance issues with the esp32. I never worked with IoT devices or these boards but I would like to dabble since they are relatively inexpensive but it got me to look up and find out that the esp32 is manufactured in a 40nm process node. Are there like newer iterations with more performance at similar power budgets manufactured at more advanced nodes?

Interesting, thanks for sharing . Especially Ahref, Intercom and at some point the AWS bill make up the vast majority of the expenses. Chatwool on Railway for 5$/month would probably do the trick instead of intercom. At first glance 200-300$ per month for AWS once your credit runs out seems like _way_ too much. If I understand your product correctly then you serve customizable widgets for signup campaigns and persist the emails with some metadata for analytics in a database. I would like to be proven wrong but to me that sounds like a 29$/month supabase pro bill or a 20$/month neon.tech bill if you like managed services. Or given the numbers on the landing page this should(!) easily be doable with a hetzer server for about 30$/month. Depending on the exact usage Cloudflare R2 might also be significantly cheaper than AWS S3.

This whole saga is so exciting to watch. Maybe someone with a background in material science and/or engineering can help me out with a question: _If_ LK-99 turns out to be a room temperature superconductor how likely do you think it will be that we figure out how to to mass produce it in sufficient quality and quantity so that it can in fact be used for all the exciting applications we have for rtscs? I mean it's one thing to create a lab sample at 10% yield the size of a thumbnail but another thing entirely to create super long cables or massive amounts of motor parts or large sheets of it for MRTs. Is it a matter of time? Or could some applications remain elusive because of other potentially undesirable properties of the material (brittleness, heterogeneity, you name it)?

A few little findings:

- There is a rogue pr-4 on the free tier pricing card which makes the content sit left-aligned instead of in the center of the card.

- In dark mode the clerk login / signup link at the bottom is hard to read in that dark blue color

- Upon signup and first login the initially created conversation is not selected.

Other than that it looks like a good MVP. It looks like you are using Shadcn/UI which I am a big fan of and I'd like to give a shout out to them here.

+1 for hot mulligan! Would add - Tiny moving parts - Bouquet - I feel fine - Hail the sun - Everything Anthony Green has been involved in - Body thief

Ostraca sounds like a heavier mix of Saetia and the old stuff from Raein. I dig it!

I usually eat 3 meals a day or two meals and a snack and I brush my teeth about 20-30 minutes after that so I guess it mostly due to my wish to get my mouth feel fresh and prevent staining if I had something like kimchi. 2 minutes twice per day is probably sufficient most of the time but the amount of abrasion and gum damage will probably also heavily depend on the hardness of the brush, the specific toothpaste and and the of course how vigorously you brush. I asked my dentist about this last year and he said it's fine and my gum health and enamel seem to confirm that.

Super exciting stuff and it got me thinking: could we genetically engineer bacteria to get rid of morning breath? I would like to think (and so does my dentist) I have very good oral hygiene* and I get regular checkups where everything is fine but when I wake up the taste in my mouth and my breath are really unpleasant. The last time I had to be on antibiotics for a longer time I noticed it was severely lessened but came back just as strong after.

* My routine includes brushing my teeth 3x per day for 3 minutes with an ultrasonic toothbrush, I use an interdental brush at night, I use a tongue scraper in the morning and at night and I do an occasional chx mouthwash and semi-regular salt water rinses.

Just out of curiousity I let Wolfram Alpha divide the total land area by the global population and got roughly 0.0191sqkm per person. That’s about a 130m distance if people were positioned on a grid. Then I found that roughly 57% of people live in cities. Treating the size of those as a negligible term that leaves us with roughly 210m. Then I found that 57% of land is considered uninhabitable which cancels out the city factor, so we are back to roughly 130m. I think this is a neat number as it really puts being kilometers or even tens or hundreds of kilometres away from others into perspective. The human density distribution on earth is really all over the place.

I’ve never not been a developer in a „scrum“ team. And each org butchers the framework in one way or another. Dedicated scrum master? Meh, let the PO do it. Let’s estimate stories based on the time it takes to complete them and eventually measure our velocity based on days worked on a ticket (and pat each other on the back when a sprint without holidays finished). Let’s plan the next 12 months like this based on loose terms and draw some deadlines in the sand - which we will absolutely insist on later but worry not it’s still scrum because you guys do the scrum related meetings. Let’s have a status report during dailies because instead of fostering collaboration middle management would like to know whether they can update their weekly progress PowerPoint mid sprint. Let’s document every minute on every ticket, meeting or 2 paragraph chat message to help someone from QA because transparency. Let’s have a disjoint backend, frontend and testing subteam on the same scrum team and discourage cross functionality because of efficiency but still have them all sit in the same refinements twiddling thumbs. Let’s strip the PO of every ounce of decision power so they become an imperfect messenger of non-agile middle and upper management. Let’s have plannings and refinements together. Let’s do a mid-sprint planning for all the production bugs which we will not estimate. Let’s have 5 backlogs - product, bugs, technical debt, experimental and misc - and let’s ignore the technical debt one. Let’s not listen to developer feedback and rather tell them which tools to use for the job. Let‘s never document anything technical even if it is super obscure but incredibly important because we should rather work on features that increase revenue in the short term. Let‘s never pair - that’s twice the time booked on one ticket and we can’t have that. Let’s have a pre and post refinement. Let’s write todos at the end of a retrospective which nobody will ever be assigned to or complete.