uniqueuid

BERTopic is great, but some people forget that even magic, er, UMAP+HDBScan on embeddings cannot solve some problems:

- statistical tools (including LDA and variants) define topics to be coherent latent clusters of words/embeddings. These correspond to a mixture of real-world concepts, including events, topics, issues etc. So when you apply BERTopic, you often get clusters that represent very different things on a conceptual level

- the end-to-end pipeline is very nice, especially when adding things like cluster labeling from LLMs on top. But we should not forget that this stacks many steps with implicit errors on top of each other. It is not easy to get a transparent and robust story why one cluster solution is better than any other.

- one of the implicit choices is picking UMAP, which will tend to find very coherent clusters but "throw out" many (up to ~50%) cases into an outlier cluster (-1). Sometimes that's not what we want, and then tuning is needed (e.g. use kmeans instead).

- random footnote: cuML for really fast BERTopic is great, but seems to produce inferior solutions. Better test that before putting it into production.

With all that said, I love that now we can use this tool and debate its merits on this level, rather than everyone implementing their own homegrown and probably bug-rich version of it.

How exactly would you use technology to raise that limit?

From my experience, direct 1:1 attention is the single most important thing for children. I would (and am) actively paying for childcare not to use technology to increase that attention.

I would say that's not the key takeaway; it's about the relationship between income and children within a country.

From the conclusion:

"Past fertility research largely focused on understanding fertility decline over time and the negative cross-country relationship between income and fertility. The most important mechanism in explaining these patterns was the quantity-quality tradeoff. However, much has changed over the last few decades. In particular, fertility is no longer negatively related to income across high-income countries. Instead, family policy, cooperative fathers, favorable social norms, and flexible labor markets have become key determinants of fertility choice. Thus, a new era of fertility research has begun."

Well it's precisely what GPT struggles with! Poetic meaning is probably the deepest level on which we humans relate to text, and it's incredibly difficult to produce.

Some value is not rational.

"And the words slide into the slots ordained by syntax, and glitter as with atmospheric dust with those impurities which we call meaning." (Burgess)

Technically speaking no, because regulation is performed by national data protection authorities whereas the people you're speaking of are some members of the European parliament.

I agree especially with the second argument.

But most people who toy with LLMs will probably never make money out of them. Even those who do will often spend a lot of time getting their bearings during which the GPU sits idle. Then you begin to ramp up your use but by the time, there's a new generation of GPUs out.

That's why my recommendation is to start with something lightweight.

It's also much less frustrating to start working for a few hours on a rented A100 rather than running into OOMs all the time while fine-tuning batch sizes and waiting for the nth highly quantized model to download.

To be honest, the best idea for most people is probably just any GPU that you can easily afford and then rent a big iron GPU.

There is almost no way you will make back the $5k for a 40GB+ ram card, so just save yourself all the hassle and go for something that ticks all the rest of your boxes.

Non-CUDA cards may be ok if you have very simple requirements, but I'd expect many hours of debugging if you want something that's not ready to go out of the box.

Vinegar can be a stronger acid than citric acid and has the potential to damage rubber sealings. That's why many manufacturers specifically advise against it.

Great to see this here.

The author Balázs Bodó is an all around great guy with very refreshing research methods and an immensely broad portfolio of things he works on [1].

[1] https://www.uva.nl/en/profile/b/o/b.bodo/b.bodo.html#Publica...

My intuition was the same, but they are doing this the way you should: Pre-registered, sufficient N for small effect sizes, comparative across countries. I'm not so sure it does not replicate.

To be honest, I don't think both are similar at all.

Parallelizing across machines involves networks, and well, that's why we have jepsen, and byzantine failures, and eventual consistency, and net splits, and leadership election, and discovery - so in short a stack of hard problems that in and of itself is usually much larger than what you're trying to solve with multiprocessing.

It's in the readme of the github project.

> In short, the main reasons why MPIRE is faster are:

    When fork is available we can make use of copy-on-write shared objects, which reduces the need to copy objects that need to be shared over child processes

    Workers can hold state over multiple tasks. Therefore you can choose to load a big file or send resources over only once per worker

    Automatic task chunking

Ray is parallelism across machines, this is only across cores.

I agree. The main problems aren't syntax, they are architectural: Catching and retrying individual failures in a pool.map, anticipating OOM with heavy tasks, understanding process lifecycle and the underlying pickle/ipc.

All these are much more reliably solved with horizontal scaling.

[edit] by the way, a very useful minimal sugar on top of multiprocessing for one-off tasks is tqdm's process_map, which automatically shows a progress bar https://tqdm.github.io/docs/contrib.concurrent/

No, weight the scientific data by its credibility and importance, as one does with all scientific data.

It seems that the author of this post has a quite weird position - that climate science should be unpolitical [1].

I don't think that a science which at its core deals with the human causes and human consequences of climate can be unpolitical.

It's definitely fair to have different views, just kind of weird to argue against politics and then make highly political recommendations. In my opinion, the key is rather to work towards a more empirical, more transparent, and more rational discourse (a goal which becomes more difficult the more economic, affective, and health pressure builds up on people).

[1] https://spectrum.ieee.org/book-review-the-climate-fix

[edit] Since some commenters took so much offense with this position, let me clarify: Of course you can try to do unpolitical science. But anything that recommends or implies solutions (or lack of problems) is inherently political. If one believes climate change is happening (as the author does), it's impossible to talk about causes and outcomes without being political.

Original link is dead, there is a whitepaper here: https://uploads-ssl.webflow.com/6495638f8c53d703ce05d882/64a...

I don't have a clue what this is supposed to be though.

Funny to see this here.

For anyone wondering: Although most people find the theory to be intuitively sensible, it is hard to test empirically due to its micro-macro links (individuals to society). So far, there are no conclusive studies showing that the phenomenon exists robustly.

Some notable criticisms include the complete omission of interpersonal and group influences and the assumption that people behave privately (e.g. in the voting booth) as if they were judged in a social setting.

So, really fascinating and useful, but nothing to base your system of belief on.

Yes.

There is the pushift dataset covering posts and comments through 2022 [1].

And the ArchiveTeam has begun crawling reddit as well some time ago [2]

[1] https://old.reddit.com/r/pushshift/comments/10bwxke/updated_...

[2] https://news.ycombinator.com/item?id=36254172

Yes but fine tuning requires a lot more gpu memory and is thus much more expensive, complicated and out of reach of most people. To fine tune a >10B model you still need multiple A100 / H100. Let’s hope that changes with quantized fine tuning, forward pass only etc.

Reactivity is great. Is there any framework for using it without a REPL?

I.e. to define a DAG of tasks and have them executed as needed? I know existing workflow engines, and they are typically not reactive but rather work on batches.

Yes, many workflows are already made up of a lot of tools that are individually great but require a lot of duct tape. I guess the best path for a new effort is to work towards making any aspect of it more automatic. That's why even entering a DOI by hand seems like a regression, when Zotero can extract it automatically in most cases (as well as ISBN for books etc.).

Some areas where I would love to see innovation, and that seem entirely plausible for papis:

- Collaboratively clean metadata (i.e. share sets of papers with collaborators in a less silo-esque way than Zotero's online libraries

- Do a better job at collating related versions (i.e. multiple related talks, preprint, online first and published version) ...

- Automatically track and import new papers into staging area

- Automatically fetch PDFs from some external source (and share with collaborators)

Most of these are possible in some existing apps, but it seems that the designers always expect people to joyfully curate every nook and cranny of their libraries, when in reality I have five duplicate versions of many papers, each missing some crucial metadata, none being linked to the one PDF file that's somehow stuck on a WebDav share, and Google scholar metadata polluting BibTeX fields but omitting crucial details such as the publisher.

This field really is another case of "we can re-land and re-use rockets, but digitally organizing a theoretically well-structured information space is impossible".

I'm happy about all movement in this space, but I don't really see the unique value proposition yet. What is special about papis?

Being hackable (python) is certainly nice, but many of the steps listed on the homepage are a lot faster and easier with other tools such as Zotero or straight BibTeX. Especially manually entering titles or other metadata on the cli - that would be the seventh level of hell for me. I want one click on a paper's webpage and be done.

That's not really the interesting part.

The interesting question is: Is there a quasi-monopolistic predatory exploitation of a historically captive market? And what are the mechanisms to destroy it?

From my perspective, it's clear that we have a multi-billion dollar/year heist by corporations whose value proposition (logistics, layout, quality assurance) are quickly eroding. They need to die yesterday.

So let's assume an (impossible) perfect efficiency of spending, they can buy how many DGX with this? 100? 50? In any case, the money's probably not even enough to be competitive in terms of hardware, let alone anything else.

Of course it's a dire problem which is not nearly taken serious enough. Interestingly enough, some fields seem to have some intrinsic motivation to replicate, e.g. biology [1]. I hope we can create similar incentives for all sorts of science.

(caveat emptor: I don't know anything about biology)

[1] https://statmodeling.stat.columbia.edu/2022/03/04/biology-as...

All the concerns are valid, of course, but there's one important caveat to cheating in academia:

Your career is a path dependency, and by cheating now, you may be brought down at any point in the future. In other words, cheaters need to make sure that their misconduct is detection-proof for a long time. That's not trivial. And once your career is tarnished, it's not easy to remain in academia, so your whole life concept is sabotaged.

Of course, some big players managed to weather scandals, but they arguably still took a serious hit to their credibility. For example, TED-famous psychologist Dan Ariely, who was the co-author of a paper with spectacularly forged data [1], and had to do an embarrassing public apology.

[1] https://statmodeling.stat.columbia.edu/2021/08/19/a-scandal-...

It's kind of surprising that single satellites can still be so crucial, given that we have at least three operational navigation systems which phones can easily use (GPS, GLONASS, BeiDou and Galileo). Are those not all truly global?

This is precisely what happened: "reformed" inmates (after a year or so) would be matched with newcomers and instructed to "help" them in their own path, i.e. to beat, humiliate and criticize them.