silveraxe93

the Javascript tag is misspelled in this article too:

Karen on Python mocks: threading.Timer

This is quite a good explanation of transformers that gets shared a lot. [link](http://jalammar.github.io/illustrated-transformer/)

And here's a super simple implementation of GPT by Andrej Karpathy. [link](https://github.com/karpathy/minGPT/blob/master/mingpt/model....)

Hey thanks for the super detailed response!

Yeah I was trying to do something that didn't quite fit with the spacy-transformers API at the time. I did get a bit of a headache trying to use thinc at the time, which was just when you guys did the redesign I think, so the docs were different from what I was seeing. I might not have searched enough though.

I didn't try it yet, but it seems that transformers got added to spacy v3 with first class support.

I did gain something from rummaging though spacy source though! NN layers were composed into module-like pieces, then added to this REGISTRY variable though a decorator. That way some things could be defined at runtime. It was super elegant.

I nicked the concept of that for my data preprocessing pipeline. Saved me a lot of time when trying new things.

re spacy-transformers. I really wouldn't recommend it. I tried using it but was a nightmare. They had a dependency on a previous major version of Thinc (spacy's NN backend) but removed the documentation for that version. I wasted a week trying to deal with it until I gave up and went pure pytorch.

Spacy v3 seems to have integrated the package functionality, so I'd go for the nightly release instead of this.

Spacy v3.0 nightly is out, which has integration with transformer models. So if you already have some familiarity with the package it might be worth a look.

It should be very similar to normal spacy usage, just instead of downloading "en_core_web_sm" etc, it's "en_trf_foo_bar"

I agree with your broader point, but I doubt they ever claimed that?

I don't think the FBI would ever say that without cracking encryption they'd catch no criminals. Only that many who don't get caught today _would_ be.

I think it's obvious that if less people speak their mind, we could have a smarter, happier humanity.

But I also think it's obvious that I shouldn't trust you to make the decision of who needs to shut up. And definitely not the government.

I mean, 0.01 is objectively a tiny number. The problem is when you multiply by 328 million ;)

Dammit, I thought of screenshotting the error, but was too lazy. Now I gave an unreproducible bug report... Surely this is karma at work.

Under the details section, it says it's valid RSS with a link to a validator. However, if you click it the results come out as not valid...

Might be good to check that out!

Hanlon's razor is great. I feel it's important to always keep it in mind once you see something stupid (or evil) being done. Stupidity is the most likely explanation, but malice is the most satisfying.

I think politicians really care about these things. But these big sweeping laws and changes are because of hubris (but then again, the job selects for that).

Society is damn complicated, everything affects everything. Making a change in one small part propagates quickly and unpredictably. But politicians don't internalise that (of course, if they did they would never get elected on the platform of "stuff is complicated, I don't know if this will help!").

So they see a problem. Our information is being used maliciously! We need to stop that! Protect the people! So some massive law gets passed with tons of unintended side-effects and it's captured by big corporations at the speed of light.

It's even more of an absurd criticism when the literal banking system is "running tabs". Banks don't move dollars around, they don't even hold these dollars! It's all IOUs and lending money that literally doesn't exist.

In some sense, this is an argument in favour of bitcoin.

You seemed to be making a programming related joke, which I responded with another one.

I wasn't really trying to make a point there, just wanted to be (arguably) funny.

If my dad died of Tetanus, should I be against memory safety?

This is just so damn pointless. It will cause some "great" controversy, people will be called white supremacists for refusing to stop saying master.

Now even if this catches on, then yeah OK I'll start naming my branches as main unless I want to make a political statement, which would just add to confusion when developing.

Meanwhile these people will pat each other on the back, yell "yeah! We sure made a difference." while black people keep getting fucked by the system and their lives don't get improved one iota.

just... why?

I was reading a post yesterday about how someone found difficult to gauge the level of violence in the current protests. As media is so polarised that you either get shots of peaceful protests or complete mayhem, depending on the paper's bias.

This got me thinking: What if we get the police radio transcripts, and extract every mention of an address. Then have a central hub of drones that get dispatched automatically to any address of "interest" filming.

It's a bit crazy, most likely illegal. But definitely would be cool to see! Arguably truly unbiased "reporting". If you want to know the situation, just plug in to the feeds and get a decent overview of how it looks.

His point is that plans like this are generated regardless of the current state of the country, therefore it says nothing about how polarised or at risk of rebellion America is today.

My application to CocaCola got rejected, but I guess Pepsi is fine too. ;)

Lambdas are assigned to function arguments/parameters. Pretty sure assigning a lambda to a variable is an anti-pattern.

While I agree with the main thrust of the article (Chesterton's fence, etc.), the author seems to take as a given the fact that Netflix is sliding back into becoming as expensive as cable, because the content providers are setting the prices.

But Netflix's value is not from competing against cable on price, it's competing against piracy on convenience.

Content providers will never again see the same profits, because the de-facto price of content is zero. It's too easy to pirate, and morally grey enough that people are willing to do it.

Netflix got so big because it captured the market of people who were willing to pay for the convenience. If this gets too high they will just go back to pirating.

Sorry if I came out a bit snippy out there. But yeah I assumed you meant python without numpy, etc.

A lot of the criticism I saw was because the core routines did not need to be packaged up. There were a lot of common data structures reimplemented, etc.

I don't think the model had many novel routines. It could be built just using industry standard and tested tools in python, R, Julia (if you really want speed) etc. But it reinvented the whole ecosystem in one big ball of C.

tbh, this should have been built on STAN or similar. There's so many variables and assumptions that the output is completely dominated by the parameters chosen. Seeing the distribution of outcomes instead of a point estimate would be actually useful.

It seems the government was following this document at the start: https://assets.publishing.service.gov.uk/government/uploads/...

The reason it seemed they were doing nothing are these passages:

ii. Minimise the potential impact of a pandemic on society and the economy by:

• Supporting the continuity of essential services, including the supply of medicines, and protecting critical national infrastructure as far as possible.

• Supporting the continuation of everyday activities as far as practicable.

• Upholding the rule of law and the democratic process.

• Preparing to cope with the possibility of significant numbers of additional deaths.

• Promoting a return to normality and the restoration of disrupted services at the earliest opportunity.

There's way more, but I've honestly not read it all. But there was a plan, drafted before this epidemic.

Public opinion was turning against the government, but it actually kept course for some time. Something I was honestly impressed with. What made it drop the plan was Neil Ferguson's study.

There are many reasons for criticising the plan. This article is pretty good. https://www.theguardian.com/politics/2020/mar/29/uk-strategy...

What really gets me is that if the lockdown was the correct decision, we arrived there for the wrong reasons.

This paper had such an outsized impact that it should be held to a higher standard. And it's scary (but not really unexpected) that the government is making decisions of this magnitude based on such a shaky foundation.

Being -- capable of writing quality software -- should be a base requirement for -- Anyone developing software used for academic research -- in any field -- in which the conclusions are based upon the correctness of this software.

Not all academic research depends on writing software.

I guess we are in agreement then ;)

If academics are writing predictive modelling software, I ask that they are qualified to do so.

Any serious attempt at modelling this over python would use the pydata stack (numpy, pandas, etc), which run on top of C++ anyways.

I couldn't agree more. We're in the middle of a reproducibility crisis. It's extremely important that researchers start putting more effort into quality research instead of quantity. "Publish or Perish" is killing academia.

Where in my comment did you see me proposing that "being a fully experienced & knowledgeable software engineer should be a base requirement for all academic research (in any field)"

Go back and re-read it please. I'm saying you need to be capable of using the tools for your job.

If I was working as a software engineer, I'd not be asked to perform heart surgery. A more realistic example would be a software engineer being tasked to develop software in the medical industry, where I would expect them to understand the subject matter enough to ascertain their software is not crap.

This model didn't just influence decisions in healthcare. It single-handedly changed the UK government's strategy over this pandemic.

From what I understand the UK was planning on beating COVID by creating herd immunity, similarly to Sweden. Then this model came out and everyone started yelling that Boris wanted to kill your grandma.

The problem is that it's impossible to have an intelligent discussion over this. This pandemic became a partisan issue. We're not discussing whether one of the most impactful decisions made by a government this generation should be based over absolute trash code. You're either uncritical of the lockdown or "anti-science".

I disagree so strongly with this that I had a visceral reaction while reading it.

C++ is a tool, not an end product. If you're not qualified to use a tool correctly it's not the manufacturer's fault, it's yours.

Why do so many people believe that good software development is not part of their job? If you write code then you're a developer, no matter your job title. If you write shit software, saying you're a researcher is not an excuse.

I don't think it increased the amount of discussions, just centralised it. Before it would happen between developers in the same repo, now it's all in the same place.

I think that a good thing. Formatting discussions are important (arguably), but it should be decoupled from a specific project.