Hacker News Reader

Comment by jesprenj | Hacker News Reader

Wouldn't it be better to just register http as a protocol with fsspec and use

    http://csvbase.example/username/dataset/table

instead of creating a whole protocol handler just for an online service?

View on HN

http is already registered as a protocol and you can indeed do that already. It's printed on every table page. eg on https://csvbase.com/calpaterson/opcodes-6502:

    import pandas as pd; opcodes_6502 = pd.read_csv('https://csvbase.com/calpaterson/opcodes-6502', index_col=0)

But writing back is harder. It's not easy to make pandas do an HTTP PUT and then insert the HTTP basic auth and so on. Plus (not discussed in the article) there is a cache to avoid redownloading the data when it hasn't changed, which is my personal bete noire in "data science" such as it is.

Ah, now I understand the motive! I wasn't thinking about writing back to csvbase at all.

What happens if N projects attempt this?

Reply on news.ycombinator.com