BookWyrm is a federated Goodreads replacement | Hacker News Reader

BookWyrm is a federated Goodreads replacement

wedistribute.org

327 pointsdredmorbius88 comments

GoodReads is all but dead for many. A few months ago, they deprecated their API keys (without notice).

I started building a GoodReads-equivalent but realized that many people care more about learning from multiple media formats (blogs, videos, research papers, interactive explorables, podcasts, courses etc) rather than simply reading books. Because there are too many projects attempting to build a new GoodReads, I ended up building something more oriented for multimedia learners.

It’s up and running, open-source, built with Ruby on Rails and supports protocols like ActivityPub, RSS etc. There is a companion browser extension, integration with Slack groups, Twitter and more, and a WIP mobile app. Like GoodReads, you can build your learning lists and embed them on your sites.

It participates in fediverse so your reviews can be broadcasted to your followers on Mastodon, PeerTube etc but full federation is not yet there because it has an underlying knowledge graph that will need to be synchronised across instances.

I think many HN users may be interested in checking it out:

https://learnawesome.org/

https://github.com/learn-awesome/learn

One of the cool things that the knowledge graph enables is that you can follow “topics” and get learning resources like latest research papers for THAT TOPIC in your feed - even outside the main site. This is the power of ActivityPub which is really a distributed pub/sub for the open Web. My motivation for building this feature came from the reproducibility crisis: I want to be alerted when something I believe has come under question or has been falsified.

Where do you get the books from? I'm often surprised when I start reading an obscure Dutch second hand book that Goodreads has it. Is the ISBN database open data?

OpenLibrary (relatively less known project by Internet Archive) is doing good work on collecting identifiers for books and works, which LearnAwesome leverages. But if you check in app/utilities/book.rb, you will notice that we have our own data enrichment pipeline to work with multiple sources.

If you mean ISBNs in general, that's not available. However plenty of libraries around the world do publish their own list of ISBNs, and you'll find dozens of datasets and search engines that combine those into one.

You're gonna have to move away from western languages entirely to find a book that's not available in one of those.

Goodreads is owned by Amazon, and a lot of the data in various digital sources (both real and correct and random or mistaken) seem to come from people listing physical books for sale through Amazon. (Library catalogues are another common source)

The ISBN codes seems to be in some weird legal gray area.

The codes can be broken down so you can tell (to some degree) which language, geographic region and publisher was involved and whether the ISBN is valid (as in meets a checksum constraint) and has been "issued" (a continually updating list) managed in a decentralised manner but since it's an older system, it wasn't ever really designed for this kind of usage.

It also hits the problem, that these days you care a little less about the actual physical object, which ISBN was designed around, but certainly a trove of data hidden there.

GoodReads isn't about learning though. It's about books. I think there's value and potential in the niche you've chosen. I want to point out that the "learning" niche is very different than the "book review" niche and comes with its own set of very strong competitors.

This is all new to me. Who competes in this space? Or what are some good keywords to search with?

Learning? Just search for "learn <subject>" and you'll see a bunch. I remember my university had a partnership with one called Lynda.com, but that seems to have been acquired by LinkedIn.

I recently came across the API issue too; from their page on APIs [0]

> Goodreads no longer issues new developer keys for our public developer API and plans to retire the current version of these tools.

I generally find that I trust their book ratings more than any other sites, though, seeing as they are user-generated, and also given the amount of users being in the 10s of millions.

BookWyrm seems pretty fantastic and promising from what I see, but I don't know that it would attract the current Goodreads user. I'm really hoping it gets some traction.

[0]: https://help.goodreads.com/s/article/Does-Goodreads-support-...

> I generally find that I trust their book ratings more than any other sites, though, seeing as they are user-generated

There are far too many reviews for advanced copies seeded by publishers, though, and there’s no consistent disclosure or way to filter these out.

That, and the rating system is far too simplistic. When you’re optimizing for engagement then having users spend a lot of time reading reviews is probably good. It’s not optimized for time spent to find a good book to read, though.

Agreed. I was more so referring to the law of large numbers, hoping that from their large user base it's more likely to get a more representative rating than a site with a smaller user base maybe suffering from some sampling bias.

Wow you've got a lot going on. Do you have any stats on usage you feel like sharing? I'm curious how people have discovered this and how it has grown since this seems outside any circles I am around except HN.

So, more than 1,000 users have signed up as of now. But because all content is public (does not require login), most traffic is unauthenticated. Some topics are ranked decently in Google, so users come and pick a resource and leave. Which is completely fine (that’s how Google itself works) but that does mean that the social features are currently underused.

What exactly is wrong with Goodreads? I keep hearing a lot of hate about it, but the reasons seem never expanded on.

I use Goodreads to maintain a public bookshelf of books I have read or want to read. It's also nice that I can check what my friends are reading. Occasionally it's nice to read other people's views on a book. I typically tend not to agree, but it's nice that these exist.

That's it. Isn't that what Goodreads is about? Seems like a nice website and perfectly suited for what it does. Am I missing something?

Lots of discussions over the years about it here.

https://news.ycombinator.com/item?id=20904549

https://news.ycombinator.com/item?id=24454221

For me it works ok. But that's because I like you only use it to track. But if the other stuff worked better I might have used that too. A better recommendation system. Not insane loading times would maybe make me browse more etc. So much untapped potential here.

It does feel a bit neglected/dead (except for the fact that people still use it a bunch) - the changes come rarely. In some sense this is nice! It's a stable piece of software that does what it should do.

One thing I find lacking is that it seems to mostly be used by english-speakers. So the one case where I'd be curious to use the social networking features, for non-English-lanuage literature, it falls flat. There's no equivalent to it that I know of for the German-language world (that people actually use), which is irksome, because I'd actually use the social networking features there (whereas for English literature I know what I like and don't care what other people read, I don't know German literature as well).

But not becoming terrible is a nice side-effect of this...it's not even a managed decline, it's just pottering along okay! But it is odd. I wonder what their high-level strategy as a business is.

In the German-speaking world, LovelyBooks [1] is quite popular. It seems to be strongly focused on fiction, though.

[1]: https://www.lovelybooks.de/

- It sucks at recommending books. You may stumble upon some book in the home feed, but its recommendation is as good as if it didn't exist.

- Lists suck. For example, it never anticipates even something as simple as abandoning a book (though you can create that).

- There's like a whole industry of faking ratings and nobody cares about it enough to do anything about it.

- It's owned by Amazon, but Amazon basically didn't touch it since the acquisition almost a decade ago.

Well, they did so recently to kill the API access, which was its #1 selling point to me. I liked using it by not visiting their ugly design ever, but that's not a possibility anymore.

> Amazon basically didn't touch it since the acquisition almost a decade ago.

This is what I love about it. When something works for me (and Goodreads does), I don't want it to change.

It's not a good thing when there are glaring issues that remain unfixed, like books receiving 1 star reviews while they're still being written.

Personally, as a Goodreads user I don't care about the review ratings.

I'm there for: a) making a list of things I want to read b) finding more books by authors (including upcoming) to add to that list c) release dates for books on that list (and sorting that list by date) d) seeing what other people are reading

All of those tasks were time consuming before Goodreads and I'm extremely happy that some let's-refactor-our-website-for-whatever-reason push hasn't broken my use case.

1 and 5 star ratings for unreleased books are dumb but humans are really bad at 5 star rating systems anyway.

I find there's far more value in the number of ratings, but in the end books are often about personal taste.

> It's owned by Amazon, but Amazon basically didn't touch it since the acquisition almost a decade ago.

This is a good thing. In the recent year they updated the design in subtle ways to make it nicer to use and less rough around the edges too which was a nice surprise.

Goodreads will always win simply because it's where all the users are.

Like reddit or any huge site its usefulness isn't in the presented top layer (recommendations/front page/popular list of x) it's the depth of millions of users.

If you read a relatively obscure book and look it up on goodreads then you read through the reviews to find users who gave a high quality write up and then look at that user's profile. This will lead you to other users and books that are actual quality.

You could create a smaller website with a higher quality surface level but it's going to be infected with the bias of a small "elite" userbase making it a basically useless echo chamber.

In other words, stop trying, let's just stick with the current terrible status quo. That's a supremely unhelpful comment to make, intended to discourage anyone from improving things ever. Shall we all just go back to a cave and knock some rocks together?

It's a reality check for anyone who makes a better goodreads and doesn't understand why it never gains traction.

It's on par with complaining about facebook and creating your own better version of facebook while completely ignoring the reason why everyone is on facebook to begin with.

While I generally agree that everything needs work, I also believe don't fix it if it isn't broke.'

Yeah. It's poised to become Amazon's Google Reader, but just as with Reader there's nothing truly wrong with it aside from lack of maintenance.

You get spammed with hundreds of fake profiles trying to add you is the biggest disappointment I've seen myself firsthand (obvious phishing links included). There is a lot of botted activity.

Very excited to see a decentralized alternative to Amazon-owned Goodreads & LibraryThing, the latter which I recently learned is 40%-owned by Amazon through a subsidiary AbeBooks.

I'm not too familiar with the fediverse model. I noticed that registration is closed on the primary instance bookwyrm.social.[1] Is the ideal use case of bookwyrm for each user to host their own instance? Like this one: https://book.dansmonorage.blue/

As a user, I wouldn't want to sign up on someone's hobbyist bookwyrm instance only for them to shut it down later and lose all my data.

1. https://docs.joinbookwyrm.com/instances.html

That info is out of date, bookwyrm.social is in open registration now. A more accurate list of instances is available here: https://joinbookwyrm.com/instances/. I signed up on bookwyrm.social (here: https://bookwyrm.social), though it took a notably long time to receive the email confirmation message.

The whole instance is quite slow now.

Email took something around 20min for me.

Probably the HN effect. Give it a day and I'd expect the site to speed up as the traffic falls.

Yes, I'm pretty sure that's it but it was still working.

Just a bit slow.

yes, it's running slower than usual because of the traffic spike (more registrations in an hour than over the last two months)

> As a user, I wouldn't want to sign up on someone's hobbyist bookwyrm instance only for them to shut it down later and lose all my data.

Mastodon goes around this problem by only linking to servers that adhere to their covenant[0], which includes sane stuff like daily backups, >1 person with access to the server, minimum of 3 months notice before shutdown, and moderation.

So, any community you see here[1] promises to give you a notice. I'm assuming other software will eventually reach the number of servers where something similar makes sense to implement.

[0] https://joinmastodon.org/covenant

[1] https://joinmastodon.org/communities

As an avid Goodreads user, Id say 80-90% of active GR users are female teenagers. Id suggest promoting on booktwitter or booktok instead if you want more traction. There’s also StoryGraph in the running with the same if not more features and a pretty strong network effect already, but still miles away from GR. GR is the laggiest platform I have ever used, and the tech updates are minimal at best, but people still use it primarily because of the network size. Just pure network size.

> Id say 80-90% of active GR users are female teenagers.

This explains a lot about the lists I see being popular on there.

> As an avid Goodreads user, Id say 80-90% of active GR users are female teenagers.

Lol, I would have said "50yo vanity-obsessed male geeks" instead, judging by the activity in my feed.

I recently made a bit of an effort to use GR a bit more, to go through some of the challenges and increase my reading rates. The anglocentrism is tiring, though - all the "best" lists are dominated by mediocre Anglo writers. If I remember correctly, when "social lit" sites first exploded, they had basically segmented by language - I know Italians used Anubii rather than GR, for example. I picked GR because I was still somewhat in love with the anglosphere, I think I'm ready to move to something a bit more "i18n", but it's not clear what that might be.

If you want good lists, should look at shortlists for book prizes. Some are really good e.g. The Man Booker Prize.

Just had a look. Unsurprisingly, the demographic of the user base is skewed to IT professionals (at least at this time). Whether that’s a good thing is subjective, but I personally want broader perspectives on literature. I think this will be a difficult problem for bookwyrm to overcome.

I honestly wonder sometimes if HN is the best place to market a lot of the things that get posted here. A lot of people on HN are developers and I think they post here because it’s _their_ community but in reality it’s actually not a good fit for a lot of products. Maybe some dev/SAAS things, but something like Goodreads? That should be marketed to readers, not developers. The only reason it gets posted here is because the development side is interested (supposedly) in the decentralized technical nature of the product, but that actually matters very little to most people.

I don’t see this as marketing to get new users. It’s more showing peers something interesting you found or are working on. Of course they’ll be happy to get some users from hn, but I suspect that isn’t really the goal here.

That this isn't implicitly clear at minimum says a lot about general shift in tone.

This is a problem with any similar thing that already has a heavily established site. One is my hobbies is boardgames. The longest running site is Board Game Geek (BGG). It’s age shows (this was the front page of the site until very recently https://www.boardgamegeek.com/dashboard), but more modern competitors have had issues trying to get a foot hold. The most recent one is Board Game Atlas. They have tried all kinds of things to lure users, such as contests, but I’m sure the percentage of people using it is a fraction to BGG.

hi irrational, one of my hobbies too, currently playing hive solo, mint works an warp's edge). I understood the bgg site was overhauled a few years ago to make it more responsive. I suppose it's more modern and looked better on mobile but I liked the old dated interface as well. How does it show it's age and what would you have wanted? I subscribe to a few geeklists and have my favorite game a subforums bookmarked. I rarely go through the homepage but if I do I like the serendipity of it all. So much content. A "digital garden" where the flowers are the games

We get this service early and for free. To advertise this to the general public should be the least we can to.

With less than a thousand users, we can't really say anything about the demographic of the user base.

Let it grow for a few months and see.

It really feels like "federated" is to tech geeks (i.e. HN people) what "blockchain" is to a lot of non-tech people.

What does federated do for me at all in solving this problem?

Shows that the architects took an above average time considering the space, and that users may be turned off by data lock in due to previous community betrayal.

None of this federated stuff will ever gain mainstream acceptance. I would bet my house on it. It adds multiple layers of complexity and a ton of UX trade-off, and doesn't solve any problems for a typical user.

I wouldn't make that bet. Mastodon isn't mainstream yet, but it does have almost a million active users:

https://fediverse.party/en/mastodon

A million active users is at least two orders of magnitude away from "mainstream".

this worship of solving user problems pretends like the one or two companies running the big huge massive winner take all monopoly walled-garden networks are going to keep being good, going to keep doing things in the users best interest, are going to make better and better product.

it's a world view that does not click with how i view software development & progress. i see long term stewartship as hard, see most walled gardems eventually succumb to acceued bad decisions or events that they become saddled with, or poorly steered into, or simply not aligned with their users on. federation isnt an answer to a problem, it is the theory that many problems require more than one ultra-massified uber-answer. federation says that problems and their answers require experimemtation, require adaption. it believes in protocols to agree upon & communicate across people & communities with. federation is freedom to generate new interfaces & extend capabilities.

i think it's dangerous to only ask "what can help the user now". technical environments need to be diverse & potentiated, need to have creative possibilities open. federation gives us both a quick, well known, easy to adopt, easy-to-host users & communities basis, and gives us the liberty to keep exploring further. i would trade these liberties for no products, and i think users over time will see how trapped they are & appreciate living life not under the auspices of ultra-massified software titans, but instead, with smaller, more adaptive, more community based (but interoperating) federations.

Ever heard of email?

That's a stretch, as it has neither identity nor content federation. Would you call IP stack a federation protocol too?

I think so. The definition of federated is not as specific as you're making it out to be.

It's the main difference between the WWW and the centralized information systems that came before it.

I remember seeing the first Web pages at CERN, clicking through them a bit and being unimpressed. You sound like me back then.

I agree in terms of users running their own thing.

What I do imagine will happen is that one instance takes off, gets itself a board or VC to answer to, removes activitypub, runs away with the win

Tech people will be mad but overall nobody will be able to do anything about it as they'll just ground-up rewrite eventually. The users won't care about the drama at all

Money is a bastard

I'd love to be wrong on it and I'm not just sitting here doom'n and gloom'n, I'm trying to ActivityPub-enable all of my projects as it happens! I do believe it's something to watch out for though

> What I do imagine will happen is that one instance takes off, gets itself a board or VC to answer to, removes activitypub, runs away with the win

Any instance which did this would be totally forgoing the network effect of being able to link up w/ other instances. So this is quite unlikely, at least in most cases.

Any instance that did this wouldn't care, they have all the users

See also Google talk, Facebook messages, literally any other thing that was once open access ever by a big company. That racist Twitter gab is an isolated Mastodon install last I heard if you want to see this in action, though I wouldn't recommend going there

> doesn't solve any problems for a typical user.

Mentioning "Federation" as a feature will not be very appealing to the average user, indeed. It is more of a technical concept. But there are features that derive from an app being federated.

Ever heard people complain they were suddenly banned for no apparent reason, and have no means to get it fixed? In a federated service you may just go to another instance (and in future migrating / controlling your own content will be easy).

A big centralized platform has a single ToS, and algorithms throw random people from the entire userbase at you. In a federated system you can choose a server with rules / ToS that appeals to you. An individual server is like a community with own culture and topics of interest.

Wanna escape ads, algorithmic driven UI? Choose any federated server (currently fediverse is ad-free). Server adds ads later on? Migrate to one that does not.

federated means your bookgroup could set up their own instance & everyone could either post their book reviewws there, or you could aggregate your own reviews into that instance. it's the freedom to start new communities, while still being able to participate worldwide.

it means that there's a protocol there, at the heart, for having multiple parties cooperating. it means anyone is free to build their own server or client. it meams we're not all trapped in one walled garden, one silo, it means the freedom to innovate & evolve.

right now there is not a lot of models for how we the peoe can host things online in an interoperable way. we can still use http & the technology that we have (unlike block chain), but we can use well defined vocabulary (https://www.w3.org/TR/activitystreams-vocabulary/#activity-t...) & protocols as a base to communicate & interweave social networking systems.

I agree with everything you’ve said, but most users will never want to host their own instance. It might even be an anti feature, since it sounds like a bunch of work! This is not a benefit that most people are going to care about - at least in the way you have explained it.

I think it’s a different view. Technical-savvy people who know how the web was conceived and thought like decentralization because it promises freedom. In my (limited) experience users just want ease of use and nice UX, and just don’t take into account the long term implications of their choices. I’m afraid, like someone said earlier, that decentralization solves a problem mainstream don’t have and don’t want to have.

I think the mainstream does care about decentralisation, but only in so far as it affects the final experience. Censorship on major social networks is a great example. People absolutely do care, but starting their own Reddit (for instance) is far outside of their abilities. Where would you even begin?

They also don’t care about how it is implemented, just so long as it works. Sometime I feel like the HN crowd cares more about what language a product was written in than how well it serves their needs!

What is required is a platform which combines the UX of a centralised service with the ability to easily spin up new instances, is open-source, security audited, etc. etc.

I agree. And also I really believe that any good software is made up of a lot of tedious and boring tasks, like editing the copies, making the QA, fixing bugs, and so on, and that these kinds of tasks are seldom done as voluntary work; what I'm trying to say is that without a company paying developers for doing this, it's difficult to have good software. Let alone to keep a service up and running 24/7.

To my knowledge, very few companies have found a way of monetizing decentralization in a sustainable and efficient way (I really hope someone proves me wrong with some replies to this comment).

I'm still trying to understand how this works at a functional level.

I see you can find reviews via ActivePub services, like Mastodon, but you can only post from BookWyrm instances? i.e., you can't have one federated social identity that can post reviews and post other fediverse content?

No. Unfortunately, the fediverse has kind of given up on the idea of a single identity using a variety of services. Every new ActivityPub services wants to own the user account and the interactions are one way or extremely limited (likes/reposts).

You're correct that each server has separate accounts, and you can't log in to, for example, but mastodon and bookwyrm and pleroma using the same account. You can do more than just like or repost though - a bookwyrm user and a mastodon user can do a variety of things like follow on another, send DMs, and reply to each other's posts.

I may be wrong, but all I have seen "federated" resembles one of the worst and most harmful websites ever created (twitter), which is really disappointing and disgusting. Likewise, the "blockchain" scene talks about innovation all day but the bulk of their creation is boring and dated, recreating the banking/finance scene using new tech with minor changes. Back in the day, people thought machine guns would produce peace!

I think the key is to seperate the catalogue of books and authors from the social and recommendation elements, which is another aspect of federation and de-centralization.

If you're aiming to lock people in and own all their data, then you need to combine the two, but then you kill the social elements by sucking in lots of disparate communities. You can do something sub-reddit like, but if you're not actually interested in sucking up everyone's information then you can slice off the catalogue part to openlibrary and wikidata and have a much easier time building your specific social thing on top.

Bookwyrm (and inventaire.io) seem to have adopted this approach, building on OpenLibrary and Wikidata catalogues.

And in the background those two data projects interact at various levels and with other open data projects that build from the same sources. For example, there's wikidata projects that try to seperate out all the acadamic authors with the same name, so you can have multiple John Smith's and find the right one and connect them across different databases.

Goodreads is amazing at what it is, which is an IMDB equivalent for books. Yeah, it is neglected, outdated and owned by a shady megacorporation, but I don't see any viable alternatives as of now.

As far as I'm aware, it's the only service that lets you easily find out whether a book is part of a series/universe, where/how it fits in there, and notifies you about upcoming entries in series you follow (email notifications have never worked for me, but at least you can check on the site).

Essentially series/universe discovery and news. This used to be massive painpoint for completionist me before Goodreads.

If anyone knows of any alternatives I'm all ears by the way.

Are GoodReads/Amazon and others cool about this use of their exported data? Can BookWyrm expect lawsuits? Genuinely curious, not trolling.

In the EU, GDPR makes it clear that as the user, you own the data. Companies are legally required to allow you to export your data, also in part to create more competition among platforms.

No idea about the US, though, where social platforms seem to sometimes come with clauses that make users waive their copyright (which would be unenforceable in Europe).

I like goodreads a lot often check reviews there. I was missing recommended books based on more than one book. Something more nuanced than books similar to x.

I created my own solution for it and published it here:

https://what3books.com

at first sight very promising: an online application that is catering to specialized interests (hence very specific data / internal business logic) yet able to be both decentralized (multiple instances) and integrating via activitypub with other (social media type) platforms in the fediverse

the moment the open source community realises that open source is not just about freeing up things conjured up in proprietary context but opens entirely new universes will be a tipping point of sorts...

Heads up: The GoodReads export takes a long time.

I'm still waiting for mine to be delivered.

There are two types of data exports from GoodReads - a full data export, and a csv download. The csv download is what you want, and it is generally very quick (https://www.goodreads.com/review/import)

Thank you, I confused them both.

Will it have fake quotes too?

the signup confirmation email for BooKWyrm seems to have stopped working.

It's just running very slowly! There have been a lot of signups and it maxed out what mailgun will send. If you don't get an email you can contact the instance admin directly

"A portion of the proceeds go towards the Yunakin land tax."

An _Anti Corporate_ organisation adopting a very corporate measure in my view.

If taxes could lead to justice, then native Americans would be far less prone to poverty, alcoholism and depression.

About the product itself, it's hard to see how this is an alternative to goodreads. The federation is designed in a way that silos communities. A great model for decentralised moderation etc. The parallel could be made with reddit, not so much with goodreads where over half of the books in existence are listed, very often rated/reviewed.

> The federation is designed in a way that silos communities.

You seem to have taken the exact opposite view of what anyone in the fediverse would consider this as. This is many connected communities. Not siloed. That's what "decentralized" means, their very first major point on the website.

If there's real gaps in how federation works to connect, the fediverse is happy to discuss, adapt, & adopt: bring on your more specific complaints! This is supposed to be the cure for silos! Everyone can run whatever they want, and we can interoperate! Interoperate & co-evolve.

Sorry I wasn't clear with what I meant stating silos. And I'm all for decentralisation, and value the open effort of Bookwyrm.

Silo: the federation aspect, from what I understand of the currently hosted communities using this tech is making the exploration of books contained within each federated host. Maybe I am wrong in which case I will retract my critic. Peertube suffers the same problem. So a constructive comment would be: decentralisation is great, and if the federation design bridges the gap so that information from each node in the system can be searched then that's a rock solid solution as a repository of books information, reviews and discussions.

Please accept my apologise, I certainly came off as negative and the comment I made was not deserved , especially considering the free and open effort there.

With PeerTube there's Sepia Search [1] which allows searching for videos across all instances. All federated networks basically need open-source, centralised search engines like Sepia Search to enable users to search across all federated instances.

[1] http://sepiasearch.org/

Reply on news.ycombinator.com