Hacker News Reader

Comment by medimikka | Hacker News Reader

Zotero is, if you're not in the market for a closed source silo like EndNote, the only game left in town. Mendeley went full on Evil, and Papers for Mac is Mac only, closed source, and missing dozens of functionalities that are absolutely a must have in academic writing and research.

An absolute must have is BetterBibTex (https://retorque.re/zotero-better-bibtex/), which adds better Citekey management and, my personal highlight, "export on add" functionality. In short, everything I save into a specific folder in my library gets exported as a .bib file right away. That way I can click a button in my browser and have the citation ready in my LaTeX editor, Word, or Obsidian (https://github.com/hans/obsidian-citation-plugin) within seconds.

The latest beta also adds full Markdown export for Notes. I tend to keep my notes on papers in Zotero, attached to the paper, but export them for filing in Obsidian (which I then feed into MkDocs for our work group's large repo).

Unless you need Apple Pages support, which Apple keeps to itself and only sells for mighty moolah to a select few, Zotero can do anything EndNote can do, is Open Source, and with that won't drive your PI up the wall with yet another expense.

View on HN

>Zotero can do anything EndNote can do, is Open Source, and with that won't drive your PI up the wall with yet another expense.

My university offers EndNote (plus Web of Science and the Clarivate stack) for free for students and many of them don't use it - they'd rather manually manage the bibliography in their Word documents. The university even ran tutorials to show them how to set it up, add citations, and export the references, thinking maybe the students didn't understand the value of bibliography management software. Nope, even with the tutorials there was almost no uptake.

It turns out the problem was EndNote all along. I tried it myself - one of the worst user experiences imaginable, and I could totally understand why even doing references by hand is preferable. Eventually I told some of my own students that I use Zotero and to give it a go even though we're not supposed to endorse other software. Two weeks later most of them were using it.

So I'm actually quite confused at the fact that Zotero is offered as a free alternative to EndNote, unless the EndNote that I've used is some weird dated inferior version...

The university just needs to set an assignment which needs to be submitted twice: once with citation style A, and again with citation style B. Anyone not using citation management software will quickly start.

The point about academic research is freedom, in every possible meaning. Including not using the right tools for the job. This ensures "natural selection" of the ideas, processes and workflows that actually work.

Having used Zotero, Mendeley and Endnote I ended up on Endnote. I like their metadata scraping, their Word plugin and that many journals offer a pre-configured reference template. But yes as a DBMS it lacks alot in the filter/grouping features and date fields don't convert format intelligently.

First manuscript in my PhD I just hand-typed the bibliography. But when you get to a few hundred in a document and over 1000 relevant to your work that you want to keep track of, ah and when you need to resubmit manuscripts with a different reference format, and once I made the jump to actually using it and figuring out a few of the quirks, I now quite like Endnote since my institution has a site licence.

Yes. Professionals don't hand-edit references in the writing document. Update the template and click refresh. That said, I also use Endnote because it's free for me.

I quite like Endnote. I'm unsure if Zotero has a Word plugin.

In Word, alt-1 click reference, alt-2.

Its done. Reference inserted. Bibliography updated. Sweet.

Did you use EN9 or EN20? I found EndNote to be usable, just not something I'd pay $130 (student/academic version) for.

It took me until the update to 20 to realise it, but it was Endnote X9 i.e. 19, not 9

Exactly my experience as a student. Picked up EndNote, hated the interface and dropped it hard. I did need something to manage my references so I ended up on Zotero which is incredibly easy to use and just works. I use it to manage personal-interest files too.

I use zotero and I really want to like it, but it's not that good. The tagging functionality seems like an afterthought, the bibtex support is only reasonable with betterbibtex and even with it I have issues. For example journal abbreviations don't work consistently, every start up I get told that the betterbibtex styles have changed and I need to restart zotero... File management is also only bearable with zutilo, and I still get issues with sync and somehow some file locations getting messed up.

The whole note taking experience is also quite soso.

I really wish there was a good bib manager around. I don't want to go into a proprietary silo (ruling out mendeley endnote etc.). Jabref is in many ways better then zotero when one uses bibtex, but unfortunately some parts of the UX are not my thing. The best app I have used was docear, the concept of a Mindmapping app together with a bib manager was great however it is unfortunately not maintened any longer.

+1 for Jabref and a bibtex database.

I compared some options in 2005: https://www.coli.uni-saarland.de/projects/irtg/contents/Coll... (Not that much changed, except that CiteULike was sunset.)

JabRef is a good option for a simple desktop GUI based program that is cross-platform, but I find myself writing "raw" BibTeX entries in Emacs or Sublime more and more often without any UI tools.

Aaah JabRef, that was my choice 15+ years ago when I was doing my PhD. I think today a combination of portable JabRef and GoogleDrive/DropBox or similar would do the trick.

I agree that there are definitely better alternatives to Zotero.

please tell us what you think they are too!

If the town is "online reference managers," you are probably right, but I would argue that reference managers are one of those areas where you really want something offline:

- Offline ensures that you do not suffer an externally caused downtime just before a deadline

- Offline ensures that you have a path for keeping your database throughout your research career, and to do system updates when _you_ want to.

- Offline ensures that if you leave academia, you will always have access to local copies of the academic papers you have referenced.

My favorite offline/local reference manager is `jabRef` [0] which stores all metadata directly in a bibtex-file. The GUI has an excellent pdf-integration, and everything is local and super fast.

Case in point: after a decade in industry, I am looking to get back into my academic fief. All the papers I ever read are in my Dropbox, and all I had to do to pick up where I left was download a current version jabRef and point it to my database which it read without any issues.

[0]: https://www.jabref.org/

Just to be clear, Zotero is primarily a local program that saves all data and files to your computer. You don't need an account to use it. There's optional syncing and a web library so you can use it on multiple devices and collaborate with others, and there's a browser extension (which JabRef uses as the basis for their extension) that integrates it deeply with the browser, but in no way is it an "online reference manager". We strongly agree with all the reasons you give, which is exactly why we designed Zotero the way we did.

(Disclosure: Zotero dev)

I use Zotero and sync the attachments to my Nextcloud instance via WebDAV. The database is synced to the Zotero servers, because that's the only option. With a bit of care on initial setup for new machines, it works wonderfully. The real value are the PDFs, some of which are of course really hard to get by. And those are "local" in the sense of "local server", yet available to all clients.

There was a forum entry that self-hosting the database is a topic being worked on. That's my number one gripe with Zotero, it just seems weird: if using multiple clients, users have to sync to the Zotero servers, yet the storage space for attachments there is minimal (which is okay; Zotero pricing looks reasonable but of course, own storage via e.g. Nextclodu is much, much cheaper). This basically forces a hybrid solution, with bring-your-own-storage. If all of Zotero's data could be self-hosted, now that would be a streamlined, great experience.

Then again, I am anal about my entries' metadata and often correct faulty data found online. Syncing that to Zotero servers is kind of like giving back? I always thought that data might help improve Zotero's own auto-fill functionalities, which I'd like helping with.

See my comments on [1] for why we don't support self-hosting. Short version: it would pretty much just be an entirely different project.

> Syncing that to Zotero servers is kind of like giving back? I always thought that data might help improve Zotero's own auto-fill functionalities, which I'd like helping with.

No, we don't expose your data to anyone else. (And tools that try to do that usually just end up filling people's libraries with junk data.)

You should sync Zotero if you want to 1) access your library on other devices, 2) collaborate with others, and/or 3) have a real-time backup of your Zotero data.

[1] https://github.com/zotero/dataserver/issues/105

I've been syncing my full-texts via syncthing for years and years now and I've never had to think about it after initial setup.

Zotero is great! :)

Thank you for clearing this up - and apologies for not researching!

Thanks for posting this - JabRef looks perfect for helping me organise and track papers to read, I had no idea it existed!

Zotero and Obsidian is a killer combination.

If anyone is interested, I have some videos on research tools that include a Zotero->Zotfile/MDNotes->Obsidian workflow: https://www.youtube.com/playlist?list=PLHmevVAAXtu3_beDLtsTm....

>> Zotero and Obsidian is a killer combination.

Could you elaborate on this a bit?

I currently keep my Zettelkasten in Obsidian and my project journals in Workflowy, and I wonder if there’s a place for Zotero in my workflow.

> Mendeley went full on Evil Could you explain what happened with mendeley? I've been using it for a while, and didn't notice anything different really.

They locked down the SQLite database so that you couldn't use it with 3rd party tools (such as your own commands, or to export). I am not sure if this got reversed, but at the time it was what triggered me to switch to Zotero. (Still using Zotero, which does everything I need it to.)

One thing I like a lot about Zotero is that they charge a subscription once you get over a certain amount of cloud storage, which gives me more confidence that they'll continue to exist in the future.

Yes, the DB was encrypted for no good reason except that locking users. Funnily enough at that point Mendeley could import from Zorero but Zotero could not import anymore. Thanks to Greydon Gilmore and GDB, I was able to get out: https://blog-academic.greydongilmore.com/phd_related/mendele...

Greydon's page is based on my original article: https://eighty-twenty.org/2018/06/13/mendeley-encrypted-db.

Hypothetically speaking: going to scihub, pirating an Elsevier paper, and then uploading proof that you had done so to an Elsevier cloud account doesn't strike me as the brightest of moves.

Not in the sense that I think you are guaranteed to see consequences, or in the sense that I've actually heard about it actually going sideways, but to my intuition it does seem like the kind of thing that might attract one of those "pay us $10k or we sue you for $10M" letters down the line.

Especially from that company.

Mendeley joined the Elsevier "Family." Elsevier is one of the big three making massive amounts of money by selling content they receive for free, paid for by the tax payer or non-profits.

I strongly believe, that this has greatly contributed to the "Joe Roganization" of the pandemic, hiding true science and research, often created under the auspices of governmental research trusts, behind a paywall. Anyone not in academia (where Elsevier charges hundreds of thousands to institutions to give their researchers access to their own papers) is more or less SOL and gets their data from Joe Rogan.

In that vain, Elsevier has a tight control on the app now. Zotero has the ability to find the open access version of a paper, Elsevier will always link to the paywalled version on its own servers. Zotero has (and Mendeley had) the ability to export libraries, and with that data, Elsevier removed that when they took over Mendeley development. And, worst of all, Zotero can run standalone, Mendeley now requires an Elsevier ID login. The company actively analyzes your reading data and advertises other Elsevier properties based on your interests and reading habits.

This is academia. It should be open and accessible. Elsevier is doing everything they can to change that. And Mendeley is one weapon in their arsenal.

The Joe Rogan part makes no sense to me. Only a fraction of the listeners of the podcast would otherwise go out of their way to read primary science journal articles. Reading those is a skill and requires background knowledge to be able to read between the lines, shared context between experts etc. Scientific articles aren't written for a lay audience.

Also, sci-hub exists.

> Also, sci-hub exists.

Try adding one of those papers to Mendeley. Or, for that matter, get it by request from the author. Legal and often fast. Add that. See what Mendeley does with the CiteRef.DOI and CiteRef.URI fields...

There should only be one canonical DOI for any given article though--that's the whole point! I think the URI is also supposed to be to the "version of record" rather than wherever you happened to find a copy.

Are we talking about Mendeley or the general Elsevier paywalls? Seems like you are complaining about several issues at once: Mendeley's restricted features, Elsevier's paywall (that can be relevant even for non Mendeley users) and JoeRoganization, by which I guess you mean that his guests don't always confirm mainstream expert consensus (The Science).

Mendeley changes certain fields on import to always download its abstract PDF as the reference paper with a link to its paywall. In essence, those things are PDF archivers and citation managers rolled into one. If I were to send you my latest research and you'd import it into Zotero, you'd get a managed citation and my paper. In Mendeley you'd get a managed citation and a downloaded abstract version with CrossRef to their paywall.

The reason this happens is Elsevier's business model. And this business model both really wrecked Mendeley and, I believe, contributes to medical misinformation being more dominant in the world.

But Joe Rogan listeners aren't using Mendeley. These issues are separate, but you're angry at both and are somehow mixing them. The paywall can be circumvented via sci-hub. And more and more papers are now on biorxiv and similar.

And I don't think medical misinfo has much to do with Elsevier or academia. It's human nature, and the quality and degree of evidence-basedness of official communications doesn't really help.

Also, the words "misinformation" and "fact check" make my skin crawl.

If your conclusion is that the plebs is so dumb because they listen to Joe Rogan instead of The Science(TM), I think you're just digging in deeper.

Also, during the pandemic, many blue check Twitter accounts said it's a pity that so many conspiracy theorists and fake news believers are reading papers and playing around with data. They should just receive The Science, spoon-fed, and accept it, and too much thinking and reading hurts them. In other words, think about what you wish for. When other people are given access to data, they may reach different conclusions than the approved respectable media expert consensus.

this isn't about joe rogan, its about someone stealing public medical data

Not defending Elsevier's practices, but people who get their information about the pandemic etc. from Joe Rogan are probably not going to read (or even understand) a research paper from an academic/medical journal. The problem with misinformation isn't lack of access - there are plenty of accessible, high quality articles that are written for the layperson - the problem is that people are reading (either through choice or The Algorithm) misleading and sensationalised tabloid clickbait.

I find myself being surprised by people more often than disappointed.

The optimist is often disappointed and the pessimist often delighted.

> "Joe Roganization" of the pandemic

Not sure what that means? Is he hiding true science and research?

My best guess is, the influence of non-specialists who do research on their own, without the training to understand which papers are meaningful, which are fluff, which are wrong, and which are fraudulent.

The most informative papers are often behind a paywall, so the non-paywalled papers are biased towards these other less-informative and even wrong papers.

If someone should point to paper X, in order to correct a mistaken belief discussed in a public form (like HN), and that paper X cost $50 to read, many fewer people will read it to examine the validity of the paper in the first place - even if were correct.

While if someone points to paper Y, which supports that belief due to subtly wrong assumptions, but that paper is free to read, then many more people are likely to look at it. And without the training or experience to identify the wrong assumptions.

None of this requires our "Joe Rogan" to hide true science and research - paywalls already do that.

> Is he hiding true science and research?

For him it's probably sufficient to just use a heavily-biased selection of it and/or present a heavily-distorted version of its results. (As others in the thread have pointed out, the hiding is done efficiently enough by others, i.e. the heavily-paywalled scientific journals.)

> the "Joe Roganization" of the pandemic

I think that was much more caused by media trying to prime people with some ridiculous stories that at least some refused to believe. A simple but honest approach is preferred to what we read in media otherwise.

We still have sci-hub at least...

Honestly, I am starting to think the open source movement has done as much harm as the subscription model to the scientific information ecosystem. The incentives are so high now to get volume published in pay to publish journals, leading to rushed peer reviews. Also APCs skew who can publish 'well'. The APCs for some top Cell and SpringerNature journals are about ten thousand fucking dollars. Great that the public can access it but that's money that either doesn't go to research or has to come out of taxpayer funded grants and straight into the pockets of publishers with huge margins. OA has also spawned a massive predatory journal market.

> Honestly, I am starting to think the open source movement has done as much harm as the subscription model to the scientific information ecosystem. The incentives are so high now to get volume published in pay to publish journals, leading to rushed peer reviews.

What does that have to do with Open Source?

> Also APCs skew who can publish 'well'. The APCs for some top Cell and SpringerNature journals are about ten thousand fucking dollars.

What are APCs? (Are they perhaps what relates to Open Source here?)

> Great that the public can access it but that's money that either doesn't go to research or has to come out of taxpayer funded grants and straight into the pockets of publishers with huge margins. OA has also spawned a massive predatory journal market.

But even ten thousand fucking dollars sounds a lot less than what those publishers are making now, so isn't it still much better?

oof -- that should have read 'open access' and was intended to pertain specifically to the scientific publishing ecosystem. my bad.

APCs are article processing charges. They are the fees the author (or sometimes the institution, if they have negotiated a blanket deal for all of their researchers) pay the publisher in exchange for the article being made open access.

> But even ten thousand fucking dollars sounds a lot less than what those publishers are making now, so isn't it still much better?

Publishers wouldn't be transitioning so many journal to Open Access if it was hurting their bottom line, and the for-profit academic publishing industry is about as profitable as they come with margins approaching 50%. They can do it under the guise of a principled stand. But it can create some perverse incentives. Also, previously they made money from subscription fees, so the bargaining power of whole university systems would negotiate rates. Now it for the most part falls to individual researchers, who don't have that economy of scale type bargaining power, which removes that check on inflation. A few Uni systems have cut similar deals to cover APCs for their faculty as I mentioned earlier, but it isn't yet the norm. /rant

So now the scientists get to PAY to be published?!?

Holy shit. If all this is for "peer review", it's time to scrap how they get that and develop a whole new peer review system.

(Are the Swedish Academy's science committees so superficial that they just compare citation numbers for awarding Nobel prizes? If not, what system do they use -- and why aren't they open sourcing it?)

Yeah increasingly scientists have to pay. It has become the 'principled' thing to make the publishers richer in the name of open access. Sometimes there are waivers for researchers from developing countries. Unfortunately funding bodies haven't universally caught up to the expanded $ needs. Nor have institutions.

There is a model with journals like PeerJ with relatively cheap APCs and a deal where you could sign up for a life membership and publish there indefinitely (but I think coauthors would need to as well). They also only publish based on scientific merit and aren't selective on 'impact', which is good. However, that means they aren't seen as prestigious, so you can't exactly build a career and get promotions going that route.

RE: peer review, in part bc of disruptions due to COVID journal editors have been having a harder time getting people to volunteer to peer review. There has been a lot of talk about moving to a model where reviewers get paid, but again that seems like it would set up some perverse incentives.

RE: Nobel, I don't think it is about citations, pretty sure the judge the impact of a particular body of work/discovery/contribution, qualitatively.

> RE: Nobel, I don't think it is about citations, pretty sure the judge the impact of a particular body of work/discovery/contribution, qualitatively.

Yeah, I pretty much knew (or at least assumed) that. My question was more a half-facetious way of saying academia in general should also do it that way in stead of continuing the current stupidly simplistic quest for journal citation counts.

Sure, I know nobody can afford to invest the same kind of work for every one of the millions of papers produced each year as the Nobel committee does for a few potential laureates. So it would have to be some kind of distributed system, based on the whole international scientific collective... An adaptation of the Slashdot rating system, or something? :-)

In addition to the other responses to this question, they also got rid of their browser plugins that let you quickly add arxiv pages/pdfs/etc in one click with automatic metadata detection. Worked from any browser on Android and desktop. I've yet to find anything like it.

They got bought by Elsevier who I don't trust for a moment and they are pushing some weird social media angles and the whole thing smells of proprietary vendor lock in, walled-garden etc. Zotero is much more hacker-friendly.

Mendeley for iOS system was discontinued and taken off from App Store for “no reason”. Users are forced to use their web version in iOS system.

I forget why I didn't like Zotero - I am using iLibrarian [0]. It is positioned as a SaaS, but the source is available and you can host it yourself. But I can't speak to how it compares to other software as I am not continually using it. It seems I just like collecting scientific papers rather than actually reading them.... Maybe if I live to 150 will I manage to read them all.

[0] https://i-librarian.net/

And I wholeheartedly second the fact Mendeley is Evil. I got sick of the bugs.

Oh damn. It looks extremely close to what I want (It's responsive and the PDF UI might even work on my e-reader), but it doesn't seem to support importing web pages?!?!?!?!?!?!?!?!?!?!?!?!?!?!

What a massive and disappointing oversight. Otherwise I could've seen it becoming my new research manager.

It extracts the metadata, but not the body, yes. Retrieved websites are generally frowned upon in my field (medicine, virology), and that's why I never really needed this. We're pandoc-ing our internal MD files to PDF and attach them to the relevant HTML pages, which makes this somewhat a crutch but circumvents the "9 out of ten reviewers hate websites" problem.

I print the webpage to a PDF and store it that way, then fix up the metadata as I see fit. And that is a good feature - make a request on the Github repo and see if the lead developer responds.

I thought the same thing until I discovered Paperpile. I highly recommend checking it out. https://paperpile.com

On the technical side, anyone know how their browser extension to save webpages.. works?

I'm writing a personal reader / reference tool. More of a knowledge base than a formal reference tool, but i want similar functionality, and so i've been researching ways to locally archive web pages to cite in notes. To save effort, i was trying to reuse existing methods in Reader-like libraries. The best i found so far, i think, is Telegram's library of "Instant View" parsers. A community managed set of scrape instructions basically, XPath and whatnot, that i could write an implementation for and - in theory - support everything Telegram does.

This is a fair bit of work though. I debated instead simply saving the whole webpage, but that may require saving the JS and i really don't want to execute JS as part of the citation.

Anything i might be missing here? Instant View is the only community managed format for scraping i've seen, i'm surprised this isn't more commonly outsourced to communities for Reader modes and whatnot

What made you decide to take notes on a paper rather than markup the pdf it's self? I have recently been reorganizing my citations as I transition to bring an independent investigator and I have been debating which method I prefer

> Papers for Mac is Mac only

As of a few years ago there is also a Windows app, and you can use it in the browser. https://papersapp.com

>Zotero can do anything EndNote can do

My biggest frustration with Zotero is that it has no real support for standards documents.

I do not even want to think of what grad school would have been like without Zotero. A shudder-inducing thought.

I believe Pages is free for Mac users. I do agree that Zotero is a an awesome product.

Yes, Pages is, but the only citation manager integrating with it is EndNote, because Apple keeps the API locked down and expects quite the payment and revenue share to open it up.

On the upside, no one with a little bit of academic cred would use Pages.

We've started doing the "final assembly" of preprints and other self-formatted things in Pages. Placing Figures is still annoying in Pages, but far, far less than in Word.

LaTeX would be nicer but....biologists.

This is totally leaving the original topic, but how do you work figure and table management? In my "smaller" papers, I have 50+ figures (dose-response curves, antibody prevalence curves, etc.) and 10+ tables. Just referencing them based on a handwritten caption, the thought makes my skin crawl.

We've gone full in and do most papers in R Studio directly. R Markdown and R Latex work OK, Zotero is integrated, and for the slower members of the team we offer pandoc conversions of their docx and html. But Pages... it does not seem to be made for anyone doing science :(

Our workflow is decidedly uncool and old-fashioned.

Scripts produce figures as PDF files. We aim to have the script produce more-or-less final figures, but sometimes the last 5-10% is easier to do manually.

We don't bother putting them into the manuscript text while editing, and the captions are usually on a separate page, as most of our target journals seem to require. The manuscript gets written in Word + Endnote, with ad-hoc figure reference like See Figure XXX_DOSE_RESPONSE_XXX that get find-replaced at the very end.

This is definitely a local optima (if that!), but works pretty well for us especially since it's a pretty diverse group: some computationally-minded folks might be happier with LaTex, but we've also got 85 year old prof, DoD folks, etc who want track changes and familiarity.

I think GP is referring to the ReadCube Papers app, which is actually also available on Windows, mobile, and as a web app:

https://www.papersapp.com

However, in light of its monthly subscription pricing, I'll be sticking with Zotero which does everything I need on a desktop computer, and also with Linux support. Zotero being free and open source gives me confidence that my data won't be encrypted in a user-hostile way like Mendeley did to its users after it was acquired by Elsevier.

Reply on news.ycombinator.com