Zotero: Free, easy-to-use tool to collect, organize, cite, and share research | Hacker News Reader

Zotero: Free, easy-to-use tool to collect, organize, cite, and share research

zotero.org

747 pointsrammy1234201 comments

Zotero is, if you're not in the market for a closed source silo like EndNote, the only game left in town. Mendeley went full on Evil, and Papers for Mac is Mac only, closed source, and missing dozens of functionalities that are absolutely a must have in academic writing and research.

An absolute must have is BetterBibTex (https://retorque.re/zotero-better-bibtex/), which adds better Citekey management and, my personal highlight, "export on add" functionality. In short, everything I save into a specific folder in my library gets exported as a .bib file right away. That way I can click a button in my browser and have the citation ready in my LaTeX editor, Word, or Obsidian (https://github.com/hans/obsidian-citation-plugin) within seconds.

The latest beta also adds full Markdown export for Notes. I tend to keep my notes on papers in Zotero, attached to the paper, but export them for filing in Obsidian (which I then feed into MkDocs for our work group's large repo).

Unless you need Apple Pages support, which Apple keeps to itself and only sells for mighty moolah to a select few, Zotero can do anything EndNote can do, is Open Source, and with that won't drive your PI up the wall with yet another expense.

I believe Pages is free for Mac users. I do agree that Zotero is a an awesome product.

I think GP is referring to the ReadCube Papers app, which is actually also available on Windows, mobile, and as a web app:

https://www.papersapp.com

However, in light of its monthly subscription pricing, I'll be sticking with Zotero which does everything I need on a desktop computer, and also with Linux support. Zotero being free and open source gives me confidence that my data won't be encrypted in a user-hostile way like Mendeley did to its users after it was acquired by Elsevier.

Yes, Pages is, but the only citation manager integrating with it is EndNote, because Apple keeps the API locked down and expects quite the payment and revenue share to open it up.

On the upside, no one with a little bit of academic cred would use Pages.

We've started doing the "final assembly" of preprints and other self-formatted things in Pages. Placing Figures is still annoying in Pages, but far, far less than in Word.

LaTeX would be nicer but....biologists.

This is totally leaving the original topic, but how do you work figure and table management? In my "smaller" papers, I have 50+ figures (dose-response curves, antibody prevalence curves, etc.) and 10+ tables. Just referencing them based on a handwritten caption, the thought makes my skin crawl.

We've gone full in and do most papers in R Studio directly. R Markdown and R Latex work OK, Zotero is integrated, and for the slower members of the team we offer pandoc conversions of their docx and html. But Pages... it does not seem to be made for anyone doing science :(

Our workflow is decidedly uncool and old-fashioned.

Scripts produce figures as PDF files. We aim to have the script produce more-or-less final figures, but sometimes the last 5-10% is easier to do manually.

We don't bother putting them into the manuscript text while editing, and the captions are usually on a separate page, as most of our target journals seem to require. The manuscript gets written in Word + Endnote, with ad-hoc figure reference like See Figure XXX_DOSE_RESPONSE_XXX that get find-replaced at the very end.

This is definitely a local optima (if that!), but works pretty well for us especially since it's a pretty diverse group: some computationally-minded folks might be happier with LaTex, but we've also got 85 year old prof, DoD folks, etc who want track changes and familiarity.

> Mendeley went full on Evil Could you explain what happened with mendeley? I've been using it for a while, and didn't notice anything different really.

They locked down the SQLite database so that you couldn't use it with 3rd party tools (such as your own commands, or to export). I am not sure if this got reversed, but at the time it was what triggered me to switch to Zotero. (Still using Zotero, which does everything I need it to.)

One thing I like a lot about Zotero is that they charge a subscription once you get over a certain amount of cloud storage, which gives me more confidence that they'll continue to exist in the future.

Yes, the DB was encrypted for no good reason except that locking users. Funnily enough at that point Mendeley could import from Zorero but Zotero could not import anymore. Thanks to Greydon Gilmore and GDB, I was able to get out: https://blog-academic.greydongilmore.com/phd_related/mendele...

Greydon's page is based on my original article: https://eighty-twenty.org/2018/06/13/mendeley-encrypted-db.

Mendeley joined the Elsevier "Family." Elsevier is one of the big three making massive amounts of money by selling content they receive for free, paid for by the tax payer or non-profits.

I strongly believe, that this has greatly contributed to the "Joe Roganization" of the pandemic, hiding true science and research, often created under the auspices of governmental research trusts, behind a paywall. Anyone not in academia (where Elsevier charges hundreds of thousands to institutions to give their researchers access to their own papers) is more or less SOL and gets their data from Joe Rogan.

In that vain, Elsevier has a tight control on the app now. Zotero has the ability to find the open access version of a paper, Elsevier will always link to the paywalled version on its own servers. Zotero has (and Mendeley had) the ability to export libraries, and with that data, Elsevier removed that when they took over Mendeley development. And, worst of all, Zotero can run standalone, Mendeley now requires an Elsevier ID login. The company actively analyzes your reading data and advertises other Elsevier properties based on your interests and reading habits.

This is academia. It should be open and accessible. Elsevier is doing everything they can to change that. And Mendeley is one weapon in their arsenal.

The Joe Rogan part makes no sense to me. Only a fraction of the listeners of the podcast would otherwise go out of their way to read primary science journal articles. Reading those is a skill and requires background knowledge to be able to read between the lines, shared context between experts etc. Scientific articles aren't written for a lay audience.

Also, sci-hub exists.

> Also, sci-hub exists.

Try adding one of those papers to Mendeley. Or, for that matter, get it by request from the author. Legal and often fast. Add that. See what Mendeley does with the CiteRef.DOI and CiteRef.URI fields...

Are we talking about Mendeley or the general Elsevier paywalls? Seems like you are complaining about several issues at once: Mendeley's restricted features, Elsevier's paywall (that can be relevant even for non Mendeley users) and JoeRoganization, by which I guess you mean that his guests don't always confirm mainstream expert consensus (The Science).

Mendeley changes certain fields on import to always download its abstract PDF as the reference paper with a link to its paywall. In essence, those things are PDF archivers and citation managers rolled into one. If I were to send you my latest research and you'd import it into Zotero, you'd get a managed citation and my paper. In Mendeley you'd get a managed citation and a downloaded abstract version with CrossRef to their paywall.

The reason this happens is Elsevier's business model. And this business model both really wrecked Mendeley and, I believe, contributes to medical misinformation being more dominant in the world.

But Joe Rogan listeners aren't using Mendeley. These issues are separate, but you're angry at both and are somehow mixing them. The paywall can be circumvented via sci-hub. And more and more papers are now on biorxiv and similar.

And I don't think medical misinfo has much to do with Elsevier or academia. It's human nature, and the quality and degree of evidence-basedness of official communications doesn't really help.

Also, the words "misinformation" and "fact check" make my skin crawl.

If your conclusion is that the plebs is so dumb because they listen to Joe Rogan instead of The Science(TM), I think you're just digging in deeper.

Also, during the pandemic, many blue check Twitter accounts said it's a pity that so many conspiracy theorists and fake news believers are reading papers and playing around with data. They should just receive The Science, spoon-fed, and accept it, and too much thinking and reading hurts them. In other words, think about what you wish for. When other people are given access to data, they may reach different conclusions than the approved respectable media expert consensus.

this isn't about joe rogan, its about someone stealing public medical data

There should only be one canonical DOI for any given article though--that's the whole point! I think the URI is also supposed to be to the "version of record" rather than wherever you happened to find a copy.

Not defending Elsevier's practices, but people who get their information about the pandemic etc. from Joe Rogan are probably not going to read (or even understand) a research paper from an academic/medical journal. The problem with misinformation isn't lack of access - there are plenty of accessible, high quality articles that are written for the layperson - the problem is that people are reading (either through choice or The Algorithm) misleading and sensationalised tabloid clickbait.

I find myself being surprised by people more often than disappointed.

The optimist is often disappointed and the pessimist often delighted.

> "Joe Roganization" of the pandemic

Not sure what that means? Is he hiding true science and research?

My best guess is, the influence of non-specialists who do research on their own, without the training to understand which papers are meaningful, which are fluff, which are wrong, and which are fraudulent.

The most informative papers are often behind a paywall, so the non-paywalled papers are biased towards these other less-informative and even wrong papers.

If someone should point to paper X, in order to correct a mistaken belief discussed in a public form (like HN), and that paper X cost $50 to read, many fewer people will read it to examine the validity of the paper in the first place - even if were correct.

While if someone points to paper Y, which supports that belief due to subtly wrong assumptions, but that paper is free to read, then many more people are likely to look at it. And without the training or experience to identify the wrong assumptions.

None of this requires our "Joe Rogan" to hide true science and research - paywalls already do that.

> Is he hiding true science and research?

For him it's probably sufficient to just use a heavily-biased selection of it and/or present a heavily-distorted version of its results. (As others in the thread have pointed out, the hiding is done efficiently enough by others, i.e. the heavily-paywalled scientific journals.)

Honestly, I am starting to think the open source movement has done as much harm as the subscription model to the scientific information ecosystem. The incentives are so high now to get volume published in pay to publish journals, leading to rushed peer reviews. Also APCs skew who can publish 'well'. The APCs for some top Cell and SpringerNature journals are about ten thousand fucking dollars. Great that the public can access it but that's money that either doesn't go to research or has to come out of taxpayer funded grants and straight into the pockets of publishers with huge margins. OA has also spawned a massive predatory journal market.

> Honestly, I am starting to think the open source movement has done as much harm as the subscription model to the scientific information ecosystem. The incentives are so high now to get volume published in pay to publish journals, leading to rushed peer reviews.

What does that have to do with Open Source?

> Also APCs skew who can publish 'well'. The APCs for some top Cell and SpringerNature journals are about ten thousand fucking dollars.

What are APCs? (Are they perhaps what relates to Open Source here?)

> Great that the public can access it but that's money that either doesn't go to research or has to come out of taxpayer funded grants and straight into the pockets of publishers with huge margins. OA has also spawned a massive predatory journal market.

But even ten thousand fucking dollars sounds a lot less than what those publishers are making now, so isn't it still much better?

oof -- that should have read 'open access' and was intended to pertain specifically to the scientific publishing ecosystem. my bad.

APCs are article processing charges. They are the fees the author (or sometimes the institution, if they have negotiated a blanket deal for all of their researchers) pay the publisher in exchange for the article being made open access.

> But even ten thousand fucking dollars sounds a lot less than what those publishers are making now, so isn't it still much better?

Publishers wouldn't be transitioning so many journal to Open Access if it was hurting their bottom line, and the for-profit academic publishing industry is about as profitable as they come with margins approaching 50%. They can do it under the guise of a principled stand. But it can create some perverse incentives. Also, previously they made money from subscription fees, so the bargaining power of whole university systems would negotiate rates. Now it for the most part falls to individual researchers, who don't have that economy of scale type bargaining power, which removes that check on inflation. A few Uni systems have cut similar deals to cover APCs for their faculty as I mentioned earlier, but it isn't yet the norm. /rant

So now the scientists get to PAY to be published?!?

Holy shit. If all this is for "peer review", it's time to scrap how they get that and develop a whole new peer review system.

(Are the Swedish Academy's science committees so superficial that they just compare citation numbers for awarding Nobel prizes? If not, what system do they use -- and why aren't they open sourcing it?)

Yeah increasingly scientists have to pay. It has become the 'principled' thing to make the publishers richer in the name of open access. Sometimes there are waivers for researchers from developing countries. Unfortunately funding bodies haven't universally caught up to the expanded $ needs. Nor have institutions.

There is a model with journals like PeerJ with relatively cheap APCs and a deal where you could sign up for a life membership and publish there indefinitely (but I think coauthors would need to as well). They also only publish based on scientific merit and aren't selective on 'impact', which is good. However, that means they aren't seen as prestigious, so you can't exactly build a career and get promotions going that route.

RE: peer review, in part bc of disruptions due to COVID journal editors have been having a harder time getting people to volunteer to peer review. There has been a lot of talk about moving to a model where reviewers get paid, but again that seems like it would set up some perverse incentives.

RE: Nobel, I don't think it is about citations, pretty sure the judge the impact of a particular body of work/discovery/contribution, qualitatively.

> RE: Nobel, I don't think it is about citations, pretty sure the judge the impact of a particular body of work/discovery/contribution, qualitatively.

Yeah, I pretty much knew (or at least assumed) that. My question was more a half-facetious way of saying academia in general should also do it that way in stead of continuing the current stupidly simplistic quest for journal citation counts.

Sure, I know nobody can afford to invest the same kind of work for every one of the millions of papers produced each year as the Nobel committee does for a few potential laureates. So it would have to be some kind of distributed system, based on the whole international scientific collective... An adaptation of the Slashdot rating system, or something? :-)

> the "Joe Roganization" of the pandemic

I think that was much more caused by media trying to prime people with some ridiculous stories that at least some refused to believe. A simple but honest approach is preferred to what we read in media otherwise.

We still have sci-hub at least...

They got bought by Elsevier who I don't trust for a moment and they are pushing some weird social media angles and the whole thing smells of proprietary vendor lock in, walled-garden etc. Zotero is much more hacker-friendly.

Hypothetically speaking: going to scihub, pirating an Elsevier paper, and then uploading proof that you had done so to an Elsevier cloud account doesn't strike me as the brightest of moves.

Not in the sense that I think you are guaranteed to see consequences, or in the sense that I've actually heard about it actually going sideways, but to my intuition it does seem like the kind of thing that might attract one of those "pay us $10k or we sue you for $10M" letters down the line.

Especially from that company.

In addition to the other responses to this question, they also got rid of their browser plugins that let you quickly add arxiv pages/pdfs/etc in one click with automatic metadata detection. Worked from any browser on Android and desktop. I've yet to find anything like it.

Mendeley for iOS system was discontinued and taken off from App Store for “no reason”. Users are forced to use their web version in iOS system.

>Zotero can do anything EndNote can do, is Open Source, and with that won't drive your PI up the wall with yet another expense.

My university offers EndNote (plus Web of Science and the Clarivate stack) for free for students and many of them don't use it - they'd rather manually manage the bibliography in their Word documents. The university even ran tutorials to show them how to set it up, add citations, and export the references, thinking maybe the students didn't understand the value of bibliography management software. Nope, even with the tutorials there was almost no uptake.

It turns out the problem was EndNote all along. I tried it myself - one of the worst user experiences imaginable, and I could totally understand why even doing references by hand is preferable. Eventually I told some of my own students that I use Zotero and to give it a go even though we're not supposed to endorse other software. Two weeks later most of them were using it.

So I'm actually quite confused at the fact that Zotero is offered as a free alternative to EndNote, unless the EndNote that I've used is some weird dated inferior version...

Did you use EN9 or EN20? I found EndNote to be usable, just not something I'd pay $130 (student/academic version) for.

It took me until the update to 20 to realise it, but it was Endnote X9 i.e. 19, not 9

Having used Zotero, Mendeley and Endnote I ended up on Endnote. I like their metadata scraping, their Word plugin and that many journals offer a pre-configured reference template. But yes as a DBMS it lacks alot in the filter/grouping features and date fields don't convert format intelligently.

First manuscript in my PhD I just hand-typed the bibliography. But when you get to a few hundred in a document and over 1000 relevant to your work that you want to keep track of, ah and when you need to resubmit manuscripts with a different reference format, and once I made the jump to actually using it and figuring out a few of the quirks, I now quite like Endnote since my institution has a site licence.

Yes. Professionals don't hand-edit references in the writing document. Update the template and click refresh. That said, I also use Endnote because it's free for me.

I quite like Endnote. I'm unsure if Zotero has a Word plugin.

In Word, alt-1 click reference, alt-2.

Its done. Reference inserted. Bibliography updated. Sweet.

Exactly my experience as a student. Picked up EndNote, hated the interface and dropped it hard. I did need something to manage my references so I ended up on Zotero which is incredibly easy to use and just works. I use it to manage personal-interest files too.

The university just needs to set an assignment which needs to be submitted twice: once with citation style A, and again with citation style B. Anyone not using citation management software will quickly start.

The point about academic research is freedom, in every possible meaning. Including not using the right tools for the job. This ensures "natural selection" of the ideas, processes and workflows that actually work.

I forget why I didn't like Zotero - I am using iLibrarian [0]. It is positioned as a SaaS, but the source is available and you can host it yourself. But I can't speak to how it compares to other software as I am not continually using it. It seems I just like collecting scientific papers rather than actually reading them.... Maybe if I live to 150 will I manage to read them all.

[0] https://i-librarian.net/

And I wholeheartedly second the fact Mendeley is Evil. I got sick of the bugs.

Oh damn. It looks extremely close to what I want (It's responsive and the PDF UI might even work on my e-reader), but it doesn't seem to support importing web pages?!?!?!?!?!?!?!?!?!?!?!?!?!?!

What a massive and disappointing oversight. Otherwise I could've seen it becoming my new research manager.

It extracts the metadata, but not the body, yes. Retrieved websites are generally frowned upon in my field (medicine, virology), and that's why I never really needed this. We're pandoc-ing our internal MD files to PDF and attach them to the relevant HTML pages, which makes this somewhat a crutch but circumvents the "9 out of ten reviewers hate websites" problem.

I print the webpage to a PDF and store it that way, then fix up the metadata as I see fit. And that is a good feature - make a request on the Github repo and see if the lead developer responds.

I use zotero and I really want to like it, but it's not that good. The tagging functionality seems like an afterthought, the bibtex support is only reasonable with betterbibtex and even with it I have issues. For example journal abbreviations don't work consistently, every start up I get told that the betterbibtex styles have changed and I need to restart zotero... File management is also only bearable with zutilo, and I still get issues with sync and somehow some file locations getting messed up.

The whole note taking experience is also quite soso.

I really wish there was a good bib manager around. I don't want to go into a proprietary silo (ruling out mendeley endnote etc.). Jabref is in many ways better then zotero when one uses bibtex, but unfortunately some parts of the UX are not my thing. The best app I have used was docear, the concept of a Mindmapping app together with a bib manager was great however it is unfortunately not maintened any longer.

+1 for Jabref and a bibtex database.

I compared some options in 2005: https://www.coli.uni-saarland.de/projects/irtg/contents/Coll... (Not that much changed, except that CiteULike was sunset.)

JabRef is a good option for a simple desktop GUI based program that is cross-platform, but I find myself writing "raw" BibTeX entries in Emacs or Sublime more and more often without any UI tools.

I agree that there are definitely better alternatives to Zotero.

please tell us what you think they are too!

Aaah JabRef, that was my choice 15+ years ago when I was doing my PhD. I think today a combination of portable JabRef and GoogleDrive/DropBox or similar would do the trick.

What made you decide to take notes on a paper rather than markup the pdf it's self? I have recently been reorganizing my citations as I transition to bring an independent investigator and I have been debating which method I prefer

Zotero and Obsidian is a killer combination.

If anyone is interested, I have some videos on research tools that include a Zotero->Zotfile/MDNotes->Obsidian workflow: https://www.youtube.com/playlist?list=PLHmevVAAXtu3_beDLtsTm....

>> Zotero and Obsidian is a killer combination.

Could you elaborate on this a bit?

I currently keep my Zettelkasten in Obsidian and my project journals in Workflowy, and I wonder if there’s a place for Zotero in my workflow.

I thought the same thing until I discovered Paperpile. I highly recommend checking it out. https://paperpile.com

I do not even want to think of what grad school would have been like without Zotero. A shudder-inducing thought.

If the town is "online reference managers," you are probably right, but I would argue that reference managers are one of those areas where you really want something offline:

- Offline ensures that you do not suffer an externally caused downtime just before a deadline

- Offline ensures that you have a path for keeping your database throughout your research career, and to do system updates when _you_ want to.

- Offline ensures that if you leave academia, you will always have access to local copies of the academic papers you have referenced.

My favorite offline/local reference manager is `jabRef` [0] which stores all metadata directly in a bibtex-file. The GUI has an excellent pdf-integration, and everything is local and super fast.

Case in point: after a decade in industry, I am looking to get back into my academic fief. All the papers I ever read are in my Dropbox, and all I had to do to pick up where I left was download a current version jabRef and point it to my database which it read without any issues.

[0]: https://www.jabref.org/

Just to be clear, Zotero is primarily a local program that saves all data and files to your computer. You don't need an account to use it. There's optional syncing and a web library so you can use it on multiple devices and collaborate with others, and there's a browser extension (which JabRef uses as the basis for their extension) that integrates it deeply with the browser, but in no way is it an "online reference manager". We strongly agree with all the reasons you give, which is exactly why we designed Zotero the way we did.

(Disclosure: Zotero dev)

I use Zotero and sync the attachments to my Nextcloud instance via WebDAV. The database is synced to the Zotero servers, because that's the only option. With a bit of care on initial setup for new machines, it works wonderfully. The real value are the PDFs, some of which are of course really hard to get by. And those are "local" in the sense of "local server", yet available to all clients.

There was a forum entry that self-hosting the database is a topic being worked on. That's my number one gripe with Zotero, it just seems weird: if using multiple clients, users have to sync to the Zotero servers, yet the storage space for attachments there is minimal (which is okay; Zotero pricing looks reasonable but of course, own storage via e.g. Nextclodu is much, much cheaper). This basically forces a hybrid solution, with bring-your-own-storage. If all of Zotero's data could be self-hosted, now that would be a streamlined, great experience.

Then again, I am anal about my entries' metadata and often correct faulty data found online. Syncing that to Zotero servers is kind of like giving back? I always thought that data might help improve Zotero's own auto-fill functionalities, which I'd like helping with.

See my comments on [1] for why we don't support self-hosting. Short version: it would pretty much just be an entirely different project.

> Syncing that to Zotero servers is kind of like giving back? I always thought that data might help improve Zotero's own auto-fill functionalities, which I'd like helping with.

No, we don't expose your data to anyone else. (And tools that try to do that usually just end up filling people's libraries with junk data.)

You should sync Zotero if you want to 1) access your library on other devices, 2) collaborate with others, and/or 3) have a real-time backup of your Zotero data.

[1] https://github.com/zotero/dataserver/issues/105

I've been syncing my full-texts via syncthing for years and years now and I've never had to think about it after initial setup.

Zotero is great! :)

Thank you for clearing this up - and apologies for not researching!

Thanks for posting this - JabRef looks perfect for helping me organise and track papers to read, I had no idea it existed!

> Papers for Mac is Mac only

As of a few years ago there is also a Windows app, and you can use it in the browser. https://papersapp.com

>Zotero can do anything EndNote can do

My biggest frustration with Zotero is that it has no real support for standards documents.

On the technical side, anyone know how their browser extension to save webpages.. works?

I'm writing a personal reader / reference tool. More of a knowledge base than a formal reference tool, but i want similar functionality, and so i've been researching ways to locally archive web pages to cite in notes. To save effort, i was trying to reuse existing methods in Reader-like libraries. The best i found so far, i think, is Telegram's library of "Instant View" parsers. A community managed set of scrape instructions basically, XPath and whatnot, that i could write an implementation for and - in theory - support everything Telegram does.

This is a fair bit of work though. I debated instead simply saving the whole webpage, but that may require saving the JS and i really don't want to execute JS as part of the citation.

Anything i might be missing here? Instant View is the only community managed format for scraping i've seen, i'm surprised this isn't more commonly outsourced to communities for Reader modes and whatnot

I highly recommend going for the beta version which features a (game-changing) built-in pdf reader[1], with annotations (searchable and synced with the beta iPad app) and many more improvements.

Zotero is an amazing tool and I could not imagine writing papers without it. For collaborative writing, it syncs the bibliography with overleaf[2] and google docs. The browser add-on allows to build your library with one click while browsing, and it parses wonderfully meta-data from all (most) publishers and pre-print servers. Then my unique bibfile (symlinked in every project) is continuously updated to reflect new items or manual changes.

But I use it even for non-publishing side of research: one can save webpages (with snapshots) and write notes with formatting around them. My personal research workflow involves note-taking with the markdown app Zettlr[2], from which I can cite my Zotero library using simply `@[Author:Title:Year]`. I can then build notes from a twitter thread and stats.stackexchange.com answers, then connect it with papers and blog posts that way, and a specific chapter of a book with my annotation on the cloud. It not always straightforward as a workflow but it works wonders.

And the team is very responsive on the forum and on twitter. If one of the dev is ready this, thank you so much for making researchers life way easier !

EDIT: I just realised I can share my libraries online. A bit ashamed because it is very messy, but in case you are wondering what a Zotero library looks like, here is the link to mine [4] (without the notes and the embedded pdf and data files because I cannot share all of them. Moreover, I don't use the folder structure anymore so its very messy).

[1] https://www.zotero.org/support/pdf_reader_preview

[2] I recommend the BetterBibTex extension for anything related to tex, it'll save you a lot of time

[3] https://zettlr.com, another great app.

[4] https://www.zotero.org/wosk/library

Ooh, cool! I'm spoiled by liquidtext, and while this isn't at the same level yet, I'm glad they're headed down this path because liquidtext recently went Scrooge McDuck with the pricing.

Thanks for pointing out the built-in PDF reader. I use the beta version but was quite busy last term, and hadn't noticed this new feature. It's handy because it lets me highlight and take notes without affecting the PDF file.

Personally, I don't like using the built-in PDF reader. Its annotations are in a proprietary format and aren't reflected in the PDF itself (which is how Zotero can keep track of them, because they're in its own format). That may work for some users/use-cases, but I like annotations to the PDF to be burned into the PDF, in case I'm using other programs as well. (I know you can export the PDF, but that's a whole other step).

Admittedly, the upside of using the built-in PDF reader is that you can have a seamless experience between desktop and mobile. But the reader itself doesn't have all the features I need.

Personally, I use Zotero + PDFExpert. The killer feature being to crop (desktop) or auto-crop (iPad) PDFs of papers, which often have an obscene amount of whitespace.

The nice thing about Zotero is that it's open enough to support both of our usage habits.

Annotating Pdfs also works fine pre-beta with an external Pdf-Viewer that is capable of hilighting and adding comments, like for example Foxit or Pdf-Annotaor. Simply double click to open the Pdf in that viewer (has to be default app for file type), annotate, save file and back in Ztero it is possible to extract the annotations into notes with the Zotfile plugin.

I am a longtime user of Zotero and a paying subscriber too. I use Zotfile extension to manage PDF files in Zotero. The best feature of this extension is bulk renaming filenames using a predefined filename format such as authors-year-article name. Most academic papers pdf files have weird names such as "1-s2.0-S0007681321001221-main.pdf". Before Zotero and Zotfile I used to manually rename them. Then I found out that Zotero extracts metadata automatically from PDF files and can use them for renaming the files. Zotfile gives you more control. Check it out: http://zotfile.com

I saw dstillman has commented here. Dan, thanks a lot for your tireless work to make Zotero the best reference manager!

Yes, it was pure joy when-in 2009- Zotero beat Reuters: A Virginia Circuit Court judge dismissed a lawsuit this morning against George Mason University’s Center for History and New Media.

Thomson Reuters Inc. had sued the university in a Virginia court in September for at least $10-million in damages, claiming that Zotero, a free software tool created by the university, made improper use of the company’s EndNote citation software.

Zotero is a plug-in for the Firefox Web browser that is designed to help scholars store and organize their online research. The program, which could convert EndNote files, had been downloaded over one million times by September.

George Mason University said in November it had not renewed a site license for EndNote, and would not make any changes to its software.

A spokesman for the university confirmed the case had been dismissed but declined to comment further. Officials at Thomson Reuters were not immediately available for comment on the dismissal. — Marc Beja https://www.chronicle.com/blogs/wiredcampus/judge-dismisses-...

Past related threads:

Zotero: Personal Research Assistant - https://news.ycombinator.com/item?id=22694891 - March 2020 (206 comments)

Mendeley encrypts users' database after Zotero provides an importer - https://news.ycombinator.com/item?id=18977461 - Jan 2019 (119 comments)

Zotero: An open-source tool to help collect, organize, cite, and share research - https://news.ycombinator.com/item?id=17606929 - July 2018 (163 comments)

EndNote maker's lawsuit over open-source Zotero dismissed - https://news.ycombinator.com/item?id=646121 - June 2009 (6 comments)

GMU sued for Zotero - https://news.ycombinator.com/item?id=319975 - Sept 2008 (3 comments)

What I (and believe many others) really want is a self-hosted Zotero server. See question has been asked:

https://forums.zotero.org/discussion/73721/zotero-self-hoste...

https://www.reddit.com/r/zotero/comments/d270kv/zotero_datas...

https://github.com/zotero/dataserver/issues/105

https://softwarerecs.stackexchange.com/questions/48335/foss-...

https://groups.google.com/g/zotero-dev/c/9FdSz0kIG_4?pli=1

Unfortunately, there is no officially supported way to do this. The official dataserver package does not have an instruction: https://github.com/zotero/dataserver

There is an unofficial package from 5 years ago https://hub.docker.com/r/facciolo/zotero_dataserver-docker

I’m in the same boat. Plus, the iPad app was not great at the time, which is very important to me (I read and annotate papers a lot on my iPad, it is an ideal device for this). I dug a bit to see if it was likely to be resolved soon, and it did not seem like it. So I find another solution, which in retrospect was a good thing since that was 5 years ago and the situation has not changed since.

FWIW I ended up using Bookends, which can use iCloud for hosting and synchronisation across devices. I wasn’t thrilled initially because it’s a fairly obscure app, but both the Mac and the iPad apps are great. The developer is very friendly and reactive, and it’s been rock solid for years now.

Is this (https://github.com/iandol/bookends-tools) the Bookends you referring to ? I don't care about citations. All I need is a PDF organizer have categories / tags and can add notes / make annotations.

https://www.sonnysoftware.com/

Great Papers alternative if one wants Mac/iOS support and relatively good (but not perfect) importing from a Papers library.

Yes, that’s the one. I had been using Papers ever since version 1 beforehand, but I had to look for an alternative after the clusterfuck that was version 3.

I used to use Mendeley, but after the evil empire bought them out and added anti-features like preventing export of the DB (or something like that, I don't remember exactly what it was anymore), I switched to Zotero. I was on the verge of exiting academic research at the time, so I don't have as much experience of Zotero, but it seems very solid. Strongly recommended!

(Occasionally I wonder about how Zotero development is going. IIRC they were planning to switch from XUL to Electron many years ago, maybe that plan got shelved or they just don't have the manpower to do it..)

Back in February someone in the forums wrote:

"Zotero won't be moving to Electron anytime soon.

We are planning to move to a more current version of the Firefox base later this year." https://forums.zotero.org/discussion/comment/376000/#Comment...

> added anti-features like preventing export of the DB (or something like that, I don't remember exactly what it was anymore)

The application's internal SQLite database was encrypted. The user-facing export functionality is still there, as are the public APIs used to interact with the data stored online. One of the engineers involved is a friend of mine. From what I understand the change to the internal DB was an over-zealous response to GDPR concerns.

I have been and still am a Zotero user. But I found it unfortunate that back in grad school many of my peers still used Mendeley. The school library even had workshops/classes in which they taught how to use the software. I found it ironic that despite paying loads of money to Elsevier for the journal subscriptions, they were advertising their software too!

In all likelihood, Elsevier were subsidising or oturight funding those classes.

TANSTAFL. Especially in consumer technology "training", which is really sales-pipeline-building, and hooking addicts for life.

You might find FOIAing the library or univesity interesting, for contractual relationships.

The best thing about zotero is that becuase it is open source, you xan customize it a lot. You can even add sci-hub addon that lets you get the pdf from sci-hub for non open source sources. So if I 'm interested in a paper for reading, citation or anytjing else I just add it using one click (using browser extension) then it is ready for reading. I can do bibrex (using the addon). I can use zotfile which give much functionality, I add iPAD sync and highlight the paper and then extract that to zotero.

Sci-hub addon of course ia not even listed or recommended by zorero community (for obvious reasons) but this mix is very good punishment for elsevier for all the horrible things they do to science. And how even their Mandeley is evil now.

I heard an article on the radio the other day (BBC) about some researcher developing meta data standards to embed bibliography data into PDFs. Basically stash the relevant refs from your bibtex (or other) database into the PDF in a machine readable way and develop open source tools to parse that and allow navigation to online or local sources. Seems like a pretty good idea. But I can't remember who it was / what it's called. Anyone?

Hi, that was Vint Cerf and me, Frode Hegland. The project we talked about is called Visual-Meta. You can read about it here: https://visual-meta.info or contact me with any questions: frode@hegland.com Thanks for listening! :-)

Sounds great! Curious to hear why you're basing the standard on BibTeX rather than BibLaTeX? Seems like a step backwards given how many more types the newer standard supports

As a grad student, Zotero is a lifesaver! Thanks to the people developing it!

I use Google Drive for my pdf storage using some Zotfile acrobatics (awesome plugin btw) to manage my actual pdfs. I plan on switching to Zotero's cloud storage to support development as soon as I can.

That being said, I wish I had a nice workflow for reading papers on my iPad like I do on PC (Windows specifically, I know there are software for the Mac/iOS ecosystem but I don't have a Macbook so...): Open Zotero -> double-click on the paper -> Bam, the pdf opens in Acrobat where I can highlight and annotate, and save to pdf directly for future use.

The closest I got to this on iPad was to use the Zotfile plugin and use the Send to Tablet feature as described here [0], and opening and annotating the pdf on iPad.

Another interesting workflow seems to be using Obsidian to store the relationships between papers using annotations and highlights from them [1]. Another way to do this would be LiquidText but I paid 20+ USD separately for both the iPad and Windows Store versions, and it asks me to pay up again so I don't know what's up with them...

Still, I hope Zotero keeps being developed as it's the best, and perhaps the only open source research management software out there.

[0]: https://www.youtube.com/watch?v=e7novaC_O_Y

[1]: https://www.youtube.com/watch?v=_Fjhad-Z61o

EDIT: Added clarification of OS I am using.

I use Zoteros cloud service and the PaperShipapp. It works really good even though the app hasn’t been updated in a while.

Did you try this https://www.zotero.org/support/ios_beta? Works great for me.

Ah, I have heard of it! Unfortunately it seems the slots are filled so I am waiting for the final version.

Hope the iOS version fixes my itch!

I write a lot of academic papers and never use this kind of software. EndNote creates uneditable references and slows down Word, Zotero had some limitations after which they charge a monthly fee, and none of the reference libraries are easy to share with external clients.

Most other medical/technical writers I know only use these tools as a “reference library.” It might be useful for someone writing a thesis, but almost all professionals avoid these because they slow things down, break, and require updates that cause issues.

I use API requests to get formatted references direct from PubMed, and use FileLocator Lite to search multiple PDFs in reference folders with RegEx. QPDF is useful for unlocking PDFs. This system has worked quite well for me and I don’t have to worry about my data getting locked into aging or unsupported software platforms.

What reference library is easy to share with external clients then? Also, what do you mean by "external clients"?

Clients, editors, etc. Internal clients might be an editorial department within a holding company that works across agencies, for example. I know I am getting downvoted, but it’s a reality that these systems have caused issues for companies like mine and have not been widely used in professional publishing, mostly due to interoperability issues.

Zipped folders of PDF files sent via e-mail.

Sharing zipped files of PDFs amongst people is quite possibly a copyright violation. Do you have distribution licenses for all the material you ar referencing?

I am curious as to how you share PDF files with others ensuring that you have distribution licenses for all of the files. Not even RightsLink from the Copyright Clearance Center automatically purchases those distribution licenses. What I do know is that other ways of sharing files do not have the same privileges of communication applied to e-mail in the law, so that is the reason why this is still the most commonly used strategy.

Only if you want to sync pdfs using the Zotero server does Zotero cost money. Anything else is completely free, including syncing your db (without pdf attachments). If you want to sync your pdfs you can point Zotero to a webDAV server, this has worked flawlessly for me.

Fair enough, but not every client is going to want to use the same operating systems, processors, be allowed to install third party software, etc. This might work in a strictly academic setting with few stakeholders, but in professional publishing, interoperability makes these systems unpopular. Downvoting that won’t change reality. I guess there is not much interest in feedback or possible routes to improvement.

You're getting down voted because you comment seems uninformed. AFAIK Zotero is available on every OS that Firefox runs on, for example. Also you talk professional publishing, what does that exactly mean? Zotero is a reference manager, so largely aimed at academic writing (because they tend to be the ones who cité and use references the most). I would be surprised if any other setting has more stakeholders, scientific articles often have lots of coauthors, from different groups and I can sayas an academic keeping track of thousands of papers by grepping through PDFs is not really feasible.

So maybe your use case does not match what a reference manager is for, but you also have not brought any informed criticism.

Let’s say you are working on a paper with 6 coauthors for a major medical journal, hypothetically. Dr. So and so in Russia, another in Japan, a third in California, and a few others. Would you mind reviewing this draft? Oh, yeah, by the way, you need to install Zotero before you can edit the references. How do you think that is going to go? How about dealing with a graphic designer in Hungary who can’t copy the text out of the document because there are weird hyperlinks that copy out with the references and don’t work with the InDesign. Oh, and the PM at DDB says the citation manager crashed and could you please resend the PDFs. I have worked with some big name journals and there are major interoperability issues that make this software undesirable for publishing in many scenarios.

I'm not really seeing it. In most of the workflows I've used, one person is responsible for maintaining most of the document including the bibliography. The other coauthors will add in notes requesting to add in a new item, etc.

Zotero is used primarily for managing the PDF library, the actual writing will happen in Word, LateX, etc. The reference list produced by Zotero is just text.

I don't see any real requirement that everyone working on a project would need to actually use Zotero, though I guess it would be handy to keep a coordinated shared library of papers.

Maybe I'm wrong and it is a very valuable tool I should be using. I just haven't seen it used in my industry over the past 2 decades. All I have heard about are major problems arising with use of this kind of software. Surely that experience has to count for something? I don't have any problem with others using it. I am just saying that when I submit a paper to a major journal, these kinds of tools generally are not used because of interoperability issues that inevitably come up when multiple people work on a project.

For what it's worth, I've seen the same sort of interoperability issues with all similar software. Author A wants "the latest copy of the Endnote file" but nobody's using Endnote; Author B insists on emailing PDFs as attachments and uses their email as their personal literature database; Author C has set up a Dropbox full of PDFs but nobody uses it; Author D has created a shared Google spreadsheet with links to each paper but it's always out of date; etc etc etc.

These aren't issues with any one piece of software, they're issues with coordination, as are the examples you mention.

Yes, I think that makes a lot of sense. Maybe the issues I am referring to are more to do with the general frustrations of putting together a paper with multiple authors and less to do with specific software.

This is why I just use Google docs and Google scholar... when i need to do the refs, I just alphabetize in a sheet. Frontiers has low barrier citation expectations. If I have to do ACM numbering I do it in word, as last step.

This is the way. Numbering should happen right before submission.

Maybe it actually is something I should try and it sounds like there have been some major improvements over the past few years. I dislike EndNote, but that doesn't mean the same issues exist with Zotero, so maybe I will give it a try. I should also define my terms better. When I talk about professionals publishing, I am referring more to corporate publishing houses, agencies, and pharma and chemical industry publishing. Academic publishing is probably the most professional type of publishing of all, so I should clarify that.

You can use something like Nextcloud to run your own webDav server, if that's something you are okay with. I haven't tried this, but I believe you can then share out files using Nextcloud via a unique link. Actually, I know the sharing works, just not sure if it's easy to do with a Zotero back-up of PDFs.

The most valuable workflow is the one you have been using and happy with. I wouldn't wish a changed workflow on anyone in this state. I would only suggest another way for someone who is unhappy and already regularly changing. Stick with what works.

Interested to hear more about this, so how do you create citations and bibliographies? How do you format the bibliographies according to each journal’s style?

It is true that there is no good tool for editing references collaboratively. Previous attempts at this have resulted in some poor sod having to redo all the citations.

A RegEx-based parsing system in JavaScript helps with the reference formatting. EndNote makes references uneditable (unless you also have a copy of EndNote) and often cannot adjust to a particular journal's idiosyncrasies...but I can do that easily with home-built software.

Zotero may be great, and I cannot really make an assessment one way or the other. All I know is that hardly anyone in my industry uses it. It's really a nonstarter. Often, people need special permission to get access to open-source software and IT departments do not look kindly on installation of these tools.

Not to mention, I can't think of one piece of software or database I was using in 2003 that I can still access and use except for plain text files. This kind of software has no longevity. For these reasons, the academic publishing industry has not really standardized the process.

After a copy of RefMan broke, following the EndNote acquisition, I remember a time when I had to reverse engineer the database so I could still access the contents. It was a big tabular file and there was a much easier way to make it work with a simple browser-based tool and a few lines of code. I made it work and showed it to my boss, who summarily rejected the solution, even though it worked much better than the original (broken) software and broken database files for hundreds of prior projects. He emphasized that he wanted a paid solution. I still don't understand the logic. These are the kinds of nightmares that I associate with any kind of proprietary software solution. Often, in publishing, we need something that will work now and will still be accessible in twenty years. That's hard to do with free software.

I see, so you wrote your own solution which you have to maintain and extend for each different citation style… pros and cons to that obviously.

Why is it so important that a citation system works in 2003 and in 2021? Papers once published become historical artefacts essentially.

I am thinking of one particular journal that had a problem with historical reference libraries under RefMan that all broke after the EndNote migration. A library of thousands of purchased articles, including pivotal trials that are commonly referenced, essentially became inaccessible after RefMan was purchased by EndNote. I believe EndNote also remotely inactivated the software and engaged in some other chicanery. That experience turned me off of reference manager software. Maybe Zotero has gotten it right this time! I don't know. I just know that few things last in tech and least of all libraries of data housed in software that may or may not be updated or supported by future systems.

In my field, we typically just use plain bibtex files. With a proper bibtex file and a decent journal latex template, the template should do the vast majority of the work towards formatting the bibliography the way the journal wants.

Typically there are some remaining warts (both due to suboptimal templates and to limitations of bibtex itself and its community/ecosystem like the address vs. location conundrum, those who have found it will know what I'm referring to), but often they are insignificant enough to just deal with them in the typesetting stage, and when they aren't they don't require much manual work, to me it's not even near the top in the ranking of inconveniences of publishing a scientific paper. And I'm not sure this kind of software would solve such nuances anyway.

Some years ago I tried several of this kind of referencing software, including Mendeley and a couple others (not Zotero because at that point it was Firefox-only IIRC, which was a red flag for me, I'm particular about using the browser I like) and to be honest I found them more trouble than they were worth, although seeing the general positivity here towards Zotero I think I'll give it a try now.

Of course, fields that use Word instead of Latex are a totally different world where probably nothing of my comment will apply.

I have tried Endnote, Mendeley, Paperpile, Sciwheel and Zotero (back when it was firefox only, which was a long time ago).

I currently use and pay for Paperpile. I found the user experience to be the best for my purposes, mainly the ease of importing articles. The word plugin is hassle free as well. They seem to be pretty competent developers eg Mendeley’s iphone app wouldn’t sync properly for literally years, whereas Paperpile’s app was good from day one.

The downside of Paperpile is lock-in to the Google ecosystem (only works on Chromium browsers, although firefox and safari support is coming). There is also no quick way to tag or categorise papers at the time of import from the web, you need to go to the app.

Sounds like Zotero is pretty good now though, might have to check it out again.

I use Zotero as a bookmarking too. It saves the files and articles offline.

Same!

Do you know if you can do full text search for saved webpages? That would be useful.

Yes, Zotero can do full-text search of PDFs, webpages, and text files. It can also sync the full-text index separately so that you can do full-text searches without keeping the files on every device.

(Disclosure: Zotero dev)

Can you do highlighting in the browser as well? Then it might be absolutely amazing.

I outlined what I'm looking for here: https://news.ycombinator.com/item?id=29776997

Do you think this might be possible with Zotero after all?

Use Hypothesis to highlight/annotate. Optionally import them into Zotero with the Hypothesis-to-Zotero tool, if that's where you want them.

I tried Zotero and equivalent softwares (mendeley) a few times but never caught.

My typical workflow involves a big gdoc that I can share with my advisor. I paste the title, year, author and number of citations (to know which articles are the most influential). And, most importantly, a link to the pdf (sometimes through sci-hub).

Then I write notes under those articles, pasting screenshots of equations that I want to find easily.

When I have to write my paper, and this is the only painpoint, I have to paste the titles to google scholar and copy the bibtex code. I might write a simple bot at some point to do it for myself.

I'm taking every advice to improve this!

Workflow:

1. copy doi and extract in into zotero (automatic looks for pdf). 2. better-bibtex (zotero add-on) will automatically generate the .bib entry. 3. tex editor automatically recognize bibtex entry. 4. write all the relevant stuff with proper formatting (ready for paper). 5. git and github for sharing.

that looks very cool!

I found https://gagarine.medium.com/use-sci-hub-with-zotero-as-a-fal... to get the pdf from sci-hub\

Thanks!

It seems Zotero still doesn't support 3rd party cloud service (Dropbox, Google Drive, iCloud etc).

I really recommend Paperpile, especially its Chrome extension. In my opionion, nothing beats Google Docs on collaborative writing.

I believe that they support WebDav.

https://www.zotero.org/support/kb/webdav_services

You also have an option to use the Zotfile extension to sync with other services.

Agree on Paperpile as some others have commented. Dead simple to use. For annotating PDFs nothing beats handwriting, so for that I use iPad 12.9” plus GoodNotes app.

Zotero is amazing. The iOS beta makes reading PDFs in your library pure bliss. I could have self-hosted a file store, but I went with the paid one instead to send the devs a little love. (And money.) They’re sooooooo awesome. Super responsive when I’ve asked questions, and the beta has been rock solid.

(That said, any devs here? Why haven’t you shipped this epic app?)

Deep appreciation for excellent, open-source software that is really polished.

I am doing research on a subject where almost all of the source material is in printed material, meaning that I have a lot of scans (photographs actually) that I want to reference. At first, I just extracted the facts, but then I discovered that I have to beter track the sources. What I would like is a process where I transcribe parts of the images, turn those text in some semantic format, describing the facts, and then have a tool to combine these facts and to resolve conflicts, because, yes, the source materials contain errors. I have thinking of developing my own tools for this, but if I could use some existing tool that would be nice.

The same people developing zotero are also developing a tool called tropy that might be partially what you're looking for: https://tropy.org/

Disclaimer: I have never used it myself, I'm just aware of it existing.

This is great news! Thanks for the link!

I like Zotero and used in in producing our latest manuscript, but in Google Docs, with 140 cites it became unwieldy and would break the sync between the in-text cites in the bibliography. So I had to save new versions of my Google Doc every time I added a cite so I always has a backup to return to in case the sync broke again.

I hope the Google Docs devs can talk with the Zotero devs to make the Zotero integration more airtight (I understand these issues don’t flare up as much in Word).

I use Zotero to manage the research with:

- the FireFox connector for ingest,

- export to bibtex, and

- the "Zoo for Zotero" app from the Play Store to pull the content down to a Samsung tablet for portable reading.

Off topic, I suppose, but this seems like the place to ask if anyone knows how to find a book's library call number (like Dewey Decimal or LCCN) from knowing the ISBN for the book.

I've played around with various MacOS apps, EndNote, and a few other programs, but I've always found the solutions awkward. Does Zotero support this functionality? All I really need is a public SQLite database of book information that contains both ISBN and Call Numbers.

I have a library of (approximately 6000) books that need spine labels. Scanning ISBN's is easy with inexpensive hardware, generating barcoded spine labels isn't hard (using python + LaTeX), and applying them in an archival fashion isn't difficult (using laser printed non-acid paper and 3M Book Tape). The main difficulty is determining accurate Dewey Decimal call numbers for each book. It takes me up to several minutes per book to find this information by hand on the internet.

I'm wondering if there is a way that works programmatically and is less expensive than subscribing to the the WorldCat Search API, a service that makes sense for larger libraries.

Just another user here, sharing some tips:

Zotfile allows automatic renaming of attachments, which includes moving the PDF file to a different folder. This means my zotero account still has plenty of free storage, while the space-consuming PDFs are stored in my university's effectively limitless cloud drive.

I use Recoll to fuzzy-search the text of the PDFs, or just pdf2text piped through grep for simpler searching.

Recoll is great, it's here: https://www.lesbonscomptes.com/recoll/

Like many others, I write docs in markdown and use pandoc and citeproc to render to Word, then libreoffice to convert to PDF since it reliably retains the formatting from a lightly-styled Word template. With citation style sheets I can be reasonably confident that citations and references will automatically take care of themselves - being tweaks for edge cases.

I use pp to pre-process the markdown, mainly to allow "includes" so I can write chapters/sections as separate files then combine them. All using a horrible render.sh script that I copy into every new project and tweak as required :)

I'd like to additionally suggest syncthing to sync the PDF folder and, as you suggested, use the free account only for the metadata.

Works like a charm and I have all my literature synced up.

"Zotfile allows automatic renaming of attachments, which includes moving the PDF file to a different folder. This means my zotero account still has plenty of free storage, while the space-consuming PDFs are stored in my university's effectively limitless cloud drive."

Wow, did not know that. Have to test this! Thank you!

I just want something where I can import stuff (most importantly web pages, pdfs, epubs), read (on my e-reader), mark things/take notes, search and turn highlights into citations.

The desktop version of Citavi and Zotero seem extremely overloaded. The web version of Citavi is the closest to this I've seen (it has a mode for highlighting), but it still absolutely sucks.

Maybe I don't get the hang of how real researchers or scientists do things, but so far what I mention above is the core of what I really need (mainly for personal research, but also the limited amount of scientific writing I've done).

I read about Zotero theoretically supporting these features, but when I last checked it out (about half a year ago) it looked antiquated, seemed to do a lot of things I don't care about and made the things I want to do so hard I was immediately disappointed and stopped bothering.

Are there any products (preferably FOSS, of course) that cover my use case? Is it possible with Zotero after all? I have tried Worldbrain Memex, but it also has such cumbersome UX it's not worth using.

As a counterpoint to your sentiment: I am really glad Zotero devs are not spending any time on polishing the UX. I loathe modern user interfaces where a lot of functionality is hidden just for the looks.

Zotero does cover your use case very well and I don't actually see your criticism or UX concerns.

For easy importing of books and papers use the magic wand icon in the tool bar. This allows you to import metadata via ISBN and DOI. Use the FF plugin for importing web pages.

Tags allow you to cluster entries together.

On the right side it is possible to connect items that have a direct connection.

Double click on Pdfs to open them in a Pdf reader that can highlight and annotate. Save the file and use the Zotfile plugin to extract the annotation into notes.

Zotfile and Zutilo are recommended plugins.

I've tried to use Zotero over the years, and I always stumble with it because it didn't have a good handling of arXiv id's. The arXiv is one of the primary way of finding papers in my field (cosmology), and I really want to support ingestion via them, looking up actual paper details from them (while retaining the arXiv id), and including them in bibtex exports. I check back from time to time, but I don't think that's massively improved, but I would be very happy if anyone else knows otherwise.

Although it's gotten a little maligned here, I've actually switched to Readcube Papers recently, it supports everything I list above, and has been pretty good for everything else. It's a closed source and subscription which isn't ideal, but in the end I'm willing to pay for something that supports my workflow. I do worry a little about getting locked in, but ultimately, provided I'm willing to ditch any annotations, I think I can just export everything as bibtex and import to something else.

Zotero has supported adding items via arXiv ID since 2018, and saving from arXiv.org since forever. There's not currently a dedicated field for arXiv IDs, but they're added to the Extra field and included in exports (and accessible however you want via Better BibTeX export).

(Disclosure: Zotero dev)

Thanks, that's good to hear! I'll give it another look next time I look at switching things up.

Isn't Zotero the company providing Wikipedia's automatic citation feature?

I'm pretty sure it says "powered by zotero"

This looks really cool, but apparently it relies on a web service to extract pdf metadata:

>* The Retrieve Metadata feature uses a Zotero web service to find item metadata. The Zotero client sends the first few pages of text from the PDF to the web service, which uses a variety of extraction algorithms and known metadata from Crossref, paired with DOI and ISBN lookups, to build a parent item for the PDF. The Zotero lookup service doesn’t require a Zotero account and doesn’t log any data about the content or results of searches.*

Does anyone know of open source alternatives which are offline friendly? I’ve tried tesseract and was impressed but the layout analysis and OCR made enough mistakes that tasks like bibliography parsing became considerably more difficult.

Is anyone using Zotero as a bookmark manager for articles to read later?

I’d love to try it for this purpose!

Yup, just install the browser addon and it works for this too.

Yup.

Using the Zotero extension, it is basically one click setup with Zotero installed. Zotero automatically downloads a copy of the webpage, and pulls any associated metadata it can find. For example, I saved this for later use: https://archive.org/details/pdfy-MgN0H1joIoDVoIC7. Zotero pulled the names of authors, publishers and everything else it could find.

I wouldn't say I'm a power user but Zotero is amazing. It loads my university citation style no hassle, has a browser plugin which works pretty good.

Ive never managed to get the Zim-Wiki plugin to work though.

All in all an excellent piece of software.

Zotero works with LibreOffice as well. It helped me tons while writing my thesis!

This is pretty timely. I was a big user of Papers 3, but I refused to "upgrade" to the Readcube Papers 4 version, as it switched to a subscription model that does nothing for me. Papers 3 just about still works on my comparatively up to date version of OS X, but it refuses to upload some flavours of pdf and hard errors more frequently than it used to.

So I have been eyeballing Zotero, and a little put off by possible migration snags. Has anyone performed this migration? What was the outcome?

Zotero is a great tool for references. I am using it for 10+ years mostly for research papers in LaTeX+bib but also for things like "Read It Later" and notes. I find it more straightforward to use than EndNote or Mendeley and Zotero works perfectly fine with my library of about 7K entries. Also it has a useful feature that lets you save your PDFs on a WebDAV server of your choice like pcloud.com or self-hosted.

Similar to other commenters I would prefer some things over how they are done by Zotero. But what holds me back from switching to another solution is clearly the community. There are enough plug-ins to make Zotero work more as I wish and the people at the forum (https://forums.zotero.org/discussions) are very helpful.

No one has mentioned iLibrarian yet. No idea how its feature set compares to Zotero or Mendeley (I haven't used them much as I found them annoying in some ways) but there is a free self hostable version as well as the SaaS product. It is sufficient for managing my archive of papers.

https://i-librarian.net/

I learned it in college and a month ago I was googling to remember this software and I found nothing. So thank you for posting this !!

Zotero saved my life writing research papers. You can get all kinds of plugins to Sublime or Google Docs for adding citations.

I use zotero for managing research and bibliographies. But it’s so weird that their web version (the only one I’ll ever use) doesn’t make it easy to share bibliographies with other people.

It’s such a weird feature that’s missing and it really important when collaborating on a paper.

It’s free and better than the alternatives I’ve tried (endnote, mendeley, Google docs)

You can edit your settings to make your web library publicly shareable :)

I don’t want to make it publicly shareable. I want to share the references with a group of collaborators.

Once I bailed on Mendeley, I considered other options, but found that most of my needs were satisfied by simply:

- a canonical naming scheme for references in my single BibTeX file - a directory of PDFs using those names

I have some for-fun little vim macros to e.g. use “open” (OS X) to open a PDF corresponding to a canonical reference name.

Paperpile is a good alternative. I don't mind paying for them, though only hitch is that they are dependent on Google Drive alone to save your papers. Not technically open source, but they have come a long way with exciting features on the roadmap.

Presumably there's no way to use Paperpile "offline"? The classic example of editing a manuscript while on a plane is something that does happen.

Before the beta version with PDF reader & annotations, the best one was Citavi (https://www.citavi.com/en) - which is still better for large projects.

Is there an easy way to "share" a URL from iOS/iPhone to Zotero? They have an iOS app in closed, full beta, with an App Store version apparently coming soon. In the meantime, are there solutions?

You can paste the URL or an identifier (e.g., DOI) into the save page [1], or even create an iOS shortcut that encodes the current URL into the 'q' parameter for that page.

Historically the solution was to use the bookmarklet [2], but you might need to disable "Prevent Cross-Site Tracking" in the Safari settings for that to work reliably.

But yes, the beta iOS app [3] supports full saving of metadata and PDFs via the Share sheet.

(Disclosure: Zotero dev)

[1] https://www.zotero.org/save

[2] https://www.zotero.org/download/bookmarklet

[3] https://www.zotero.org/support/ios_beta

What makes this different than Devonthink other than the citation management?

Zotero has a Wikipedia article about it and the other thing doesn't.

Tangentially I made a Discord server called Awesome Knowledge Management https://discord.gg/XPNeDSQE2j

Inspired by the "who wants to collaborate" post this weekend https://news.ycombinator.com/item?id=29759115

If the folks interested in Zotero and similar products would like to share their experience and also chat about any other similar knowledge management tooling please come join us!

It's good to see this is still around, I was using it in the early 2010s while working in a county health department.

Great app, but I find myself accidentally bookmarking sites via the Chrome extension with no way to undo except for manually deleting the entry in desktop.

Any consumer app would've had this implemented by now but that's the tradeoff of being FOSS.

https://github.com/zotero/zotero-connectors/issues/301

Has there been any progress on self-hostable Zotero Storage? I got kind of frustrated being forced to rely on their sync hosting and haven't used it in years.

Edit: it looks like they support WebDAV, which I now have set up but didn't in the past. Maybe I can try it again using that protocol.

I miss citeulike. I used to pay for their upgraded service.

We tried to migrate a reference library with more than 17k entries, but it never worked out beyond 2k.

Sorry to say: might be fine for small personal projects, for anything serious I don't see how.

What problems did you encounter with this large library?

The import via API failed too often.

What were you trying to migrate from? And what do you use now?

Something's not right there. People use Zotero with libraries with tens of thousands of items all the time. (Some people have hundreds of thousands of items, though that gets dicier.) Not sure exactly what you were seeing, but we'd be happy to debug in the Zotero Forums if you wanted to try again. It's possible to configure a third-party plugin such that it affects performance, for example.

(Disclosure: Zotero dev)

Isn't zotero backed by sqlite db? So these row counts are nothing much to cope with, sqlite would do just fine.

Do you mean it did not work out for you performance-wise or the export process failed?

I’ve been using Zotero since it was in beta in 2007, and have a library with tens of thousands of items. As dstillman says that’s hardly unusual. It works perfectly (it’s just a SQLite db with a mature, well-designed schema after all), and syncs perfectly across five devices.

Funny, we tried probably your solution for imports? urschrei/pyzotero is yours? Failed all the time. To be fair we got a lot of 5xx, so maybe we were just out of luck. Anyway time ran out with that project.

I can imagine many reasons why attempting to create 17k items via the web API without some kind of batch processing and/or rate limiting may not have been a successful endeavour.

Pretty cool

Again?

Installing this in Ubuntu is a PITA and I always just give up. No I'm not going to start """symlink this and that""" when my aim is to save references for myself.

I found Zotero very confusing to use. But maybe I am not the target demographic. I have always just used vim txt files for notes, using cli tools like grep to find what I need.

It is very good for also downloading and keeping track of PDFs. It is primarily targeted at academics. And for many of us, it’s a great tool for keeping track of what papers we are currently reading, or want to keep for later. It can be hard to manage citations, PDFs, etc… in a text document. I know some people do, but I like being able to use a bookmarklet/extension to add a new paper to my lists.

To add to the other comment, Zotero can also download the PDF for you if you just have a citation (if it can find it) and it also integrates into Word, Google Docs, etc. for citations when writing research papers.

It's especially useful for multiple people writing the same research paper as the library can be shared.

Generating the citation text in whatever format people like is a time saver.

More like a life saver. It is hard to imagine the time it takes to fix references in a review article, never mind a PhD thesis, even with a proper bibliography tool. A bit less so with LaTeX, but still very useful.

It sounds like you might not be the target demo. Zotero is mainly for generating bibiliographies not note taking. It has notes in it but it is pretty basic.

I love and I use exclusively Linux but sometimes I despair:

> How do I install Zotero?

Mac Open the .dmg you downloaded and drag Zotero to the Applications folder. You can then run Zotero from Spotlight, Launchpad, or the Applications folder and add it to your Dock like any other program.

After installing Zotero, you can eject and delete the .dmg file.

Windows Run the setup program you downloaded.

Linux Download the tarball, extract the contents and run zotero from that directory to start Zotero.

For Ubuntu, the tarball includes a .desktop file that can be used to add Zotero to the launcher. Move the extracted directory to a location of your choice (e.g., /opt/zotero), run the set_launcher_icon script from a terminal to update the .desktop file for that location, and symlink zotero.desktop into ~/.local/share/applications/ (e.g., ln -s /opt/zotero/zotero.desktop ~/.local/share/applications/zotero.desktop). Zotero should then appear either in your launcher or in the applications list when click the grid icon (“Show Applications”), from which you can drag it to the launcher.

Ehhh, I am not doing all of that.

Wait, there is hope, it also says:

> Debian/Ubuntu-based Distros > A longtime community member maintains zotero-deb, a lightweight wrapper for > the official tarball.

Let's go there!

"PSA

I'm in the process of transferring the hosting of these packages to the Zoero organisation. Until that is done, the following options are available:

download from this repo (re)install using curl -sL https://github.com/retorquere/zotero-deb/releases/download/a... | sudo bash caveat: github has made recent changes to how they're hosting release files, which triggered a long-standing bug in apt. If you hit this problem, see this thread for a workaround. download from sourceforge (re) install using curl -sL https://downloads.sourceforge.net/project/zotero-deb/install... | sudo bash caveat: sourceforge uses a mirror system that updates haphazardly and which may redirect you to a mirror that is down. If you get errors, try again in a few hours."

Seconded, some effort by the core team towards packaging Zotero in a deb would be very welcome.

RPM would be nice, too. But maybe that's too much to ask for...

Alternatively, what about AppImage? At least that will run "everywhere".

So will flatpak if you're using that on your system (which most distros make very easy to use)

There's a flatpak version available:

https://flathub.org/apps/details/org.zotero.Zotero

Reply on news.ycombinator.com