Ditching Zotero for a Text File
atthis.link
atthis.link
Also, as my unenlightened colleagues insist on using ridiculous tools like MS Word to author papers in, it's useful to have Zotero installed for the ability to link a BibTeX file to a WYSIWYG word processor when collaborating. Zotero helps you play well with others.
I don't think Zotero is perfect by any stretch, but I'm glad that at least it allowed academia to reject both Endnote and Mendeley. Anything that punches Elsevier in the face gets a tick in my book.
Zotero integrates well with various online services. This I think is Zotero's most valuable feature. The simple fact is that not everything exports BibTeX. Can you get a library catalog to export BibTeX? Perhaps in CS, most publications provide good BibTeX, but a lot of journals I (a mechanical engineer) deal with don't provide BibTeX at all to my knowledge. But I can simply press a button in Firefox and import the bibliographic data embedded in the web page into Zotero. (No, Google Scholar is not a good solution here because it's frequently inaccurate and incomplete.)
I'm a technical guy who can handle BibTeX fine, but I still prefer the Zotero UI over using BibTeX files in a text editor and shell, even if I'm only generating BibTeX files.
The author discusses using non-standard fields like keywords to store extra data. I would recommend that to store additional context about the document. Zotero can store even more than that, including web pages, files, and additional notes. I like that Zotero saves a snapshot of the journal article web page in most instances. Journals sometimes do go offline, so it can be nice to have the web page. I often have detailed notes about particular documents in notes in Zotero. Yes, you can do that with BibTeX, but I could see the extra fields cluttering the BibTeX file. (Contrary to what the author states, the note field probably should not be used as they describe because it's printed in some/most? bibliography styles.)
The search in Zotero is more powerful than grep. Try returning bibliographic entries where one field contains X and another field contains Y. I think you can probably come up with a grep solution for that, but it's so convoluted that it's probably never used in practice. You could use a BibTeX searching program like biblook to get around this problem.
Don't get me wrong. Zotero is far from perfect. It can be slow, and at this point I would prefer a TUI reference manager. But overall, it's the best option I've tried.
If this kind of feature could be replaced by txt files I probably would be using it, but, no...
I think this has an unnecessary risk of hallucinated bibliographic data. For anyone doing something similar in the future, it would be more reliable to make a LLM generate a list of DOIs and have Zotero import the DOIs.
It's not my cup of vodka, to be sure.
Each one can choose their own tools of course, but we often see llms shown as solutions to something presented as unsolvable before, while it already was pretty amenable to normal tools.
I see that as a symptom of being intellectually lazy, or not knowing the value in mastering a tool via deliberate use over time.
Once one invests the required time to use a tool properly, they can reap way more benefits from said tool when compared to jumping tools at minute's notice.
As mentoned somewhere else, zotero's metadata retrieval is like 50% hit/miss in my experience - as in for the papers i've imported it was able to ID about half of them. An the quality of that data is not that great either. Lots of OCR errors, sparse bibtext fields, author names wrong, etc.
I have tried filling some missing fields in my books list with help of an LLM, and none of the books were rare. I allowed it to search the internet and it hallucinated so bad that I nuked that version of the list and reverted all the changes.
It's more accurate with all the holes in the data.
Really? This does not reflect my experience at all. I've been using the Beaver plugin and also Claude Code, and they have been spot on.
I'm a graphics programmer, i keep a lot of journals for research, and i needed the least painless method to get them sorted. Even if, say 1% is misclassified or has wrong metadata, it will not impact what I do the least bit. And those can be corrected with ease.
Like another poster here, trusting an LLM with my reference database -- the ultimate source of truth -- is not a step I'd be willing to take. All it takes is a single hallucinated reference, and your career would be forever tainted. It's not worth the risk.
Sadly, Zotero seems to have removed this killer import feature in later versions, which is the reason I keep using version 6. It feels like later versions have been a route to dumbing-down the interface, prioritising simplicity (and an ultra-low-contrast interface) at the expense of functionality. (If you can still drag-n-drop PDFs straight in with the new versions, someone please let me know?)
I'm on version 9. I did the drag and drop for some of the files, but Zotero has only like 50% hit rate with finding the correct bibtex, and those are not that high quality either. People talk about hallucinations using LLMs, but I found them to be more reliable and produces richer metadata than Zotero's system. You can even use the Beaver plugin to correct the metadata, if you find them dubious or low quality.
>All it takes is a single hallucinated reference, and your career would be forever tainted.
Of any references I use, I always read the PDFs. The entire point of me using Zotero is to manage PDFs and being able to read them. If the title and author data in the item view don't match the PDFs then it's easy to notice. And even if i get the reference wrong, i'm not worried about my career, because it wouldn't matter.
I like me some cli, but if I was to do something similar it would be with csl-json and jq/nushell. I cannot see the advantage of grepping a bibtex file, let alone having to manually manage the related articles.
A friend and I have been working (slowly, over the past few years, since we are both academics) on an abstraction layer over BibTeX called Autobib: https://github.com/autobib/autobib
Broadly speaking, Autobib is a CLI around SQLite database of BibTeX records. But in addition to plain BibTeX, Autobib is aware of 'external data providers' (like DOI, MathSciNet, OpenLibrary, arXiv, zbMath, etc.) and will automatically retrieve data bibliographic data from these data providers. The provenance of the data is stored alongside the record itself, and this can be used to retrieve updates, prevent duplication of data, etc.
The killer feature is: if you have a file (say `file.tex`) and the keys are in a format which Autobib can automatically recognize (say, you use citation keys like `doi:my/weird/doi`, and there is support for custom formats and aliases) you can run `autobib source file.tex` and it will write to standard output a sorted BibTeX bibliography for your file. This lets you trivially maintain a per-project bibliography which you can check into source control locally and which exactly corresponds to the paper itself.
But otherwise, Autobib is "just a wrapper over a BibTeX bibliography"! When you edit an existing record, you are just editing a BibTeX record. There is integrated search, directly on the BibTeX fields themselves.
There are some extra features, like support for attachments, fuzzy search, undo-tree support, headless edit, auto-normalization, soft deletion, replacement, merging, etc. The database format is relatively simple and open (currently not particularly well documented, but this will change when it stabilizes) to allow introspection by other tools.
The tool also strives to play nicely with other tools (structured output, composable, etc.)
These frontends are necessary however because researchers in non-computer related fields are not trained/proficient at command line tools. Many of them need help installing software. Many of them don’t use raw text files much either. They use MS word files instead etc.
The reason I like Zotero is because I can use it offline, have my all documents synced (the backup feature is one of the best things) when needed and I can organize my papers according to project/paper I'm working on. Actually one thing I do wish Zotero would add is tab groups which will allow me have papers open but grouped by project. It's based on Firefox which does support this so I hope it's a matter of time.
What a primitive way of searching: no real-time feedback, no way to easily correct a typo based on such feedback, no way to act on the results, e.g. edit the found reference right there in the list of results without opening a file (and doing the search again to find the spot?)
The rest of the interface is ok.