Zotero: Personal Research Assistant

The book 'how to write smart notes' is a great writeup un that topic.

This post is quite good aswell: https://www.lesswrong.com/posts/NfdHG6oHBJ8Qxc26s/the-zettel...

I implemented the zettelkastem system with vimwiki for myself and I think it's great for that if you are into vim.

Yes that was kind of vague! Zotero has an interface for taking simple text notes[1] that can either be stand-alone or attached to bibliographic entries like books or articles. In my Zotero library I have folders for different media like journal articles, conference proceedings, books, proposals, etc but don't organize them by subject. So for my literature notes I just attach them directly to the bibliographic entry. I usually try to keep these short and just write down interesting ideas or thoughts rather than summarizing the content (if they're short I'm more likely to actually write them!). As a separate folder in my Zotero library I just have one big pile of stand alone notes that I like to call my "memex" (analogous to Luhmann's Zettelkasten). By flat folder, I just meant that I don't attempt to organize the notes into a subject hierarchy (e.g., "galaxies / spiral / high-redshift"). The stand-alone notes can then be linked to each other with the "related items" feature (to other notes and relevant journal articles mostly). Luhmann's main insight I think was in not trying to impose a universal hierarchy on his writings but to organically grow a system of inter-connected thoughts through these item-to-item links. The mental act of identifying the most important links helps cultivate original thoughts. Needing to manually walk the notes puts you in a sort of conversation with yourself and means you engage with your past writings and ideas. Zotero isn't perfect for this, but the system is very simple, and can be done with paper cards, a single text file, a wiki, or special purpose software. But since my most frequent work flow is "thoughts related to journal articles", Zotero's bibliographic functions and the fact that that's where the PDF files are, make it much simpler to use for me than a stand alone tool.

[1] https://www.zotero.org/support/notes

First thought for easy digitization of hand written notes is an iPad with pencil by your workstation. A BKM for integrating that into your Zotero workflow isn’t obvious though.

For what it's worth, Pinboard also has an API that you can use to manage your data, including making backups.

I use it for exactly that, especially since he's quite upfront (and funny) about how hooped everyone could be if he's ever hit by a bus.

Just wrote a short summary of my method to a friend. Might not be terribly readable for people who haven't used Roam, so take it for what it's worth. Hope it's for some use :-)

What I do is an interaction between the following:

Article <-> Tags <-> Hypotheses <-> Synthesis <-> Prediction <-> Research

So I might read an article, then tag the article/figures with the relevant tags. If it's for a specific project, I keep a list of tags on a separate page. This ensure that I use the tags consistently, so that I can find all the relevant articles later.

I purposefully keep the tags general, like "psychiatric conditions" rather than "depression, mania, schizophrenia" etc. When I reach the hypotheses page, this means that I can form relatively general hypotheses, and then nest them into gradually more specific hypothesis that are specifically supported by the articles.

Then, I use those hypotheses to form new predictions. These predictions turn into new research questions, which I either explore via the literature or doing my own experiments.

An example of this might be: "Wood 2018" <-> #Procedural Memory #Habits <-> Habits are part of procedural memory [[H]] <-> Habits are part of procedural memory [[H]] & Procedural memory is stored in the basal ganglia [[H]] <-> Damage to the basal ganglia disrupts habits [[H]] <-> Seger 2011

This affords me a lot of flexibility in total processing time; I don't spend a lot of time forming hypotheses that aren't interesting/necessary for me, I can quickly tag a lot of sources, and I can quickly collate those sources on a specific topic when necessary.

You might say that the first half of the workflow is "induction", the second half "deduction and confirmation".

I would love it if you had any questions or comments, but just writing this up has been useful for me as well.

It also plays well with Latex :) There's a BibTeX output style that works really well.

Zotfile + Better Bibtex is the ticket for me

Pairs well with a USB barcode scanner.

Zotero is/was a XUL application, and the whole application used to (also) be a Firefox plugin. So the similarities are not unexpected.

Firefox's bookmark and history manager is atrocious x) Reminds me of IE5 or something like that.

I like the Firefox bookmarks manager a lot, especially the tags you can apply to a bookmark then search on.

If I get good enough with Lisp, I want to write an importer that takes all the tags from my bookmarks sqlite db, write them into org-mode files with each tagged bookmark listed in the relevant file.

The first sentence should read “personal data”, but you’re right, the copy is not great. To be fair though, you compiled bits of sentences from completely different sections to make it look more contradictory than it is.

To be clear, Citationsy has no tracking and collects as little personal data as possible.

When you cite something we keep your citation data on our servers, of course. You can download all your citation data at any time in various open formats (BibTeX, CSL-JSON, etc).

I am indeed using that. The bug reports claim it is a Zotero problem and not a better bibtex problem.

Thanks for the response. I am aware of that functionality. I, however, usually find papers using the browser (either searching google scholar or on arxiv/scirate). Zotero, then lets me click one button (most of the time), and the entry is available to be cited using a nice citekey (firstauthorYear), and the pdf is placed in my pdf folder with the right filename.

With all due respect to the excellent work you do (many thanks for that), my workflow does not require any manual work most of the time.

OP's criticism were fair enough to not deserve a dismissive response.

I for one am thankful for their warnings.

Thanks for your work! It's nice to hear that Zotero seems to have privacy as a feature and not a side thought.

Zotero's privacy features even extend to where data is stored: you can bring your own WebDAV server [1] and have Zotero store data there.

[1] https://www.zotero.org/support/sync#webdav

...and you don't automatically upload pirated scihub papers to Elsevier cloud storage, like Mendeley does. That feature alone makes you a winner on the privacy front as far as I'm concerned!

Thank you for your work.

> designed as a local tool

Nod, a local tool. I have various expectations of my local tools. And if I, say, start Zotero in the morning to read a paper, then exit it for a meeting, then return to it afterward, and then exit for lunch, then at least my own expectations for a local tool are, for example, in tension with those four centralized timestamps. As are the varying tcp routes as I move my laptop among buildings. As is the request when I surf to the NYTimes during lunch.

So what does privacy best practice look like? One comment here suggests the ability to fork and edit the code. Another notes the linked documentation, and being more ethical than Elsevier. The linked page notes the existence of scattered opt-out options. And also "You can avoid these requests by keeping Zotero open while you browse the web."

My own understanding of privacy best practices, includes data exposure being opt-in rather than opt-out, and those privacy preferences being easily seen and changed in one place. My impression is Zotero doesn't do these.

And that's just Microsoft-style privacy practice. It would be even nicer to have knobs, like "check for updates every <start/day/week/...>".

> Criticizing Zotero for privacy, of all things, is a bit bizarre.

I'd be fine with "we have limited resources; know privacy is important; are improving; know we have work to do to implement best practices, are working towards it".

But my own fuzzy long-term impression has been, that such recognition has not been proportional to the potential degree of privacy exposure.

Love Zotero. I'd love to install it on my own server instance, although, I think by renting the Zotero storage space now, I'm helping support you and the whole Zotero project.

Specifically switched over to Zotero because of the non-profit status. The only privacy feature request I'd make is to allow some kind of self-hosted sync, i.e. a deployable TLS sync server + preferences entry to specify a sync server ip/port. I imagine it would take load off you guys for syncing, and people would end up hosting sync servers for small groups on university networks.

This conversation is now days old, and people have moved on, but here's one quick belated thought. No reply needed, just fyi, fwiw. I was at an embassy party years ago, and apparently a friendly European embassy, in a US city, had a couple of people keeping track of local research, and doing heads-up for their national industry and academia. That's just one embassy, in one city. I suggest Zotero has very different threat profiles across different fields and research topics. And for some, the possibility of state actors should be included. Which suggests a need for users to easy notice and adjust their exposure profiles. And is a reminder that the user data Zotero servers are least likely to compromise, is data that's never seen at all. Fwiw. Thanks again for your work.

Indeed. There does seem an issue of what baseline to use. Elsevier isn't usually thought of as a useful ethical baseline, but here it's a displaced competitor. Companies embed a variety of telemetry. And while much open source doesn't, some does, and this is perhaps increasing. And yet, if say Firefox always maintained an open but empty connection to mozilla, would it be adequate to suggest "well, no, that's not under privacy preferences, but it's documented on our privacy web page, and can be disabled by editing about.config.mumble"? Perhaps with the shift from desktop to phone, expectations of what it means to be local are changing? "Your core data is local, not hostage", but now "of course the app chats on the web... doesn't everything?"?

I do this exact same thing! If I write a new Zettel that references something I read in a book (for example), all I do is get the ISBN of the book (usually off of Amazon), use Zotero's auto-import feature, copy the reference and paste it into my note. The whole thing takes all of ~20 seconds, it's very efficient.

I know Zotero has a lot of other features, but 99% of my workflow with it is what I described above.

What I do is just annotate absolutely everything. I try not to think much about structure. What matters is that my thoughts are put down into text. Structure and organization comes later.

Once your second brain is searchable, you can quickly find those keywords and the context thanks to Roam bidirectional links. This is one way to approach your notes, when you know what you're looking for.

Another way to approach your notes is by browsing. You can create a page with the structure or outline that you come up with, and then fill them with links to blocks in other notes. In here, you can practice a step of progressive summarization and rephrase your notes.

I'm actually trying to free myself from the habit of organizing/structuring/planning. It gets in the way. Still obviously experimenting.

I'll usually just start putting things into Daily Notes, but if a connected set of notes gets too long, I'll make a separate page and add a [[]] link in the Daily Note.

Roam is way of using a wiki. i.e. you could quite easily set up a wiki to act in much the same way as Roam.

The journal part of Roam, the page-per-day part, is very useful. You write down what you're doing each day, use tags and links to build up a second brain, so to speak.

Let's say, today I'm working on kubernetes. I add a tag for k8s, which is a link to the k8s page, where I have all sorts of interesting links and notes dumped. When I'm on the k8s page, I also get a list of pages that link to the k8s page, which helps me to remember some other subject that might be relevant.

It uses the Zettelkasten Method: https://zettelkasten.de/posts/overview/ and you can also get a lot of value from reading "How to Take Smart Notes" by Sönke Ahrens.

The value also comes from it mirroring your thoughts and experiences, as I said, the second brain thing.

(I'm terrible at explaining stuff, sorry :( I hope I didn't bore you to bits)

Automatic backlinks + all blocks can be referenced or embedded. It allows you to remix your thoughts effortlessly. It's like an idea blender.

See my comment reply to Chris2048 below your comment.

It's a wiki like system that helps you to create a second brain of sorts. You write down what you do, and link between lots of different pages, such as "aws" to "ec2" to "vm" to "vmware" to "vmotion". Roam (and org-roam, an Emacs module that I use) shows you what links back to a page, which is incredibly helpful for remembering knowledge and how things fit together.

Does the new version handle citations/bibtex export? I really want to use polar but that’s the main value of zotero for me atm.

Would it be possible to export annotations into a markdown+wikilink syntax to import into Roam?

I love the idea of Polar for document management, and Roam for knowledge management, so I'd love to find a way to use both

Would love to see an option to choose which pdf viewer Polar launches. Wouldn't mind losing out on some features (for lack of integration) as long as I can use my Evince.

Can Polar automaticlaly extract the metadata from a journal article?

Also, while I can see tags being useful, I don't think it'll really be Zettelkasten-like until you can link from one annotation to another.

Consider adding a link to getpolarized.io on your youtube video descriptions. I opened your video, tabbed away, came back, watched it, got excited, and struggled to find your URL (which I happened to open in another tab from this comment :P)

I'll be giving polarized a look this weekend and compare with zotero!

They are planning to charge $12-15 USD with an annual plan.

I used Mendeley to keep track of papers during my PhD. It was really a fantastic piece of software.

Too bad they sold out to the dark side. After they were acquired, I switched to Zotero, but I wasn't doing active research anymore by then, so I don't have as much experience of using Zotero. On the surface it looks pretty good though.

I agree that Elsevier is toxic, and wish they didn't own Mendeley. Unfortunately, at least the last time I compared them, Mendeley was way better than Zotero for my personal use and just couldn't use Zotero. The cloud sync and sharing, the lit search, and all were just so much more compelling. I do think its been long enough I might poke at Zotero again and see if its caught up.

If you were on the iPhone mac side, papers had a brilliant solution that was appealing

I use zotero and store the pdf files (renamed and organized with ZotFile) on OneDrive. I can read papers and make highlights/notes to pdfs and they will sync to all of my devices. These notes can be extracted and made searchable within Zotero by Zotfile.

At first I preferred the way that Mendeley did this with its built in pdf reader, but now I'm happy Zotero delegates to the pdf software of your choice.

Mendeley and Zotero are both free, but both have storage limits beyond which you have to pay (Zotero is 300M, Mendeley is 2G). If you only store references and annotations you'll likely never exceed those limits, but if you use them as a paper archive you may. (I have a 3G archive)

Does it have any kind of export functionality at all?

Yes you are, thank you very much!

Can you please link to your plugin?

You are amazing and have saved me so much time. Thank you so much.

That works if it's the author itself, and the author has at least a passing familiarity with BibTeX. Much of the bibtex that you can download from journal cites is itself pretty botched up.

I believe most of them are the same, but with a different name. So, for my convenience, I don't need to think what citation style a journal uses, I just go and download the citation style with its name, even if I have the same style installed for another journal.

I would guess a lot of those are duplicates but there are also cases where it has like APA 4 styles and one for APA5, and APA6, and so on.

That's what I do, but it took a while to get there.

Since I would say 80% of people have some level of cloud storage now (University, employer, personal), there should be a simple setting to pick one, and the location. Make it easy for people and win users!

great! good to have an alternative to qiqqa in case it disappears, although it looks like qiqqa has killed their cloud libraries and gone open source. I found qiqqa after google desktop was EOLed.

I have been quick to recommend Qiqqa whenever it seems like it might be a useful tool but I never really saw those comments get much traction - maybe because everyone was already using zotero?

Yes - try diigo.com. All my links + references lives there.

I'd be interested to learn more about Zotero's history and whole space. Do you mind if I reach out to you with more questions? Email's on my profile :)