Bootstrap a Research and Writing Workflow

I’m pretty happy with my research and writing setup. It works well enough for me to make recommendations to people. Here, I try to write-up my current workflow and the tools involved. Besides, the Notetaking for Historians has a much better and thorough tutorial on all of this.

Introduction

In research, you always collect, manage, and consume material such as scientific papers, other textual material, images and videos etc. Guided by your inquiry, research question or project, you produce something new out of the collected and worked on material and your own thoughts. In the last stage, you want to hand your new thing over to other people.

I have three tools I need regularly: Zotero, Obsidian, and Tropy

All of them are free. Zotero and Obsidian offer paid services that you generally don’t need, except if you want to make your life easier or become a power user. Zotero, Tropy and hypothes.is are even open-source. Something I always like. Open-source software enables data sustainability.

In Obsidian you’ll work with markdown files, which are basically just slightly structured plain text files that you have on your own computer. Zotero works with a database that you can sync to an online service. It also has fantastic export abilities. Tropy is an application to collect and manage images that you need for your research.

There is only simple rule in place, that pretty much defines what tools I’m using:

Own your data!

Data that you are producing needs to be valued as work in the current capitalist environment. So if you didn’t get paid for it, you still (have to) own it. It also makes it easier to shift tools and setups if you own your data. Owning data means being freely able to read, move and copy, transform, or delete it.

Overview: Tool and workflow outline

Basic Setup Tools

Please use Firefox as your browser… You could use any other browser, but why would you want that?

• Download and install Firefox
• Download and install Zotero
• Download and install Obsidian

Basic Setup Plugins

Plugins enable the communication between Firefox, Obsidian and Zotero.

• Firefox
  • Zotero Connector for Firefox. This enables you to safe all kinds of material from the web, from websites, to papers, to videos as a reference in Zotero. I said reference, but we’ll talk about that later.
  • Install the hypothesis bookmarklet
• Obsidian
  • Zotero Integration: Import and use references that you collected in Zotero as well as automatically import your reading notes from Zotero.
  • pandoc: This is a plugin that makes Obsidian able to talk to pandoc. Export your writings, including well formatted references, into documents of your choice, including PDFs or Word files
  • Footnotes: Since footnotes in Markdown are a bit tedious to produce, this plugin comes handy, but it messes up a bit.
  • Tidy Footnotes: Cleans the footnote mess…
• Zotero
  • Install the ZotFile plugin installed. It’s helps to work with the PDF files in such basic things as automatic renaming, so all your files have the same naming convention (yes, that is important, you don’t want to have a mess, do you?).
  • Also install the Better BibTeX plugin. This thingy enables Zotero to produce proper citation keys and make wonderful exports of your collections. That will be important later.

Explanations of the setup

Firefox

With the Zotero and hypothes.is plugins in place, you can easily start to collect websites, references and PDFs or annotate online articles.

Zotero

In Zotero you collect references to things you want to keep, read, work through, cite in your texts, etc. The references can either just point to another place, like a bookmark, but they can also contain attachments. These attachments are usually either a locally saved snapshot of a website, in case it ever goes offline, but you still need the content, or in most cases, a PDF of the text you want to read.

When you do academic writing, it is best practice to reference your sources that you built upon. As is typical with an academic process, this can be tedious and painful. Zotero tries to take the pain out of it. You can have a folder with all your sources, for example, and with a few clicks have a proper formatted list of sources that you can paste into your document.

Zotero is also a wonderful tool to read and annotate PDFs. Have a look at the PDF Reader and Note Editor guide. It syncs with mobile, so you could also read on your phone or tablet. I usually make all reading notes in Zotero, even if I don’t have a PDF. I simply add the book as a reference (for example via worldcat.org) and attach a reading note to it.

Obsidian

This is where the magic happens. Alternatively, you could have a look at Zettlr. Compared to Obsidian, it is completely open-source, and the developer is super sympathetic. But it’s too sluggish for me and the plugin ecosystem is practically non-existent, making it hard to extend. Obsidian is basically a text-editor with superpowers. One of the basic ideas is, that you can write in something and easily link to another file, over time building up a strong knowledge network.

The goal is actually to get as much material as possible into Obsidian. This is where you spent most of your productive time. Going over your notes and annotations, your resources, piecing them all together and at last, come up with your own material. How exactly you work in Obsidian is up to you and a whole other rabbit whole I fell in several times…

Import notes and annotations…

…from Zotero

There is a good explanation by Notetaking for Historians. I have my own import template, that focuses on the notes I made during reading:

---
title: "{{title}}"
aliases:
    - "{{title}}"
authors: "{{authors}}"
year: {{date | format("YYYY")}}
types:
  - How-To
tags: [{% for t in tags %}"{{t.tag}}"{% if not loop.last %}, {% endif %}{% endfor %}]
---
# {{title}}

- [[#Bibliography]]
- [[#Abstract]]
- [[#Notes]]

## Bibliography
{{bibliography}}

{% if abstractNote %}## Abstract
{{abstractNote}}{% endif %}

## Notes
{% persist "notes" %}
{% endpersist %}

{% if markdownNotes %}{{markdownNotes}}{% endif %}