BBE W1: Personal Notetaking Desiderata Walkthrough
Personal Notetaking Desiderata Walkthrough
(Build a Better Exobrain Week 1, Project 1)
A key step I took in becoming a better notetaker was listing out what I was looking for. That’s what I’m going to try to walk readers through today. So take out your current favorite bullet-point software, and let’s get started.
Brief Explanation of Desiderata
For people who haven’t run into the term before, desiderata (singular desideratum) translates pretty literally to “something desired.” It usually comes up in a mathematical context*, where desiderata are typically presented as a list of propositions feeding into a search for the system that best satisfies them.
A good solution or system would keep as many of these propositions true as it can.
(Possibly with different weightings; the truth of some propositions may be more important than others.)
As a simplified illustrative example, take this SMBC comic on theodicy. A theologian is searching for a system that satisfies 3 propositions regarding God:
God is Omnipotent
God is Omniscient
God is Omnibenevolent
These propositions would be his desiderata.
The recognition that you might not find a single system that completely satisfies all desiderata (see again, the SMBC comic) is inherent in the framing. It’s part of why you might call something “desiderata” instead of “requirements.”
Prompts
Here are some optional prompts, as a starting point.
What Situations are you Trying to Solve?
If it’s mostly for tracking due-dates, maybe pick a calendar.
If it’s memorization, consider a flashcard system.
If it’s for taking sequential notes on a lot of reading, consider an annotator.
Would you benefit from being able to reference other notes in a non-hierarchical fashion? Or is a strict tree more than good enough? Are you going to sort by topical tags? Do you need a search bar for words?
etc.
What Type of Note Taking?
Software
If this question stumps you, you might want to check out the partner post’s description of categories of notetaking software.
Some categories:
Gantt Chart (& other multi-threaded)
Personal wiki
Level of standardization?
Templates?
Built into the software, or personalized?
What specialized features would you like?
What formats does it need to accept?
Length of Input?
Physical
If you’re open to it being purely offline, you might also want to consider a physical notetaker.
Physical notetakers can be:
Paper
Blank
Lined
Grid
Drawing
Folder
Color
Notebooks
Small
Large
Binder
Index Cards
Day Planner
...possibly with pencils, pens, colors, or labels. Day planners held a special place in my heart for a long time, and I still use notebooks for sketching and quick thoughts.
If you would benefit from applying a standardized format, you might be able to print instances of it out.
How Much Time Up Front? How Much Upkeep?
Here’s where you think though how much up-front time and maintenance time you’re willing to put into this.
You might want to reference XKCD’s Is It Worth The Time chart.
If you’ll be doing long-form notes and are willing to put in a bit of time up-front, I recommend learning GitHub-Flavored Markdown. For more about why, read “An Advertisement for Markdown.”
How Important is Retrievability?
There’s an important batch of related questions:
How Important is Retrievability?
What platforms will I access the software from?
Offline Access?
Sharing, Privacy, Security?
If it’s a calendar? Keeping old information might not be all that important.
If it’s math or programming notes that you’re likely to reference forever? Don’t lock it up in a weird format, on a site owned by a start-up that might go under in the next year.
Do you want to be able to share it? Conversely, do you want to keep it extremely private from others?
Password managers are basically flashcards with fancy encryption, but I sure wouldn’t keep my passwords on a flashcard site.
My Personal Notetaking Desiderata
There were my personal notetaking desiderata:
Markdown (required)
Markdown import (required)
LaTeX equations inline (required)
Offline access (required)
One of…
Easy to Transfer
Definitely won’t be discontinued
Wiki
Links Notes to Notes
Multi-platform
I want to be able to access and edit it from multiple computers
Frequent Cloud Saving (or similar)
Good recovery (I can’t count the number of times I’ve overwritten notes)
Quickly opens large text files
Some of my old journal files are comically huge
Searchable (bonus)
Templating (bonus)
Smart handling of merge conflicts (bonus)
Can draw in it (bonus)
Vim shortcuts (bonus)
Regex replace (bonus)
Search for It
Now comes trying to find notetaking software that actually fulfills some of your desiderata.
How I got through this was finding some wonderful Wikipedia comparison charts out there that made this pretty easy. Note that they can be pretty out-of-date, though.
Wikipedia has a Comparison of Note-Taking Software chart that seems quite extensive.
There are also a number of sub-charts covering more specific categories, such as Wikipedia’s Comparison of Wiki Software (meta, much?).
If you know a close fit that doesn’t quite work, you can look for Reddit or Quora threads about alternatives.
Remember: If you came up with a long list, you don’t have to satisfy literally every desiderata. Perhaps it’s okay if some are left uncovered.
If need be, you can split up the requirements and find 2 systems. I’d try not to do more than 3; it can be a lot of upkeep.
Once you have it narrowed down to a few, try them out. You might also want to read “How do you Want to Handle Misc?” After a few days, make a pros and cons list.
Tiddlywiki Pros and Cons
Pros
Markdown import (required)
LaTeX equations inline (required)
Wiki
Extremely easy to link notes
Searchable
Loads quickly
Easy to download and transfer
The whole thing is an html file
Multiple files can be open at once
CloudSave (in combination with GoogleDocs and tiddlydrive)
Not a Dying Baby Start-Up
Not going to stop being supported
Html files will run regardless of support
Can draw in it (a little)
Programmable for Super-Users
Great utility functions (ex: list-links, transclude)
Powerful list filters (ex: tags, edited last week, contains code....)
Cool plug-ins
Can write own templates and macros
Pre-existing macros exposed to editing or copying
Cons
Peculiar Markdown syntax
So very, very weird...
I’m good at regex. I wouldn’t use it if I weren’t; I’d be too worried about export issues.
Hard Learning Curve
Not quite enough documentation
GoogleGroup and exposed macros help somewhat
Some annoying macro quirks
Can’t safely edit in 2 windows at once
Can edit 2 notes at once within same window
Somewhat mangles multi-line code in imported markdown files
Particularly annoying since the formatting of the code box is exactly the same as its TW5Markdown baseline
Close Miss
Roam is good software that lies somewhere in-between an outliner and a wiki. When I tried it a year ago, it didn’t quite fit my needs (slow-loading, no offline, LaTeX handling is worse, a little too tree-like, chance it stops being supported). But it does offer what might be a gentler learning-curve than Tiddlywiki, and has definitely been a great fit for some people I know.
If you liked Workflowy and want to try something a bit fancier, Roam is a good choice.
For people who like Roam and are curious to try Tiddlywiki, there is a Tiddlywiki with some added Roam-like features named Stroll.
(If you try Stroll, I’d be curious to hear about your experience. I’ve considered trying it, but am using basic Tiddlywiki at present.)
Miscellaneous
An Advertisement for Markdown
If you want to use some minimal formatting (italics, bold, links), and you expect to take online notes a lot, I heartily recommend learning GitHub-Flavored Markdown (GFM).
First, it’s not that hard to pick up. Second, there are a lot of markdown-compatible systems out there, leaving your exporting options wide open. It’s still readable as plaintext, though. Many editors provide formatting previews, so you don’t have to look at pure plaintext all the time. If you grow to want fancier formatting, many Markdown editors let you insert html formatting directly into the editor to modify your text.
Markdown makes it pretty easy to see and avoid formatting mistakes. If I’m copying from something formatted in a confusing way, and for some reason don’t want to just Ctrl-Shift-V (“paste without formatting”), seeing it in markdown can be pretty clarifying.
If you’re a less-frequent note-taker than me, you might want to pick something with less up-front learning. But you probably still want good export options, and markdown is the next best thing to pure plaintext for export.
If you want to go even further down the more-typing, less-clicking, “programmer laziness” rabbit hole: Vim-shortcuts for text editors, and regex search functionality, are wonderful things to have if you can get them. I readily admit that I write most of my structure-heavy notes in an IDE, before pasting them into anything else.
How do you Want to Handle Misc?
You’re going to have miscellaneous notes sometimes. Despite our best intentions, it just happens.
Do you want to feel subtly encouraged to delete them, keep them, or integrate them?
Would you rather hide them from yourself, get them in your face with reminders, or somewhere in-between? Do you want to choose to set those reminders for yourself, or for them to be automatic?
This probably won’t give you an easy checklist item, but it’s something to keep in mind during your first look at the software. Make an uncategorized data-point to serve as a test-subject on that first day with the software. See what you feel compelled to do with it.
Similarly, test out export capabilities early so you don’t have a pile of regrets later.
Footnotes
* The fact that I use the term desiderata outside of a mathematical context has struck some people as pretty weird. I find it useful, so I keep on doing it.
** I prioritize LaTeX highly both because my pre-existing math notes already use LaTeX within Markdown***, and because my ability to retain math improved meaningfully after I started using LaTeX to write down and render equations. (Seriously! I recommend trying it, if you haven’t.)
*** Side-note: A pet peeve of mine is notetaking software where you have to dedicate an entire line to every LaTeX equation. In addition to finding this restriction annoying, such software usually mangles any math notes I try to import. Notion and Roam have both been guilty of this, and it was a big part of what ejected me from both of them.
Today’s Challenge
Write up your own notetaking desiderata
And if your current solution doesn’t satisfy it well, search for something that does!
If you think some abstract commentary on types of notetaking software would help you get started, go ahead and read the accompanying post.
My favorite note-taking software is Cherrytree. It is a hierarchical structure of notes, where each note is either a plain-text or rich-text page. The rich-text pages can contain links, which makes it kinda like a wiki, except with a canonical tree structure. OS-independent, supports Unicode, can save either each page to a separate file, or all pages to one SQLite file.
Does not support LaTeX, though.
Very good outline! You made a good pitch for Tiddlywiki as well.
I’m into the Roam rabbit hole since january and am very pleased with the direction it went. Some of your close-miss criteria are also being addressed: Offline is coming soon, loading times are now very fast, API/mobile is on the horizon as well and since payments turned on, I’m not really worried about the longevity of the platform. I can’t comment on the LaTeX though, but the very powerful transclusion based on the freely query-able block structure of the whole database is just...very very nice. Plus, the community is being crazy active with web-clippers, addons, very powerful CSS mods etc.
I’ve been mostly using a mix
vimwiki for the Babble
TeXMacs for the Prune
Anki as a bucket for short insights, with very good retrievability (from PC and phone): click “browse”, and search. (I don’t use it as much to memorize, though I really like to just be able to pop something out without noticing the ever growing mess, yet knowing that everything will EVENTUALLY come out. Also, I love to be able to easily set if I want to see the card again sooner or later. Honestly, I wish these kinds of systems weren’t limitted to “memorization apps”, and didn’t try to be so automatic in their scheduling of the pieces of content.)
Occasionally, Freeplane for mindmaps. jrnl for quick and short date-stamped logs. Probably others...
I am not too satisfied though, so this post will be useful when I have the time to give it another shot.
It’s kinda weird to me how limited the options seem to be for flashcards and annotation/marginalia. While plenty of things perform the core functionality, I haven’t seen anywhere near as many interesting experiments with these formats as I have with outliners.
And for flashcards particularly? You’d think it’d be the simplest damn thing to program!
I’m not much of an index card person, but there seem to be a lot of people who swear on elaborate index-card set-ups. That I haven’t seen more of them implemented as software confuses me too.
(It does seem like it would be a good use of flashcards to set up a rudimentary priority queue, or stack. Possibly with added randomness. I’m honestly a little surprised it doesn’t allow that.)
It has been about a year, so here’s an update.
The initial things that drew me to TiddlyWiki are still covered in this post, and still basically true.
I still love using TW, and it’s still my main personal notetaking software. Since this post, I have gone through at least 3 cycles of feeling further charmed by it, after discovering it had some additional functionality that I didn’t know it surfaced (ex: the ctrl-E “Excise & Transclude” shortcut, easy palette-swapping, changing the journal datetime format, Github saving, easy export to JSON). I’ve only run into 1 new point of disappointment: its multi-person editing inadequacies.
It has a really good hackability philosophy. TW seems to surface everything it possibly can to deliberate modification, and then tries to make modifying it easy. The software is hackable, old, and a decent number of programmers like it, so the add-on ecosystem is pretty informal but really great.
(Unfortunately, I can’t compare it with Roam’s current ecosystem, because I haven’t tried Roam recently. My fuzzy and weakly-held impression is that TW leans harder towards amazing self-contained functionalities, personalization, and a culture of casual hacking & sharing. While Roam seems better with chrome extensions, the external-facing APIs of other software, and being intuitive to use for non-programmers. I am curious what other people’s impressions are.)
Between satisfaction with TW’s basic offerings, and a good add-on environment, I have basically gone from “trying new notetaking software all the time,” to “notetaking-monogamous for 2+ years.”
Notably, my typical response to finding something else that I want my notes to do has changed from “finding a new software suite,” to “sketching what I want as a spec, then hunting for an TW add-on or function that does that thing.” In some rare cases, I’ve even written basic implementations myself. The software has done a fabulous job of rising to my growing demands of it.
(To give some idea of the diversity of things out there: There’s a TW GTD implementation, Roam-inspired TW variant, TW slide shows, calendar things, timers, several varieties of TW as a code-highlighter (ex: codemirror), some flashcard things, graphs… this list is long. A few TW environments or add-ons barely resemble their starting point in anything but philosophy, and might qualify as full-blown web-apps.)
So, a new thing I recognize as very valuable, if you want a system to grow with you: A good add-on ecosystem!
This thing wasn’t in my old list of desirables, but I wish it was: I feel horrified in retrospect that I ever put up with a knowledge-base software that didn’t have transclude or lists-from-tag-filters baked in. It saves me a lot of time and frustration to have large chunks of my overview notes use pointers to update themselves.
Something TW isn’t good at: With TW, it doesn’t look like I can set it up so that 2 people can modify the same file at once. I ended up settling for Github merges. It looks like a few people have gotten part of the way to making this dream work, but there is a lot of set-up required to run it, and it doesn’t seem to be able to do this yet anyway. (This difficulty is a serious drawback of the self-contained single-file philosophy TW has.)