Capstone, a Tablet for Thinking | Hacker News Reader

Capstone, a Tablet for Thinking

inkandswitch.com

254 pointsandymatuschak41 comments

Ink&Switch have 6 papers out on their site in the last 9 months. Each is a thorough review of an interaction style through a product they develop. Since they are an independent research lab their papers are much more approachable and fun than other, say, CHI or UIST papers I've read. These papers come off the back of 2+ years of research. Each of the individuals on the team have awesome backgrounds and Ink&Switch serves as a great starting point to do some backwards internet research if you want to see some wonderful design work. I look forward to anything else they will put out, I've skimmed each paper and thoroughly read 2.

As to Capstone itself, it seems to me a wonderfully simple tablet interface. After rolling it around my head for a while I can't believe we ever ended up with grids of icons and widgets as the "desktop" of modern touch based devices. This nestable 2D canvas scheme (don't miss the jump links, no you aren't "trapped" by this scheme) feels intuitive and affords much power to the user that they can be fine to ignore or learn at their own pace. I find that reading these papers has influenced my desire for breadcrumbs on my tiling wm laptop setup in a way I can't currently figure out but I am making small progress.

I've long been interested in OneNote. None of the other major note software systems have seemed as good. I still, however, use vim and paper. This seems like a tool that could blow OneNote out of the water. A user could build OneNote's UI in Capstone in what looks like a little bit of minor fiddling. If OneNote is a modern take on paper, Capstone is a modern take on screens.

By "screens" over "paper" I mean dynamism, transformation and computing power. Paper cannot execute, and paper simulation on computers takes no advantage of their digital environment. "Screens" however know where they live and open up capabilities to their users that reflect the device's actual power. I have shells on my desktop, I do not have shells on my OneNote. Capstone gives me a shell again. Why would I ever give it away, and use OneNote without the power and ability I expect everywhere else on my computer?

I would like a better word to sum up this idea than "screen" vs "paper" so please drop me some recommendations.

Paper = “spatial interface” in the original/most literal way the term was intended. The data is materialized into one point in space, and can only be accessed by going to that point in space. This was the original idea behind the early unification between saving files in a file-manager, and “iconifying” document-displaying windows: a file was just a hibernated window, and a window was a 1:1 representation of some particular file.

Screens = “views” or “data bindings” in the SQL or MVC sense, or “projections” in relational algebra. There’s data somewhere, and then, separately, you have a view into that data: a distinct, live object through which you interact with the data, which can have a UX optimized for a given use-case, distinct from the “objective” UX suited to directly displaying the data.

Capstone is sort of a spatial interface of view objects (rather than a spacial interface of literal documents, or a database-representation of view objects), which is novel.

Dang, I like your perspective. Thanks for sharing.

I want to explain digital displays in a name that exposes the idea of dynamic use. Of reading/writing/drawing while simultaneously affording computational ability, execution and simulation.

This is starting to sound a bit like OpenDoc.

I'm also interested in OneNote. I have a Remarkable tablet, which is nice. But all this Ink&Switch stuff looks terrific.

So far, my best workflow is using the Remarkable to read books and papers. I write down notes with fountain pen and paper, and consolidate them on Emacs, in Org Mode using a Zettelkasten style. I would love to simplify this while keeping the essentials unchanged.

If you want to go the other way and keep everything in emacs, you may be interested in something like org-noter/interleave, or simply pdftools.

https://github.com/weirdNox/org-noter https://github.com/politza/pdf-tools

Previously, from Ink & Switch:

Local-first software: You own your data, in spite of the cloud (190 comments), https://news.ycombinator.com/item?id=19804478

Slow software (260 comments), https://news.ycombinator.com/item?id=18506170

Muse: designing a studio for ideas (18 comments), https://news.ycombinator.com/item?id=19816189

I've struggled for years to articulate my ideas and express things clearly, 'specially when it comes to challenging technical problems. One day, I randomly bought a tiny 12" x 5" whiteboard while I was at the dollar store, with the intention of using it to keep a physical todo list.

I eventually reached for it while in the heat of a challenging debugging session with some colleagues, and was surprised with how much it helped me solve the problem at hand. It quickly stopped being a todo-list after this incident. Turned out my inability to articulate things effectively was not because I was stupid, but because I am a predominantly visual thinker.

The fact people are working on tools like this excites the hell out of me!

Could you share a little about the difference between the small whiteboard and a letter-sized paper notebook?

Paper in my experience can get messy real quick. Example - When drawing a diagram of some process it is common for me to want to tweak and reconfigure things to arrive at a solution. With paper, it requires you either erase, scratch out, or re draw the diagram with your desired changes on another page.

What I love about marker boards is the ability to freely draft and modify on the fly without worry. When I want to commit ideas to "storage", paper works best :)

I agree 100%! Though I prefer to store things digitally. I feel so liberated no longer having so many scratch pieces of paper around that may contain an important note or solution to a problem. I remember working through a calculus book and having about half a foot's worth of paper (notes, attempts at solving problems, rewritten solutions to problems, etc) that eventually fell victim to entropy and was absolutely useless for anything but that I kept around out of fear of forgetting all of it. At least now it's all in one box. Maybe I'll burn it this winter as kindling - or would that be too derivative? :^)

These days, anything short term -> whiteboard, anything refined and meaningful enough for reusing later -> PC.

that sounds fantastic!

fwiw I came to a similar solution but it came in the form of a battery powered eraser and a big pad of paper.

erasure is the problem, not marking. a good soft graphite pencil and an electric eraser is my whiteboard ^^

Still not as good as Microsoft's "Courier" concept from a decade ago:

https://www.youtube.com/watch?v=UmIgNfp-MdI

Interesting, thanks for sharing. Did this project reach a dead end?

according to a 2011 cnet (1) article it was months away from release but "Courier was cancelled because the product didn't clearly align with the company's Windows and Office franchises"

1: https://www.cnet.com/news/the-inside-story-of-how-microsoft-...

Yes, I was really looking forward to this.

The object/document dock in the middle of the screen on the hinge was a nice idea. NextStep had Spectre. Wondering whether there is a nice, multi purpose object doch / visual clipboard for linux somewhere.

Nothing has come close to Microsoft's One Note for me when it comes to note taking apps. The combination of One Note and an iPad with the pencil is particularly powerful.

I could never pick up the fundamentals concept of OneNote. What's the deal with it?

Depends on how you use it. 1. OCD level organisation required? One note has got you covered. Books and tabs and what not. With tagging. 2. Want to toss everything in and forget about it? OneNote does the job. Windows and iOS supported. Plays well with my handwriting. Rest of platforms haven’t checked.

> With tagging.

Actually, tagging is one of the things OneNote should do better.

It doesn't support hashtags out of the box so tagging means selecting from a list of predefined tags.

One of the things that really annoys me about OneNote (I'm a big fan of it though.)

Yes! When I first got into OneNote (2007 version?) I put a lot of effort into using and customising tags but quickly hit limits. Even if other features never updated, an overhaul of the tags system could drastically increase OneNote's usability.

It always felt odd that the same app that reliably converted my handwriting to text a decade ago had a small, fixed number of tag slots and no ability to import icons. I remember thinking "well surely this was a v1, they'll improve it in later versions" but no, not even the tiniest change.

I'll add two more things:

- internal links doesn't work on mobile.

- local shared folder mode doesn't work in later versions. This was brilliant for places that has restrictions on what they can upload to the cloud. OneNote could often replace internal wikis for smaller companies (in some areas it was even better) but now that becomes less viable as OneNote for desktop gets older.

> It doesn't support hashtags out of the box so tagging means selecting from a list of predefined tags.

It does support custom tags. I don't think there's a way to add custom icons, but custom tags are definitely a thing.

I made my top level comment with reference to OneNote before seeing your comment. Why is it that this reminds you of OneNote? I agree, and Ink&Switch certainly show that tablet and stylus are a quick moving interaction model, what more do you want from that interaction scheme? Does OneNote do it for you or do you want more? I pointed out above that I think OneNote "is a modern take on paper" but Capstone "is a modern take on screens" and I mean that OneNote for all it's brilliance only expands the whiteboard to paper. Capstone has worked in so many more powerful abilities to it's user, it takes full advantage of the fact that it lives on a screen on the computer. They wrote a great paper on end user programming in this tablet environment. That doesn't seem to even be an experiment that could exist in OneNote.

Muse[1] is their[2] more recent similar effort.

[1] https://www.inkandswitch.com/muse-studio-for-ideas.html [2] https://www.inkandswitch.com/

I have been using a Boogie Board Sync for a couple of years (the original, haven't tried the latest).

It syncs with dropbox and allows automated filing into one of four types with a stroke of the stylus.

Best note taking implement I've ever used - and I used paper for years. This replaced it. I use it to teach my children new ideas, note everything down, test theories. Give it a shot.

My workplace uses Bluescape [1] in a similar way, with a focus on multi-user collaboration. We mainly use it on dedicated touch-TVs (like a digital posterboard + whiteboard), rather than on tablets, but the thinking is the same.

[1] https://www.bluescape.com/

Conspicuously missing from the "existing tools" section, and (as I read through the page) a troubling indicator of outcomes: index cards, well beyond "spread on a dessktop" scale.

Denis Diderot, Linnaeus, Dewey, John McFee, and numerous others have used this method.

Niklaus Luhmann's Zettelkasten (https://zettelkasten.de/posts/zettelkasten-improves-thinking...) and POIC -- pile of index cards (http://pileofindexcards.org/blog/cluster/ and https://unclutterer.com/2014/06/17/the-pile-of-index-cards-p...) -- are two models that help impose a structure on chaos. Hypercard was an attemp to map index cards to computeers, and influenced both the World Wide Web and Wiki, though digitisation both adds to and subtracts from the original physical manifestation.

My own card collection is approaching 10,000 cards, at roughly 1,500 per box, and where a card can hold about 500 characters, roughly 100 words (some do, many have fewer).

(Having read, and viewed the demo.)

... what's incredibly frustrating for me is trying to convey the scale of information gathering for research, as well as types.

I'm working with my own thoughts, experiences, Web references, and published works (both books and papers, as well as other forms). The quantities I'm working with are typical of academic researchers and writers whom I admire, whose bibliographies and notes may run 100 pages or more (and are often a substantial part of a book's real value).

Some earlier descriptions:

PoIC: Pile of Index Cards https://ello.co/dredmorbius/post/u4dgr0tkxk4tk9npuvex5a

Organising and planning research https://ello.co/dredmorbius/post/fj5rzi8zmouyrmvg8yzzva

What I keep seeing are systems that work at toy levels (a handful of items, occasionally a few dozen, rarely hundreds of items). And nothing in Capstone's collateral communicates anything differently. My console file and email tools can (and do) work with thousands and tens of thousands of items, though providing sufficient context, both static and dynamic metadata, is a challenge.

Email does slightly better by way of (searchable) headers -- From, To, Date, and Subject, but cruicially; In-reply-to and References. These are elements email inherited from early-20th-century office communications filing systems, with the referrence chain actually predating most other fields. Where a message fits within a discussion stream is often far more important than what it says. "We kill people based on metadata" as former CIA and NSA director General Michael Hayden said (https://www.nybooks.com/daily/2014/05/10/we-kill-people-base...).

(JoAnne Yates and James Beniger are among the few who've studied this field, a surprisingly interesting (to me) exploration of group communications and decisionmaking.)

A simple tool to view documents (in various formats; PDF, ePub, Mobi, djview, and sanitised, standardised HTML), look up metadata, and add this conveniently, would be a huge win. I suspect copyright concerns are a major hinderance.

For both thoughts and published works, context matters, though it's very poorly supported by existing tools, and is almost always tedious to add and verify. Dublin Core metadata is a good start (https://en.wikipedia.org/wiki/Dublin_Core). My principle Android eDoc reader, Pocketbook Reader, lacks even an 'author" field, and requires multiple steps and clicks to edit, correct, or verify information. (I've ... tried reporting this -- it also lacks a useful issues-reporting mechanism.) Another tool, Pocket Reader (entirely different, now owned by Mozilla) has ben a hot mess for years, though there's been a recent update I need to explore. My observation that it gets worse the more you use it remains distressingly accurate (https://old.reddit.com/r/dredmorbius/comments/5x2sfx/pocket_...).

(Reddit itself was something of a tool, though it's ultimately failed the test.)

What Capstone most lacks is a keyboard. I'm pecking this comment out with a stylus on an Android tablet for which I'd prchased a defective-from-the-start Logitech keyboard. Though compromised, operating near my 90 WPM typing speed, when needed is ten times faster than touch input (and increases screen space by a third to half). There is no available similar replacement keyboard -- the field apparently believes in constraining peripherals not only by mamufacturer and form factor, but to specific models and variants. A small bit of standardisation would go far.

See: Tablets and Keyboards https://ello.co/dredmorbius/post/lqgtwy_rhsfbdh5cdxb1rq

Tablets are useful for their portability and consumption. After four years of constant use, and for almost wholly self-imposed limitations by vendors, litle else.

Capstone seems to follow this trend.

You might be interested in Zettlr's emphasis on Zettelkasten[1]. (Zettlr is a PIM - FOSS, portable, humanities-oriented.)

[1] https://docs.zettlr.com/academic/zkn-method/

Just to be clear, you actively maintain a collection (database?) of 10,000 index cards, spread throughout various boxes? Out of curiosity, why do you not store these digitally instead?

Yes.

Largely addressed in text and links above.

Index cards are close at hand, flexible, and low friction.

... and substantially future-proofed.

This is only slightly more future-proof than keeping text files on a computer. So slight, I'd argue the difference doesn't even matter. I don't see the advantages to this behavior over storing them digitally.

Digital index cards can be kept closer (on your phone, in your pocket, always), and arguably lower friction too, no? And plain text is pretty flexible too. IDK, I just thought the concept was strange - but if it works for you, that's great!

I love this line of questioning existing computer interface paradigms, it's not done enough.

There is a lot of good stuff about this prototype (except it being built on the web stack - what??), I'm sure some people will find a minimal UI version of Prezi useful.

For me, I wish iPad had more control (customisable keyboard shortcuts for nearly everything) - not less. I am also really interested new physical devices for interacting with computers. I bought a Bluetooth footpedal for instance - I find it really useful to do certain simple things while freeing up my hands to do others (play guitar, draw, type, whatever)

Has anyone tried https://www.tapwithus.com/ ? Looks awesome

I liked, but also not. Here is some not.

Suffix all my comments with "But yes, this was understandable in a research prototype".

Novice vs expert. The design seemed more oriented towards novice use, than the stated audience of creative professionals.

Short-term use vs ergonomics. Long-term intensive expert use prioritizes ergonomics. So while "toss off edge" to delete is useful fun, adding a "scribble out"-like alternative that avoids giant arm motion would be nice. And traversing to "toggle ink/erase" was ick. This felt an impoverished UI vocabulary, compared to what many creative professionals are familiar with.

User|content vs user|tooling|content. Experts "wear" tooling to interact with content, so tooling information is a legitimate use of UI display. There seemed a story here of user interacting directly with content, which combined with UI minimalism, degraded the tooling experience.

Content vs representation|content. Similar argument.

Hierarchy vs graph. I do realize some people like hierarchical organization - mind maps and such. I find it crippling. But it does avoid the challenge of doing dynamic graph layout while preserving user spatial orientation.

A metaphor of a zoomable static pin-board with almost no tooling... might be a sweetspot for some. But it seems limiting. Prezi with web capture and minimal tablet UI, seems still Prezi, and niche.

Tablet tech. Note the severe constraints of using current tablet tech. Cue "understandable in a research prototype". They mention exploring dials and buttons and pressure. But there's also tilt, and rotation, and 3D position from optical tracking. They mention tablet being distinct from phone and keyboard, and deemphasize text, when a keyboard could simultaneously be a multitouch/stylus tablet (with optical tracking, and a "slice of a ping-pong ball" to slide over keys).

Research vs future. This was UX research, and may well have been useful for the community of which it was part. But... just don't confuse this with what the future looks like. AR is coming.

And there seemed some unfamiliarity with related work and current tech (yes, pen holders are a thing; yes, OCR is a thing; ...). Which for research can be fine, or not, but is something for you to bear in mind.

I enjoyed the appendices page: https://www.inkandswitch.com/capstone-manuscript-appendices.... My takeaway is they got burned by the painful state of web/ink/gestures on android, and the degree of domain experience needed to not-entirely-lose there. Which truncated their intended exploration. The concept of an android tablet as a node/electron machine for inking... sigh. For non-trivial multitouch and inking, I regrettably don't know of any hardware/OS/drivers/apis/libraries stack where "it just works" isn't a joke. Maybe some native iOS?

Emacs org-mode because.

So it's a specialized iPad.

An inspiring team doing interesting work. But see previous

https://hn.algolia.com/?query=Capstone,+a+Tablet+for+Thinkin...

"See previous" is only really helpful if there's previous discussion.

Reply on news.ycombinator.com