Selfspy - Tool for Personal Data Analytics | Hacker News Reader

Selfspy - Tool for Personal Data Analytics

github.com

154 pointskrat0sprakhar25 comments

I recently wrote a similar utility. I'll definitively take a closer look at this.

I don't know if you have these features, but here's couple of features I have in mine that you might find useful:

1) I can upload a timeline into Google calendar. I do this by chunking my day into half-hour increments, and whatever I spent the most time on in that half-hour get assigned to the half-hour. (Hey, Google, please let me set the task color through the API - thanks)

2) I also sample the title window of the foreground app, but I can override this. For example, if I go running, or do the dishes, that time will be properly tracked.

3) I assign tasks to categories. For example, emacs, shell, my IDE, are all "Development". Dishes, cooking, etc are all "Chores". I'd rather track time by these larger categories.

I wrote this utility a week ago, so I haven't been using it that long. But I am liking it. One more thing: Python absolutely ROCKS! This utility is less than 700 LOC and was a pleasure to write.

Any chance you're willing to share this code?

That looks absolutely amazing. You(?) even prepared packages.

> In general, try to set your programs (editors, terminals, web apps, ...) to include information on what you are doing in the window title. This will make it easier to search for later.

So if I'm in the terminal, Selfspy can only tell what I'm doing by the window title?

No, it's just for easier data retrieval with e.g. selfstats.py. It has a -T option that filters by window title, if I understand the docs correctly.

This looks awesome, but I wish there were some built in graphs and charts for my usage, most visited tabs, etc.

Otherwise, awesome stuff!

Here is a fork by github user ljos that includes OSX support:

https://github.com/ljos/selfspy

I am actively using it, and it seems to be pretty much like the original linux version.

Actually, now I've forked this myself to streamline the installation process:

  git clone git://github.com/iandennismiller/selfspy.git
  cd selfspy && make requirements && sudo make install

I keep getting stuck at:

Traceback (most recent call last): File "/usr/bin/selfspy", line 19, in <module> from activity_store import ActivityStore File "/private/var/lib/selfspy/activity_store.py", line 9, in <module> import sniff_cocoa File "/private/var/lib/selfspy/sniff_cocoa.py", line 1, in <module> from Foundation import NSObject, NSLog ImportError: No module named Foundation

I think Foundation provided by pyobjc. I suppose it's possibly you don't have a suitable version installed in site-packages, in which case you might try running the following command globally (i.e. outside the virtual environment):

  sudo pip install pyobjc

This is very cool. I've wanted something like this for a while.

Something like this, which can also string together what I do across phone, tablet and computer would be even more interesting.

I was just thinking the same thing - potentially you could mashup Selfspy with our SnowPlow (https://github.com/snowplow/snowplow) project - so all your behaviour events get logged to S3 and then analysed...

as part of my dissertation work, i built a related self-monitoring tool, except with a focus on helping scientists/researchers record and annotate their workflow: http://pgbovine.net/burrito.html

I've been using

* Karm on KDE/Linux (tracking by virtual desktop, one for each project)

* Manic Time Tracker on Windows (very much like this but with a full GUI. Only works on Windows though)

* For emergencies on Windows there is also Timesnapper which in facts documents in screenshots what happened (in case you have to "prove" it .)

this utility seems like a real improvement on the current situation though. Thanks in advance!

I'm using Timesnapper and it's great. Not only it keeps screenshots for everything, but you can also filter by application while searching for an old website you saw or data that you forgot to save. It needs a lot of storage though. In 50GB you can keep 2-3 months of screenshots.

This is very interesting and the first aspect what stood out to me is the possibility of such a tool being used to enable a form of ongoing user sanitation with regards to security. By that I mean opens up the possibility to collect statistics upon how a user does various tasks and qith that finger print approach enable the ability to highlight acceptions to how the user works. This could in essence provide a form of ongoing verification that the user is who they say they are. That is certainly one possibility this type of approach opens up. The ability to do ongoing user biometrics.

It seems a bit violent for the HDD, doesn't it? every time I scroll the mouse wheel, I can hear that selfspy writes data! Is it really safe to use for a long time?

This is an extremely interesting tool but the thought of collecting everything that I have ever typed and storing it gives me the chills, even if encrypted....

You sound like somebody who links there history file to /dev/null. In that I agree, but early days. Though the ability to have everything you typed stored in a controlled way does enable the posibility to see if anything else if collecting that same data in any way and with that opening up the possibility to identify unknown key logging. At the very least it would be another approach and opertunity to spot the bad amongst the good.

This is amazing, I'm worried about the possible security flaws, but I don't think that could be much of a problem.

I'll take a look tomorrow and see how it works.

Glad someone found and posted this. I'd been using PyKeyLogger for the past week (http://sourceforge.net/projects/pykeylogger/), but this one has built-in analytics. Hopefully someone beats me to a Windows version.

Suppose I want to combine this with other personal time-series data--can anyone suggest a good tool and workflow?

This is awesome. I just started building an app to track misc personal stats (a la Daytum, but with an API) and this will work great with it.

If anyone is interested in collaborating on similar stuff for a commercial project (an OS X app) let me know. I'm working on an objective-c app that does some similar things.

I'm f.mischa at gmail

When I first heard the title, I thought it might be an idea I had heard about for providing analytics/funnels/charts for your personal browsing history.

I have tried tracking but data becomes too clumpsy to be of any use.

Reply on news.ycombinator.com