wakatime

Have you seen https://wakatime.com? It would provide more detail into your existing tracking, and help with your goal of spending more time on productive projects.

Task throughput isn't about the queue library. Your queue library should be fast enough, after that the bottleneck is the workload your tasks are doing. That's why you need prioritized queues so your slow long-running tasks run low priority and don't block the high priority tasks that should be fast and usually affect UX. We have 9 worker servers, each server has 6 CPUs, we process over 1M tasks per day.

I love PG but would never use it for message queueing. Postgres needs periodic table maintenance, autovacuum tuning for tables with high number of writes, and IMO can never be as good as RabbitMQ for distributed task queues. RabbitMQ is purpose built around messages and it's extremely reliable and performant.

This was multiple years ago so I don't remember details. I think something around task reliability, performance, and I think architecture flaws that meant we couldn't prioritize tasks or some other missing features we needed. We process around 50-1000 jobs per second.

Both, but unfortunately still mostly using Celery 3.2. Never had any problems with zombie tasks, since that's related to how you're storing tasks and queues and RabbitMQ is excellent at that. Zombie tasks are never a problem when using RabbitMQ.

Especially never had problems with zombie workers. That wouldn't be a problem with your choice of queue library, more an infra problem.

Here's the list of my issues: https://github.com/closeio/tasktiger/issues?utf8=%E2%9C%93&q...

It came down to I like the features of RabbitMQ:

* RabbitMQ scales messages without needing tons of RAM * I don't have to decide between not persisting messages to disk with Redis vs only using half the machine's RAM [1] * Queue and task visibility is better in RabbitMQ * Support for purging all tasks in a queue * Tasktiger had lower throughput than Celery and Dramatiq, maybe needs lazy-forking?

Things I did like about Tasktiger: * no feature bloat and it's possible to actually read it's source code[2].

[1]: To enable persisting to disk (Redis fork + save snapshotting) you must limit Redis to only using half the available RAM on a machine.

[2]: Celery is split up into multiple convoluted, bloated, difficult to read repos.

We tried Tasktiger but ran into zombie tasks and some small but annoying bugs while testing. Around that same time, we found Dramatiq and currently have Dramatiq + Celery<4.0 in use.

Can't speak for op, but for us the ones based on RabbitMQ have performed well:

https://github.com/Bogdanp/dramatiq

Stay away from Celery if you can, or stick to v3.2 because 4.x has a ton of bugs.

We tried this at WakaTime and it didn't scale well. We also tried https://github.com/closeio/tasktiger but the only ones that work for us are based on RabbitMQ:

Celery<4.0

https://github.com/Bogdanp/dramatiq

> We thought we only needed $15,000 in startup capital; this proved to be an underestimate.

When did you find out that wasn't enough money, and how much did you end up needing at that time?

Timewarrior is overkill, an automatic background time tracker is even less overhead.

Build better habits in multiple areas (sleep, entertainment distractions, work). Better habits give you more time to spend on productive things.

To build a habit of coding more, try a daily goal:

https://wakatime.com/goals

Yes, pauses and breaks can increase productivity so this company's intrusive monitoring is probably doing the opposite of what it intends. As long as the break isn't from an external interrupt.

Note: Self-monitoring that's not shared with employer is a different story.

That's why latency is always measured in p99:

https://stackoverflow.com/questions/12808934/what-is-p99-lat...

We use https://bench.co/ which also does monthly bookkeeping for taxes.

We use these to parse dependencies and auto-update:

https://www.npmjs.com/package/npm-check-updates

https://pypi.org/project/pur/

There were several stages of MVP. First usable version took a month and half to build and public launch with 2 IDEs supported was 2 and half months after starting to build.

May 3 2013 - Started development Flask website & Vim plugin (https://wakatime.com/blog/1-why-i-built-wakatime)

June 25 2013 - Finished Vim plugin and Website (https://github.com/wakatime/vim-wakatime/commit/4346a055e301...)

July 1 2013 - Started Sublime plugin (https://github.com/wakatime/sublime-wakatime/commit/b7fe36f8...)

July 15 2013 - Finished Sublime plugin and public launch (https://news.ycombinator.com/item?id=6046227)

Unfortunately I don't have WakaTime data until after finishing the Vim plugin, but everything after that I can see how long the actual coding took by dogfooding.

This page shows IDEs by most time logged:

https://wakatime.com/a-look-back-at-2018

Tip: Turn up your timeout preference.

https://wakatime.com/blog/37-when-is-time-tracking-too-accur...

The default 15 mins is very accurate, and only represents time typing not time working.

Not sustainably... I've done that myself: it's hella fun but towards the end of the week the code stopped making sense and had to sleep for a whole day. WakaTime is very accurate so my daily average is only 4 hours of actual typing per day:

https://wakatime.com/a-look-back-at-2018

https://wakatime.com/@alan

Seven years ago I solo-started an automatic time tracker for programmers called WakaTime [1] and launched here on HN [2]. Partly from listening to developers too much, I waited way too long (almost a year) before adding a paid plan, but now it generates more MRR than an SF developer salary not including stock options. Technically I make more from RSUs and stock from past startups as a regular employee, but if I wasn't lucky with those then it would be my highest income stream.

1: https://wakatime.com/about

2: https://news.ycombinator.com/item?id=6046227

For anyone thinking it's egregiously difficult to start a solo-project: You're right, but if you stick with it your persistence will pay off. For solo-products, I think grit is the deciding factor between success and failure.

Try prototyping apps/products for yourself or friends. At least if they don't become big companies they were useful and hopefully fun.

> I woke up this morning and didn't do anything "productive"

Stop thinking about it, enjoy your free time by going outside more. My most creative ideas appear when I stop looking for them.

Also, this related post is on the front page right now:

https://news.ycombinator.com/item?id=21324768

We use ipstack.com, I'm curious which have the most accurate and complete data?

Mailing lists is a trap. As your company grows it implements a data retention policy and poof, there goes all your knowledge when your old emails are auto-deleted.

Only use mailing lists for notification with links to the actual knowledge in a real document.

Me: Costs nothing because I use an automatic time tracker called WakaTime [1].

However, I only track time outside of work when invoicing contracting clients.

At work I only track feature velocity not time. At work I only use the automatic time tracker for goal setting, leaderboards for fun, and to just see my stats such as time I spend waiting for code to compile vs writing code.

[1] https://wakatime.com

Software devs use WakaTime because it's automatic:

https://wakatime.com

For non-engineers it seems they mostly use old manual time trackers like Toggl.

Is this in SF or Mountain View?

Looked at it, but setup for SSDB was easier and already was using SSDB in production. Hesitant to change something that's working so well. Do have periodic connection issues with SSDB but so infrequent it's easy to work around client-side. Using the python redis [1] client with Flask.

[1] https://pypi.org/project/redis/

We're using a Redis alternative called SSDB [1] in production because the entire data set doesn't need to fit in RAM. This saves us thousands of dollars per month in server costs. Like you say, latency gains is nominal. The main reason I see alternative databases used is one missing key feature that's custom to a specific use case.

[1]: https://github.com/ideawu/ssdb