cribwi

A similar research is done for humpback whales by HappyWhale: https://happywhale.com/ They're doing a great effort with already over 600K user/tourist submitted photos of whale encounters to track individual whales over the globe

Great approach and good write-up! I’ve implemented a similar technique before on PostgreSQL, but with the materialisation in the application backend. Still works like a charm.

>So instead, we created a special kind of trigger that tracks the minimum and maximum times modified across all the rows in a statement and writes out the range of times that were modified to a log table. We call that an invalidation log.

Does this invalidation log also take into account cases where the view has an aggregate that is based on data from a bucket other than itself? For example, a lag() or lead() might be used to calculate a delta compared to the previous bucket. Then, if a data point inside bucket 1 is added into the realtime table and bucket 1 is invalidated and re-materialised, for integrity reasons also bucket 2 needs to be re-materialised?

In some cases for removing repeating (intermediate) calculations, I generally find it easier to use a lateral join (in postgres), like

    select
        title,
        country,
        avg(salary)         as average_salary,
        sum(salary)         as sum_salary,
        avg(gross_salary)   as average_gross_salary,
        sum(gross_salary)   as sum_gross_salary,
        avg(gross_cost)     as average_gross_cost,
        sum(gross_cost)     as sum_gross_cost,
        count(*)            as emp_count
    from
        employees,
        lateral ( select
            (salary + payroll_tax)                   as gross_salary,
            (salary + payroll_tax + healthcare_cost) as gross_cost
        ) employee_ext
    where
        country = 'usa'
        and gross_cost > 0
    group by  title, country
    having    count(*) > 200
    order by  sum_gross_cost
    limit 3;

If you're running PostgreSQL, you can use the built-in generate_series (1) function like:

  SELECT id, random()
  FROM generate_series(0, 1000000) g (id);

There seems to be an equivalent in SQLite: https://sqlite.org/series.html

[1] https://www.postgresql.org/docs/current/functions-srf.html

You can assign a custom shortcut to it via Settings > Keymap and then the action "Main Menu > File > Local History". I myself have assigned the shortcut control+shift+=

Try using https://m.xkcd.com/ and click the comic to toggle the alt text

I know that specific story from the book "How not to be wrong", by Jordan Ellenberg. It was nice to read it from a different angle in this article

Part 1: https://news.ycombinator.com/item?id=7654439 http://vimeo.com/85490944

Since it has been broken some time ago: no. But when it was still working: yes, I found it rather useful. Especially with very small commits/bugfixes and such. I hope they'll fix it soon

I've been using Phabricator for almost a year now in our company. At first, I also had my questions about the workflow with arcanist (the local PHP cli software to interact with Phabricator). However, right now I really love it and miss it on every repository which works without Phabricator. Commands like 'arc feature', 'arc tasks' and 'arc inlines' provide a solid abstraction over the repository to bring the workflow to the cli.

About the unit tests: indeed, out-of-the-box Phabricator only supports running them locally (you can configure the unit engine to run every time a piece of code is been send into review). Luckily, it's not that hard to hook into the event engine (https://secure.phabricator.com/book/phabricator/article/even...) to, for example, trigger your CI-server to run the test suite for a specific revision. Also, Phabricator is still being developed actively. There are a bunch of alpha and beta applications which will only make it more powerful (for example: DryDock and Harbormaster, which will ease the work for automatic builds: http://www.guywarner.com/2014/05/integrating-jenkins-and-pha...)

In my opinion, Phabricator is one of the most powerful tools around for doing code reviews and effective collaboration management. Even with the large amount of beta applications (which are promising in many cases) the decision for us was easy to make.

Funny to see a new player entering the game to start button bashing and ruining the playing field in a matter of seconds :)

Used it on a regular basis, but since I owned multiple Android devices this tool became pretty useless. You can't filter on devices, so when I'm out with my phone leaving my tablet at home, the location history only displays a bunch of spikes of me traveling with tremendous speeds.