Graphing when your Facebook friends are awake | Hacker News Reader

Graphing when your Facebook friends are awake

defaultnamehere.tumblr.com

1,302 pointsadamch189 comments

My favorite part -

> This friend recommended nvd3.js, presumably because you’re not making real graphs in 2016 unless your graphing library is <something>.js and requires at LEAST one other <something else>.js as a dependency. Everyone looks at you like “what, you DON’T already use <something else>.js? Jeez say goodbye to your Hacker News karma. Just apt-get install npm && npm install bower && bower install-” NO STOP IT THIS ISN’T WHAT TIM BERNERS-LEE WANTED”.

edit: as huckyaus mentioned in a different thread, author did http://swagify.net/ as well. In completely unrelated news, I'm changing my handle to [Tr1Ck$h0t][LEGIT][60x7]$$$C30C0DER$$$, that will make me really popular among the cool kids.

Lol, I love how "SAMPLE TEXT" is part of every section of that :)

Tho only drawback I see with nvd3.js is that we can't generate png images of the graphs, server-side, for email of PDF export. Is there an nvd3-js-to-png.java ;)?

It's called PhantomJS :P

What? You don't already have it installed?

How can I tell?

Note: you can use apt-get install npm and npm to install phantomjs which is not a node.js program but for some reason is on npm, or you can just rent an entire AWS region to compile phantomjs from sources in your preferred linux distribution.

If the graph renders to canvas, you can get a data url for the image and send that to a server for decoding/storage.

Yeah the irony is that the link to wiki on Tim Berners-Lee is actually

http://t.umblr.com/redirect?z=https%3A%2F%2Fen.wikipedia.org...

Is this WHAT TIM BERNERS-LEE WANTED??!!!One1!?

This one got me in stitches, a collection of developer insider jokes in one huge sentence:

AND ANOTHER THING when it’s midnight and your x-axis formatting function doesn’t convert UNIX times into JavaScript date objects properly because there’s no timezone information and I dunno JavaScript was written by some guy in two weeks (yeah I ain’t afraid to call it out what of it) and your binary-search based conversion of sparse timeseries data into uniformly dense timeseries data is causing so many data points to be graphed that it’s slowly crashing Chrome and you’re watching helplessly as your RAM goes up and Chrome won’t close the tab and it just doesn’t seem right that 2016, the year of the Linux Desktop has brought us this situation I mean I thought if you had enough <something>.js libraries this stuff was meant to just scale right up so tha-

The whole article was fun. I loved it. Also to take it a step further, I wouldn't apt-get install npm, I'd first use ruby to install homebrew, then brew install nodejs, then npm install bower, then bower install. Because linux doesn't have enough package managers to manage my package managers to manage packages managed my package managers.

Indeed, the article was a great and humorous stream-of-consciousness.

For me, it was this:

> ... and I wouldn’t be embedding this ugly graph as a pdf into a LaTeX document that takes 3 passes of pdflatex to render...

Some people might take issue with it, but the writing for this had me in stitches. I very much agree with the author on graphing libraries - there are a few good simple ones, but as soon as you want anything unusual you have to jump to these big, hard to configure monstrosities. More than once I've just given up and written my own server-side generator.

I, too, am laughing way too much whilst at work over the writing of this.

I agree; the writing was excellent. The style might get old after a while, but it captured my feelings on several issues very closely. :)

+1 The javascript rant is epic! :)

nvd3 is actually really awesome http://nvd3.org/

This was the funniest writing style since the WAT talks lol

I decided the guy was British, but he's probably Australian. (His Github profile says so. Would you lie?)

Lots of casual self-deprecation. I'd recommend well known comedians, but I'm no good at remembering names.

What's striking about this writing style is that (as a junior dev) it's a lot easier to understand what he's doing.

By sticking to jokes and a simple style, I never felt confused or lost in lingo.

I was thinking at the same thing today reading some tutorial. You have these serious ass tutorials that are an horror to read. And then you have these gems, these very informal tutorials with jokes and slangs and ... and reading them is a pleasure.

I guess if you're a serious company like Apple, you can't really have that kind of tutorial for iOS dev. But that would really help.

They can be funny, but it gets old really really fast.

I find myself reminded of the "for dummies" series of books.

they are written in a humorous tone, and intersperse joke drawings between the chapters.

So are the "Head First" series of books. I would definitely recommend taking a look at them :)

I have wanted to write a series of "X for mathematically inclined idiots" for several years. If you're going to pick a baseline...

I'd probably buy one or two.

In college, I was a mechanical engineer living with a CompSci major, CompE major, and a CompSci / CompE double-major. I'd occasionally stop them when they were explaining things to me with "Explain like I'm a MechE". Slightly less self-deprecating than "mathematically inclined idiot", but same gist.

Because of this, I was also quite useful as a rubber duck for debugging.

> I guess if you're a serious company like Apple, you can't really have that kind of tutorial for iOS dev.

The fact that this is true is, to me, a slightly sad reflection on our culture. I understand that too much silliness can get in the way of conveying information, but as you point out, so can too much seriousness.

It's obviously a fine balance, and unfortunately it seems safer to err on the side of seriousness when in doubt. It's one of those things that if you sit down and ask any average individual, they will probably prefer a bit of silliness and an informal style in any situation where it's appropriate, yet somehow collectively we seem to have agreed to default to the opposite, and stick with things being quite dry and serious as a norm.

Interesting, but ultimately a little sad. Why is this the case? Why is our culture this way? Why can't things just be a little more fun by default?

I put jokes in my resume. You have to have a bit of restraint, but it can be fun and effective. An oatmeal cookie wouldn't be very good if the whole thing was raisins.

But a chocolate chip cookie would be decent if the entire thing is chocolate.

Except if you want a cookie.

(aaaaand this could go on for hours)

Therefore raisins suck and should be left out of cookies. QED.

The major problem is that people from other cultures won't get the jokes, and when you aren't in the know the colloquialisms add ambiguity.

I would have put it a little differently. It seems more about image to me than strictly "joking isn't appropriate in business." Apple has an image that is one of sleek, refined beauty. The style of writing used in this article would totally clash with that, causing nothing but confusion and weak branding.

On the other hand, there are brands out there with the exact opposite approach. Cards Against Humanity comes to mind first. Their image is explicitly one of mild carelessness and apparent candidness such that anything written too formally would seem out of place.

But indeed, there do seem to be more of the former brand type than of the latter. Having a neighbor who speaks constantly with a similar tone as the writer of this article, I can say that trying to hold my own in a conversation with him is really tiring. You eventually just run out of witty remarks. Is it possible it's more an issue of skill and time? As in, are there fewer people out there with the skills necessary to make good silly writing than there are people who can write seriously? And does it take them longer (and thus cost even more) to make writing in this style?

I rambled a little, but I think the issue is a combination of people equating seriousness with importance, as well as a general lack of creativity.

There's a really fine line there between humour and cringe, though. This article manages it well.

EXTREMELY FINE! Unlike the article, most copy-cat lol-writing styles do not fall on the good side of that line.

Yeah, like the next article I see that imitates this writing style is going to be awful.

Exactly. I'm guessing the reason is that a funny style implies a more personal and subjective sight on the things it describes, and that means it often distinguishes the important and complicated parts from the trivial ones. Serious texts often tend to threat everything the same and you have to think of the practical impact yourself.

      "too much medium,
     not enough message"

The writing style works because those jokes, asides, and little mannerisms that are normally excluded from "serious" writing are part of how we think.

When we refine our writing (or speaking) to - supposedly to make it less jumbled and easier to read - we usually end up removing a lot of the little details that made up the original thought. The writing may have much better grammar and organization, but real thought isn't perfectly organized and we end up losing some of the "character" in the original idea. (this may be good or bad, depending on the situation)

Vi Hart has an amazing overview[1] of this concept, in which she discusses the very wise words of the Edmund Snow Carpenter in his[2] book "They Became What They Beheld".

    The trouble with knowing what to say and saying it clearly & fully, is that
    clear speaking is generally obsolete thinking. Clear statement is like an
    art object: it is the afterlife of the process which called it into being.
    The process itself is the significant step and, especially at the beginning,
    is often incomplete & uncertain.

The best way to write will always depend on what the goals are, but in general the further away from the original thought you get, the more writing and other creative ideas tend to degrade. Incidentally, this tendency of a thought to degrade the further it gets from it's origin is also how plans become policy: [3].

[1] https://www.youtube.com/watch?v=bm-Jjvqu3U4

[2] probably co-written - or at least "heavily influenced" - by Marshal McLuhan

[3] http://ogun.stanford.edu/~bnayfeh/plan.html (very mild NSFW language of the form "____ happens")

>>The best way to write will always depend on what the goals are, but in general the further away from the original thought you get, the more writing and other creative ideas tend to degrade.<<

So very true...

Open a vein...share yourself...reveal yourself...share what you've learned and let the excitement you felt while discovering it flow...

For most writers, the majority of the time, your true audience will "get" both you and the ideas you're trying to share...

Sometimes you'll expand the size of your potential audience with raw enthusiasm, even when they have no idea what you're talking about...

What's with the ellipses? Always so awkward reading messages from people who always do the ellipses thing.

I think in some cases it is because people are taught that Comma Splicing Is Bad. But imo Ellipse Splicing Is Worse (and should probably be Considered Harmful ;) ). Commas or Em Dashes[0] are better, clearer, and (AFAIK) more grammatically correct.

[0]: http://www.thepunctuationguide.com/em-dash.html

As a not-so-junior dev, I didn't think he was all that funny half the time, and there was enough extraneous verbiage that I didn't bother parsing it all to find out what he was actually doing. I like the humorous style, but IMO this was badly in need of a once (or twice)-over to clean it up.

I agree with you. It must be an youth generation thing, I am in my mid 40s. At times, I find his writing style rather annoying, it feels like one of those TV adverts you see at 1am in the morning.

It is a pity because the content is quite interesting but we are here discussing about his writing style??

Do yourself and your career a favor and never refer to yourself as a junior dev ever again.

Yup, and it feels more train of thought which also makes it easier to follow. I loved it.

I swear by Highcharts. It's big, and paid for commercial use, but you can chart pretty much anything with it, make it look exactly like you want, and it's relatively easy to get started and make nice interactive charts.

No affiliation, I just use it for work:

http://www.highcharts.com/

HighCharts is awesome. I'm a big fan as well.

I've used it for school projects and work before, and was amazed by the documentation and API capabilities. Everything is configurable and easy to figure out.

I'm still not sure how the "Free for Non-Commercial" license interacts with something like the GPL - it seems like I'd have to use BSD in my projects instead if I wanted to use this in open-source stuff.

Edit: it's Creative Commons Attribution-NonCommercial 3.0. I wonder if that stops me running ads on a site that uses it.

If you really want to know you should probably ask a lawyer or the rights holder but here is an interesting page: https://wiki.creativecommons.org/wiki/Defining_Noncommercial

"The empirical findings suggest that creators and users approach the question of noncommercial use similarly and that overall, online U.S. creators and users are more alike than different in their understanding of noncommercial use. Both creators and users generally consider uses that earn users money or involve online advertising to be commercial"

FWIW in past I just emailed them asking for clarification about whether I needed a commercial license with my specific use case and they responded quickly and straightforwardly. Seems the easiest thing to do is just ask them rather than guessing or trying to interpret how the license is meant to be used.

Yep. Love HighCharts and HighStock in particular which allows you to create wonderful time series charts.

Everything about the writing was perfect. Even the tumblr name, "literally words", is great.

If you found the article funny you may also enjoy one of the author's other projects, Swagify[0].

The source code[1] for said project is pretty great.

Kind regards,

[24x7BLAZEIT]XxXhUCkYau$XxX

[0] http://swagify.net/ [1] http://swagify.net/static/js/swagify.js

Uh oh they know

Do you have stairs in your house? (i.e. do you browse SomethingAwful, because your style of sarcastic humour is pretty similar, going by your writing & the animated frog on Swagify, and yes yes i know the rules of HN...)

I love this guy's humor. I remember when I was 17 I used to write a column for flipcode.com where I thought I was being pretty funny in appropriate amounts (http://www.flipcode.com/archives/Theory_Practice-Issue_06_Ev... for example) but I like his jokes more!

But hey, I was 17 and also it was me so... I'm biased.

I used to read those articles! Fond memories.

Oh, no, I haven't really browsed SomethingAwful before, I just um I dunno make a lot of parody websites

I don't know why I swear by gnuplot and tikz. I feel like the more you use them, the more you will know how to use them, and the faster you will be.

But then I use some wysiwyg and I make something much nicer in no time... Should I really give up on these tools :(?

Depends how far you're Stockholm-Syndrome'd into using them =]

ಠ~ಠ

Nice TextFace™

nice work!

They're great, and I guess using them wouldn't really matter for this blog post. But they're useful for generating static things. As soon as you generate lots and lots of graphs dynamically, it's easier to offload it to the client. They'll download the JS library once (maybe even from the CDN, not from you) and you just push the data points which are 1/1000th the size of a rendered graph. (they'll also look better, because you can't do subpixel antialiasing without knowing the display)

It depends on the use case.

> If you reload the page you’ll see approximately fifty-bajillion network requests go off as Facebook desperately tries to load all the junk that it needs to display facebook.com.

I like this part. As a developer I've often looked at the network usage of large websites / web applications and it's always surprising to me just how...unoptimized it is as far as network connections go.

I mean Facebook loads decently enough and all I'm just surprised the first load isn't condensed into a small, handful of network calls to save on latency.

Progressive rendering and lots of small network requests can be good if the user feels its faster.

I suspect their mobile apps are more conservative about using fewer requests (and probably a binary protocol).

The Facebook mobile app on iOS is awful - worse than their mobile website. It takes an age to load, drinks battery life, is massive, doesn't contain the messenger functionality (you need another 60MB app for that).

Unlikely. People report that uninstalling Facebook on their phones increases battery life by up to 20%.

Given the age of Facebook and what you're describing, I can only assume that Facebook have just chucked fifty-bajillion interns (one after the other) at the source and told them to add more crap.

I mean, isn't that how monolithic apps tend to happen? Someone leaves, new person comes along and adds more to the cruft.

I mean, honestly, that's almost exactly what they said they do with the mobile app and how it contains so many classes / objects that it's really incredibly the app still works.

But I don't know anyone at Facebook so I have no idea how the website is handled.

> I like this part. As a developer I've often looked at the network usage of large websites / web applications and it's always surprising to me just how...unoptimized it is as far as network connections go.

Have you considered that your opinion on the best way to serve data to a billion users might actually be wrong? That maybe Facebook actually knows what they are doing?

> Have you considered that your opinion on the best way to serve data to a billion users might actually be wrong? That maybe Facebook actually knows what they are doing?

Certainly! But I've also done work where Humvees in the middle of Iraq use satellite internet connectivity with massive latency and the more connections it has to open to get started the slower everything gets.

Perhaps this is an advantage where latency is low so you can load parts of the page faster but I'd love to know some of the more technical reasons behind this!

Lets get rid of the notion that Facebook doesn't know what he's doing. He knows exactly what he's doing.

"he"?

It's a not-very-funny attempt at a Rubio joke.

I know it's a light-hearted post, but can we not?

Facebook absolutely hammers client-side system resources. Opening a few Facebook tabs, especially in Chrome, will bring a new fancy computer to its knees, and closing Facebook (either the website or the mobile app) famously substantially extends laptop/phone battery life. Facebook’s site architecture demonstrates staggering lack of respect for their users, even without considering the privacy implications.

Not really surprising, from what I’ve heard from friends about their cowboy-style internal engineering culture, and spaghetti piled on spaghetti internal code organization, where shipping new features ASAP is privileged above all other goals.

When you have endless money to throw engineers and servers at a problem, it’s possible to kinda sorta paper over a lot of horrible broken design. Not sure whether that implies anything one way or another about whether or not they “know what they are doing”.

It sort of depends on your definition of “know”. After all, their motto is “move fast and break things.”

This is the company that bragged about having 18,000 classes for their iOS app [1] and having to patch the Dalvik Virtual Machine to support new features in their Android app [2]; they are not infallible.

[1] http://quellish.tumblr.com/post/126712999812/how-on-earth-th...

[2] https://www.facebook.com/notes/facebook-engineering/under-th...

IiRC they aso exploited an Android bug to give themselves more memory. Facebook is woefully buggy and slow that many people are now either using the website or alternative wrapper apps that don't drain their battery immensely.

As someone else have mentioned most of these connections are progressive connection. Another thing to consider is their CDN usage. Facebook uses Akamai as their CDN of choice, most large to medium size telco/ ISP has their own Akamai Cache server at their NOC. So latency is not issue for the big files (images and JavaScript) after the initial reload, assuming they are not cached on the browser already.

So no matter how many connections are made when visiting facebook, it usually loads fairly fast because of aggressive caching and progressive connections. Most of the time the end user shouldn't be able to tell the difference.

"I mean Facebook loads decently enough and all I'm just surprised the first load isn't condensed into a small, handful of network calls to save on latency."

Actually it is: https://www.facebook.com/notes/facebook-engineering/bigpipe-...

> Actually it is: https://www.facebook.com/notes/facebook-engineering/bigpipe-....

Nope. Did you read through how BigPipe works? It's more than a handful; they basically load a whole bunch of mini web pages inside of another. I'm actually impressed they get better performance doing that; I would have thought condensing everything down would net much better performance. It would at least improve latency but I digress.

Yet somehow it's ridiculously fast... like, when I'm waiting for some other site to load, I almost automatically open facebook or hn in another tab, because I know those two sites will load instantly while pretty much everything else on the internet will take a few seconds.

Really? The notifications and the first story show up fairly quickly for me, but everything else usually takes a few seconds. Certainly not terrible, but I wouldn't call it "ridiculously fast" either.

0.3 seconds for me, just now. HN was 1.3 seconds.

Really?

FB just took 13 seconds for me to load fully! HN took only a second. I suspect the situation would be way worse if the browser cache was cleared too.

Don't get me started. The best thing I've ever seen is a site using Google tag manager to load an ad tag manager which loaded a different gtm instance which... Loaded the site's own js libraries.

This horror out of time and space was beaten, shot and stabbed, and the developer responsible for its invention was promoted to CTO and King of the universe.

I like to do this sort of web spelunking all the time.. But the writing and humor really make this more enjoyable than it should be! Of course Facebook leaks info to you about your friends - that is the sole attraction for people to use it! Seems like you could turn this thing into a browser extension as well if you felt daring.. Like some sort of FB snooper.

Yes the author seems to have a bit of fun with what he does.

"If you I dunno, didn’t have a lot of friends in high school, you might recognise that as a UNIX time stamp - the time in seconds since midnight, January 1, 1970. "

Great article. And a further reminder why Facebook kinda sucks.

Why does unix timestamp imply that Facebook sucks?

Not sure if you're seriously asking but OP was likely saying it's a great article because of the witty writing. The remark about FB sucking was related to the article's substance.

As someone who is not personally humiliated by my interest in computers/tech/programming, I wasn't really entertained by the constant "oh yea lol it's cause I'm a waste of oxygen that I know that, don't you hate me as much as I hate myself?" Maybe I know too many nerds with actual self esteem issues to find it funny.

Maybe you should go out more?

Were you equally aggravated by the people who said how much they liked the writing style? Sorry, won't be able to get into a long conversation, I'm heading to a bar.

It only gets funnier when you reply bitterly.

I didn't take any of that too seriously, it's obvious they had a great time with the project and I thought the write-up was quite funny.

I don't think the OP feels humiliated - if anything, I think what many of us find entertaining is that he recognises the perceptions of nerds/geeks to some folks and is able to laugh about those perceptions / prejudices them while relating to himself; just a different of way of looking at his writing.

It's not humorous because it's funny that he would have low self-esteem; it's humorous due to the irony in the juxtaposition of a demonstration of something so obviously clever and impressive and an over-the-top embodiment of a self-effacing stereotype. Maybe he's putting it on a bit thick, but I think it's well done.

This is standard British (also Australian, NZ) humour. Make fun of yourself, if it's slightly hurtful so much the better.

Example, which I'd not seem before: https://youtu.be/taTSxDVEHRM

hilariously, I'm British/Australian, so perhaps I've been overexposed to it.

This style of writing is so entertaining; it's like a funnier stream of conscious of what goes on in my head when hacking things like this together. If OP is the author, please write more.

>Similarly, I’m not sure why there are these weird spikes every three minutes (+- ~1minute) sometimes.

Could these just be keep-alive requests? For example, the mobile app checks whether it's still connected?

Good point!

I didn't think of that.

3 minutes does seem like the kind of time an engineer would code into their app.

So does that mean that the spiky periods are times when people are online the whole time?

>So does that mean that the spiky periods are times when people are online the whole time?

I'd be inclined to think so, but I don't work for Facebook, there may be an entirely different reason :)

Edit: or maybe, maaaybe, it's the point of time when people fall asleep over their mobile - they've stopped interacting with the app, it still sends a few keep-alive requests, and then logs itself out.

Could it be an unusual client, like a microsoft phone or something?

You should try to add some sort of tracking of when people start to write a message to you. If that's in any way possible. It would be really stalkery if you knew whenever someone started writing a message and perhaps decided not to send the message.

pidgin[1] has a plugin ("psychic mode") that I always enable.

It opens up a chat window when somebody start typing.

It always freaks out the few people that never send the message.

[1] http://pidgin.im/

awesome, thanks!

I don't have a Facebook account, but is there really no way to not share your available status to your friends? In Gmail you can simply sign out of Hangouts.

On a side note,

> If you’re wondering why the response starts with “for (;;);”, it’s to, among other things, encourage developers to use a quality JSON decoder, instead of like, y’know, eval().

This is wrong, as I commented on the linked StackOverflow post, perhaps a bit too strongly. But it's really frustrating to see that people have misconceptions because of incorrect answers on StackOverflow.

Dammit, I trusted that StackOverflow answer T_T

You can just turn chat off: https://www.facebook.com/help/www/215888465102253

Any messages you get when your chat is off just go to your Facebook inbox so you can still get messages even with it off you just kinda lose that online live kinda element. I mean this works just like most chat applications work. They basically all say you're online or not too.

You don't even have to sign out of Hangouts - Hangouts has an option to not display "last seen" status which will make you always appear offline (though you can still send and receive messages).

If you don't link to your answer many people are just going to discard your comment.

You sound pretty neurotic

It would be real creepy if someone does the same thing for Whatsapp, you can even predict who's talking to each other much better than Facebook. It's a bit harder to collect data from web.whatsapp.com because it's using Websockets but let me know if someone develop such tool and publish it on Github. :)

s/creepy/fun/, IMO. Too many things get called "creepy" nowadays, as if people had some serious insecurity issues.

I'm interested in why you think you could do better predictions using WhatsApp?

There's a subset of my Facebook friends, mostly older/family who are slow to reply to messages on Facebook and only use Facebook in 'down time' or whatever but are available by text all day. Since they got smartphones years ago, by text has meant by Whatsapp.

The way I was looking at it was people will get messages on WhatsApp but won't reply immediately. Same on Facebook. But people will also visit Facebook.com when ignoring messages so you still get online status info even if they aren't specifically using the chat. If you could look at WhatsApp and Facebook and combine the data you'd probably get a really accurate overview.

Yep, I thought the same thing.

There was a capability to do this, at least last year:

https://nakedsecurity.sophos.com/2015/02/17/whatsapp-spy-too...

I've done it. Just embed a javascript on to the web.whatsapp.coom page that tracks when the div says online. It works quite well.

There are even better ways which I'd rather not reveal.

You mean something like this? https://gitlab.maikel.pro/maikeldus/WhatsSpy-Public/wikis/ho...

Yep, I searched on Github but couldn't found. Using Gitlab makes sense considering Github is strict about legal issues.

Just so you know, at GitLab we will not hesitate to take down content that we think is illegal or has other issues. At this point I think that WhatsSpy is acceptable but we reserve the right to change our mind.

Since it's on a custom domain, I thought that he's using open-source version of Gitlab hosted on his personal server. Do you have control over projects in open-source version of Gitlab?

No we do not control anything when people host GitLab CE/EE on-premises. GitLab CE is MIT licensed so what you do with it is your responsibility.

I missed that when reviewing, I assumed WhatsSpy was hosted on GitLab.com. Thanks for enlightening me.

I know I'm in the minority here, but I just couldn't bear the writing style. I'm sure the content is interesting, but this article tries way too hard for my tastes. I had to give up after the first couple of paragraphs.

This is very well written, intelligent, and very entertaining. It's almost like he channelled Deadpool. Kudos.

Oh and did not know about the Copy as cURL feature on Chrome!

I did the same thing with the XMPP interface before they scrapped it and it was obviously much easier...also I used the built in graphing that's in Racket to visualize it. Also I made a thing to do desktop notifications whenever someone came online, which is actually kinda useful.

Antimatter15 has a pretty cool clock style visualization of this from 2012 [1]

[1] https://antimatter15.com/project/facebook-clock/

Nice work! I really like the idea that the web allows anyone to programmatically dig into the UI and extract data to do things. A friend and I actually made a whole API to interact with FB chat. You should check it out: https://github.com/Schmavery/facebook-chat-api. I'd really love to see what you can come up with, with some of the stuff we support.

I personally use BitlBee with bitlbee-facebook [1], and hook up my existing IRC bot to it. It works for my needs.

[1]: https://github.com/jgeboski/bitlbee-facebook

"Graphing how addicted your friends are to Facebook and Facebook Messenger"

> when it’s midnight and your x-axis formatting function doesn’t convert UNIX times into JavaScript date objects properly because there’s no timezone information and I dunno JavaScript was written by some guy in two weeks (yeah I ain’t afraid to call it out what of it) and your binary-search based conversion of sparse timeseries data into uniformly dense timeseries data is causing so many data points to be graphed that it’s slowly crashing Chrome and you’re watching helplessly as your RAM goes up and Chrome won’t close the tab and it just doesn’t seem right that 2016, the year of the Linux Desktop has brought us this situation I mean I thought if you had enough <something>.js libraries this stuff was meant to just scale right up so tha-

So, did you forget everything you learned about memory management? Or do you think Javascript really doesn't have sound memory management principles? Hell, it's not like you need to retain references to rendered points. Just dequeue them. Browser graphing libraries render to canvas which is just pixels.

Great work and nice style of writing...felt like nerd Deadpool ripping away FB :P

That's actually a great way of putting it.

You used to be able to do this with the Facebook Query Language (FQL) that Facebook exposed, sending something like this query to the FQL endpoint.

   SELECT uid, name, online_presence
   FROM user
   WHERE online_presence IN ('active', 'idle')
   AND uid IN (
     SELECT uid2 FROM friend WHERE uid1 = me()
   )

Unfortunately, the current version of the Facebook Graph API[0] doesn't have the online_presence field, so this is no longer possible. Maybe the Graph API will be updated in the future to also return the online_presence fields?

[0] https://developers.facebook.com/docs/graph-api/reference/v2....

What determines whether the app is online? What happens when the user is using the phone but FB is in the background? Does FB get some kind of update when the user is active on device? Or do OP's friend live in the FB app all day long?

Hi, I wrote this blog post.

I have no idea what the Facebook apps (Messenger and the Facebook app) do in the background.

It's easy to test that, but I didn't because #yoloswag

You're a funny person.

I followed his GitHub link at the bottom of the post and see that he's Australian. My preconceived stereotypes of Australians suddenly explains quite a bit of his writing style and humor, and makes the post that much more enjoyable.

Reminds me of the old-school user tracker (whose name escapes me) that would give you a bar graph of your friend's online/offline presence when AOL Instant Messenger was the dominant chat client.

This is awesome! Thank you for sharing the code for this. Overall I would say that this could be very entertaining to watch over multiple sites. Potentially gathering a good profile of your friends over time!

So, I can mine when my friends are online.

Perhaps, buy some targetted ads about 'SleepCycle' and show them to the naughty ones who sleep less than 6 hours. :P

Awesome and hilarious article. Id just like to note `for (;;);` is not to prevent users from using bad JSON parsers like `eval` but prevent older browsers with little to no cross domain policy from loading it with a script tag and doing evil XSS by overriding Array or Object constructors or prototypes to pull that data

Good point. Minor nitpick: I believe this is called CSRF, not XSS? But Im just repeating stackoverflow comments at this point.

No, CSRF (cross-site request forgery) is where a page tricks your browser into making requests to another domain in which you're already authenticated, in order to perform some kind of action. e.g. an img or script with a src "http://example.com/message.php?message=you+are+hax0red&s.... You can sometimes perform similar tricks with self-submitting hidden forms, or XHR. Quite easy to mitigate using nonces and referrer checking.

But.. that's exactly what while(1); and friends in json responses protect you against? someone overriding the Array constructor function and including your JSON resource from a <script src=…> ? So this is, in fact, CSRF?

No, CSRF isn't about pulling scripts etc from another site, it's tricking the browser into making malicious HTTP requests. So, it's not trying to grab facebook .com/someinterestingdata.json, it's trying to trick your browser into performing actions on the target domain by making it perform GET or POST requests such as sending Facebook spam. It doesn't matter what the response is, it's just interested in the action. A while(1) won't do much if it's inside an img tag or hidden iframe rather than a script tag.

Cool hack and awesome, fun writing, but on a more serious note : how is knowing your friends' usage/sleep patterns useful in any way? Could it be used for some dark, machiavélique purpose?

And how about for advertisers? "Get your sleeping pills here" type ads?

Another neat FB analysis:

Data Science of the Facebook World

http://blog.stephenwolfram.com/2013/04/data-science-of-the-f...

Small tip: If you don't want to be tracked, you can also turn off the chat.

Turning off chat doesn't necessarily suggest that is true. While `sticky_pool=atn2c06_chat-proxy&state=active` suggests the case, it's something to explore and see if there is a larger issue at hand.

Also if you aren't super prudent in turning of chat all the time, it would be trivially to notice those kind of anomalies in the graph.

Doesn't actually work considering Facebook shows "Active X mins ago" even if you're never "Active now" via chat.

This is crazy, I had this idea like 1 month ago and I thought I will find some free time to make this happen by the end of this month, I guess I should thank you :) Nice story by the way :)

I had the idea a month ago too. I even started working on this yesterday but got distracted. Thought I would do it today, but woke up to see this. This is even crazier.

Are you aware that there is actually no requirements.txt in your repo?

I see that it is fixed now ;)

Thank god HN doesn't have the same feature (ehh, I suppose at least). But how about linkedin and reddit?

I also bet that Whatsapp has this feature since I often see "Last seen at ...".

You can use the MQTT Facebook plugin for Bitlbee and get similar online/offline/active information right there in your IRC client without all the screen-scraping faff.

Nice little investigating!

Personally, I have chat off all the time on FB, and I don't have the Messenger (or FB) app on my phone either, so I guess I'm always sleeping :)

Love it. I knew where you were going right off the bat, but your writing made me read through it. Now I feel like a secret agent. I'm not sure why.

seriously though how are you not already using D3

This is getting downvoted, but I really thought D3.js was the most popular library for doing that kind of stuff? I'm a bit outdated I still use gnuplot :/

inb4 facebook resolves this issue by banning anyone who's connected 24/7. (that wouldn't solve the problem either way, btw -- a small group of people could conspire to pull this data at irregular intervals and then share the data with one-another to get a more complete picture while still staying reasonably undetectable if done right.)

mfw banned

so this is why I always get disconnected from Messenger!

You're really funny! Great post. Highcharts.js is the easiest js library to make quick charts btw.

This is brilliantly creepy, and so well written as to be both engaging and informative. Thanks!

Not sure if this blog post is great because of the thing he is doing or the way he writes it.

Very good post! And very well written, very humorous. Thank you for your teachings as well!

Very good and fun to read article.

Thanks for all this amazing info!

peace - [2edgy4u][ev REE DAI][24x7BLAZEIT]|ggg10Bzzz|

Loved the article. Great writing style!

Especially loved all the links to Facebook :D

Does this work for anyone, even if you're not their friend?

I would presumably work for anyone you are chatting with; that’s mostly your friends, but there are edge cases like people who asked to become your friends and haven’t accepted yet, friends of friends in a group chat, with whom you might be able to chat, etc. I haven’t tried though.

If you are inclined to, I would strongly recommend you create test accounts there to do so: https://www.facebook.com/whitehat

A. you get way funnier screen grabs, and honestly simpler controls; B. you will get love from the security engineers, which I’ve tried, and is an awesome idea.

As it says in the article, this returns a list of the statuses of your friends, he's not querying people individually. He is, however, storing the records individually, but that's a function of his program, not how the API behaves.

Hi, I wrote this blog post.

Nope, you have to be their friend.

Well, you have to be their Facebook friend. What you do in real life is none of my business.

I don't have facebook but my friends are on facebook.

This was hilarious and really cool!

Impressive. Kudos to the author.

p.s. I am hiring ;)

Great writing style. Hilarious.

So to make it useful it just needs to find the pairs of people who ,,go to sleep'' at the same time

This is random, but....

Nice with this data I might finally and truly finish my Social Alarm Clock idea and do so in which it truly improves the sound of your alarm clock; one that always makes you smile, laugh, etc.

There's been tons of social alarm clocks(from Justin Bieber to Nestle to Sony to Wakie, etc) since releasing sleep.fm in 2007 (a century ago in Internet years) yet no one has executed on the idea properly.

This is creepy, but yet very good illustration of social engineering

  Social engineering, in the context of information security,
  refers to psychological manipulation of people into performing
  actions or divulging confidential information.

I don't see how this is social engineering at all.

Because it's not.

Reply on news.ycombinator.com