Messages via JSON | Hacker News Reader

Messages via JSON

refl.me

275 pointsteko_ji124 comments

If you need something like that, go for a Telegram bot. It's really simple to make, the API is very nice and developer friendly. It's not oAuth, just a simple secret. It even accepts parameters via GET, so writing a bash script that check something and sends a message via Curl is pretty easy. There are few APIs simpler than this one. This is also more efficient, as your app only sends data out when it needs to, instead of being polled every n minutes. Everything goes through a real push service, which your phone has a persistent connection to, so the messages are delivered instantly. It also saves battery life, as there's no polling. I don't think this app will ever work on iOS, due to Apple's policies.

I think the linked app is more supposed for one-off use-cases - e.g., you're a dev/admin and want to keep an eye on some web service. You could write a simple endpoint in python and have the app poll it over LAN.

I think not needing to bother with any kind of third-party is hard to beat in terms of simplicity.

> I don't think this app will ever work on iOS, due to Apple's policies.

I've not much experience on iOS, so apologies if the answer is obvious, but is there no scheduled task API you could use?

At least, you're not supposed to keep the app working in the background in Android either - however, the app's use-case should still be doable with the background scheduling API.

I don't think a simple endpoint in Python is any simpler than sending simple requests to an external API. You can do that even via Bash, Curl and Cron. Regarding iOS, using APIs improperly is one sure way to have problems with app review, see i.e. the recent Push Kit debate.

But you have an external API - meaning, you have to get a Telegram account, register a bot (and not forget to unregister it later), think about who else can see the bot, think about if it matters that Telegram can see the data, etc. If you plan to keep this up for some time, you'll also have to keep up with Telegram's policy changes, outages, etc.

I created a Telegram bot for one of my apps, but had to remove Telegram from my phone/computer after their terrible handling of Facebook linking. I made the mistake of linking my Telegram account to my Facebook account, which gave me the wonderful 'feature' of getting a notification any time one of my (due to the linked account being promotional) thousands of friends joined Telegram. These notifications play at any time of the day or night, and there is no way to disable them. Removing the app was my only recourse as a chat app without notifications is pretty useless.

Turn off contact sync in Telegram and delete your uploaded contacts.

You can also turn off new contacts notifications which is something I also hated and discovered recently that you can do.

Thank!

Push is not always more efficient than pull. Pull puts a lower limit on traffic but it also puts an upper limit on it, plus you get an extra health check.

That's why many monitoring tools are now pull-based (Prometheus) instead of push (Graphite).

The update frequency can be set in JSON.

> Request signature format(crc): [time_stamp]_[md5(time_stamp+secret_key)]

This should probably be an HMAC construction at the very least, and MD5 in general just should be discarded entirely in favor of BLAKE2b, SHA-2 or SHA-3. And ideally, it'd actually validate the rest of the contents as well.

I think the author has made it as simple as possible for users and developers with basic skills.

While I agree with the notion of making things simple, security theater is worse than no security at all because it makes you feel safe when you may not be.

Maybe so.)

Quick and honest question: are you the author? You respond to lots of comments in the name of author, that's a rather surprising level of care about a niche piece of software.

Is that a good or a bad thing to do?

> Is that a good or a bad thing to do?

Posting comments in favor of a service without disclosing your’re its author is usually seen as a conflict of interest and, as such, a bad thing.

> Posting comments in favor of a service without disclosing your’re its author is usually seen as a conflict of interest

Not sure why, since it isn't in any practical or philosophical sense.

I judge the merits of a post by the content. I don't know if you're Celebrity X or Author of Y and I don't care much. Yes it would be useful to reference other statements, but that's something you often sacrifice in an open (even moderated) online forum. It's a strength of the platform as well.

Also the example PHP code doesn't validate the age of the passed timestamp so you could just replay a "signature" indefinitely...

Given that, you might as well just use an unchanging token or Basic Auth. Assuming https, that wouldn't be terrible for this kind of use case. But put it in an Authorization header, not in a query parameter, so it doesn't end up in logs.

Well, the app has to support it, so you're down to the will of the developer.

Sounds like a cool little thing, but wouldn't mind a standard to use on non-mobile platforms.

The documentation says it's a test of your choice.

There's another problem, if the author tells us that this is crc, while it's a MAC (not particularly strong tbh), it shows that we, as developers and users, cannot trust that cryptographic decisions of the author are of any consideration. For reference, crc is not a signature, its full name is "cyclic redundancy check", which is a really simple mathematical operation, that computes a specific remainder that comes from polynomial long division (of polynomials decided from the message). It is not any kind of cryptographic signature and given a string, it's crc and target crc, we can find another string that differs in one byte only, that has the target crc, all of this in time faster than computing whole original crc.

>we can find another string that differs in one byte only, that has the target crc

reading that has me curious; if one can do that for any arbitrary string, and then iterate that process, doesn't that stand to reason that with enough work, one could make any two given strings calculate out to the same crc? I guess maybe not because that one byte constraint isn't specified as far as where it occurs and whether it's an insertion, deletion, permutation etc, but the way I see it if you can do it with one string to another, you could likely keep chaining that indefinitely and get countless strings that come to the same crc.

sorry if this is old news or anything I was just struck by that thought while reading your comment

It can be done with any byte. In CRC, you encode your input to a polynomial somehow, take an agreed-upon polynomial (dependent on the type of CRC, not the input data) - call it P. Compute the remainder of the input when divided by P, encode it, and that's CRC. So to get any desired CRC, you just need to solve a linear equation (in quite a lot variables, but it doesn't really matter). All in all, it's really easy.

At Sun we had a server with a flaky NIC with very high error rates, but worse, occasionally would introduce two one-bit errors in packets such that all the CRCs involved were defeated. This actually caused some version control history corruption. I forget the details though. Maybe some other ex-Sun'er can chime in with the details.

The amount of bits you need to modify is not “one byte” but the same as the degree of used polynomial.

And producing two strings with same CRC is trivial to the extent that it is how you are originally supposed to use the algorithm. Notice that CRC-32 of every valid ISO9660 filesystem is 0xffffffff ;)

Yes, sorry for the lack of scrupulosity. Of course, it might happen it suffices to change one byte, especially that this byte can be at any position.

Your are correct in the "trust but verify" methodology, but the developer is Russian so it could be a translation error

Not one byte, but a number of bytes equal to the length of the CRC (typically 4).

But yes, people saying "CRC" when they really mean "checksum" or "signature" is a pet peeve of mine, and I treat it as a code smell. CRC has a precise defined technical meaning.

For the curious: CRC is linear with respect to XOR. This means that if you XOR two equal-length strings, the CRC will be the individual CRCs XORed together. It's also a bijection for messages of length equal to the polynomial (typically 32 bits): every input maps to a distinct output and vice-versa. Together, these mean it's trivially invertible for a message of length equal to the CRC: the CRC function can be represented as a square matrix over GF-2 (i.e., bits + XOR), which can be inverted with standard Gaussian elimination to produce the inverse function: generate a (short) string from any CRC.

More fun CRC tidbits: it's not a given that two arbitrary CRC functions, or a CRC and another linear(ish) checksum (e.g. ones' complement), are linearly independent. This can bite you when you try to use two different CRCs to get "more" error protection, or when you use "independent" CRCs to route work at two different points in a system. Again this is easy to verify using linear algebra (just check that the GF-2 matrix formed by the concatenation of your two functions is of full rank).

I've done it for people with any level of programming. People have already got used to that "CRC" is "checksum". But I will think about it.

Where did you find that?

Oh, I finally understand it! It is just an RSS (but with JSON instead of XML)!

I guess but that already exists https://jsonfeed.org/version/1

As far as I can tell, that's it. It's a reinvention of RSS without any notable advantages and no compatibility with existing systems.

So, a reinvented wheel that doesn't work on roads.

It's like re-inventing the wheel after everyone stopped using the wheel.

It might meet with some success. To people who don't know about RSS, it's a new thing.

> It's like re-inventing the wheel after everyone stopped using the wheel.

32% of Americans use RSS every month just for podcasts alone.

We can all thank Adam Curry for that!

I think everybody using Feedbin, Feedly, Newsblur, The Old Reader, Inoreader, etc. would wonder who your "everybody" really is.

Everybody would wonder who they are as well. :)

I think this is more focused than RSS. Doing the same with an RSS reader would be fiddly and sort of out of it's use case. If you want regular polling (every 15s say) maybe your RSS reader can do that but it isn't natural.

There is something to be said for things being similar but not the same. This app can grow features in the direction of alerting, whereas an RSS reader wouldn't be growing in that direction without becoming bloatware because it's main job is to catch you up on reading material.

I can see myself using it, and it has been something I have consider building myself - although I had a different angle - an API for a person so you can send me a message (if I trust you... maybe I give each friend a different token!) and we can avoid using email or a proprietary chat alltogether. Such a service could work in tandem with this app.

Based on that you could make a distributed "facebook" of sorts.

Once the iPhone app of this come out I can see myself setting up a little nodejs server and aggregating some stuff to send down. Weather, email, SMS, maybe some favourite google searches (when they change) or hn.algolia.com searches.

Thing is, you don't want regular polling every 15s! It's 5760 requests per day! For EVERY SOURCE! Trust me, your device battery won't like it. Won't like it at all.

> although I had a different angle - an API for a person so you can send me a message ... and we can avoid using email or a proprietary chat alltogether

If you want to exchange messages avoiding email or proprietary chats, you have a wide variety of open chat protocols and free and open source chat applications to choose from. XMPP, Matrix, IRC, mastodon, rocketchat, mattermost, ... dozens of them!

That can be a point of view but remember RSS is a fixed format and JSON not.

> RSS is a fixed format and JSON not

JSON isn't, but REFL.ME+JSON appears to be.

It would be cool to see a directory of public JSON feeds you can subscribe to - weather notifications, sports data, transit delays, etc. You could also have premium feeds that require a subscription.

Like RSS, but better

There is a nice substratum of curl services, like wttr.in or getnews.tech that follow a similar concept

I hope this is the beginning of a gopher + JSON + curl services era

So basically Yo :)

What's the difference between this and [Pushover](https://pushover.net)?

Pushover also handles a server, which you can make requests to, that sends push notifications to your phone. It looks like this project just scrapes JSON from a URL for changes, and notifies from that.

Pushover is push based, while this looks to be pull based.

You self-host the source of the messages in this case.

(I use pushover, and it rocks. But there you POST the message to pushover, here you host them on your own machine(s))

I think this is a simple and neat tool. Thank you for building it, and congratulations for launching.

Thanks!

Damn this is _such_ a good idea. Great job team.

I agree, just a weird name. Why REFL?

> short for "reflection"

https://medium.com/@re_dmitriy/what-is-refl-me-67e6eb151f27

i do not think “reflection” when i think “text me JSON events”

FWIW I agree.

It's pretty close to REPL (read eval print loop) so maybe the F stands for something else?

"reflection me" :)

What's the point? Why not just forward a message to XMPP, Telegram, WhatsApp or email? You know, to the app users actually use. Why make them install one more quasi-messenger?

Because they don't work directly with web services. Read the author's article: https://medium.com/@re_dmitriy/what-is-refl-me-67e6eb151f27

> The application with the required frequency directly checks JSON on the web service side

How charming.

What's wrong with that?

Not the poster but it may be useful to know that Andrew_nenakhov is a CEO of company that provides a Jabber messanger (Xabber).

I'm glad.

My occupation is not really relevant here, but yes, such service could be coded in maybe 60 minutes tops to accept a JSON and forward it to XMPP, and it would be way more efficient. Don't like XMPP? Ok, make a Telegram bot and receive notification with it.

Because it is the most inefficient approach possible. You either have potentially long delays before receiving a notification (if you poll service once an hour, average delay would be 30 minutes), or you do LOTS of polling, draining the battery/abusing the service. If you have 20 notifications / day and you check service every minute, in a day you'll make 24*60 poll requests, only 1,38% of which would yield some meaningful payload.

And yet, this is exactly how push notifications are implemented on mobile phones. You have the illusion to push a message from server to device while in fact you also push a message onto a queue that is polled at regular intervals by the mobile. No way around it, mobile devices don't have guaranteed connectivity as servers do.

> No way around it, mobile devices don't have guaranteed connectivity as servers do.

Not sure what "guaranteed connectivity" has to do with it, your phone is connected to network ~90% of the time, and when it is, it has a socket connection with the Google Cloud Messaging API (or iPhone equivalient) which pushes down data as it comes into the queue. If messages come in while you're disconnected, you get them all the next time data is pushed.

Ah but sockets are only an illusion that fall apart under the next tunnel. Mobiles are not constantly connected, they use a wireless packet-based network. The only way your mobile is registered as part of a network is because it is the one polling stations at regular intervals. Can't go the other way around, really. How could the network know you are now out of that tunnel? You may want to have a look at how APN are implemented on iOS to see how a push is in fact a poll from device.

No. See my answer above. Mobile devices on both iOS and Android have a built-in XMPP client that reconnects to APNS/FCM servers every time it regains connectivity.

If they’re using XMPP like others in the thread mentioned, they’re likely using STUN/TURN for signaling. IIRC according to google, something like 10% of hole punched traffic needs to be routed through the TURN server due to symmetric NAT. For all other types, a nearly persistent connection from server to client is possible with STUN/TURN, since a client can resignal when its connection changes.

No. That's not how it works.

From the Apple Developers Documentation [1], "On initial launch of your app on a user’s device, the system automatically establishes an accredited, encrypted, and persistent IP connection between your app and APNs."

This persistent connection is, in fact, an XMPP session (likely, a modified one) [2].

Google's FCM too uses XMPP [3].

[1] https://developer.apple.com/library/archive/documentation/Ne...

[2] https://www.quora.com/What-technology-does-the-iOS-Apple-Pus...

[3] https://firebase.google.com/docs/cloud-messaging/xmpp-server...

If you look at the format of APNS messages, they are nothing like XMPP. It seems very unlikely Apple uses it. That link you post does not confirm it.

Format of APNS messages is just a payload. The information about APNS being based on XMPP was widely available on developer.apple.com and was often mentioned in developer sessions in 2010..12 when I was actively following it.

The current absence of direct XMPP mentions on Apple Deveoper website might mean that they consider their internal technology not important for developers, or it might mean that Apple had changed the protocol for some reason (they could have opted to use some binary protocol, for example), but claiming that push notifications work on iOS devices by polling servers is beyond ridiculous. VoIP pushes arrive in ~1 second. Such response times would require 40000 poll requests/day or more

I never claimed APNS worked by polling. I was developing iOS apps in the 2010-2012 time frame and never saw a mention of XMPP. I simply don't believe they are using it. They must have their own proprietary solution.

> I never claimed APNS worked by polling.

I was referring to earlier comments by nicolas314, sorry if it wasn't clear.

> I was developing iOS apps in the 2010-2012 time frame and never saw a mention of XMPP

I might have been more attentive to these details because XMPP was relevant for my projects. Anyway, Apple definitely does use port 5223 [1][2] for push notifications. What protocol is known to use this port?

Speaking of proprietary, it is very likely that they have modified it (by stripping it down, mostly). Many XMPP features are excessive or inefficent for push notifications, but core functions - message routing and persistent TCP connection are extremely well suited for serving several billion devices.

[1] https://support.apple.com/en-us/HT203609

[2] https://support.apple.com/en-us/HT210060

There is no need for frequent updates for a lot of data. This application is not for instant messages.

> Because they don't work directly with web services.

Then why not make a relay, i.e. a server that forwards messages from a webclient to Whatsapp, email, etc.?

Do you realize that's the opposite direction of this? In that case you SEND something to a chat app, but this app can RECEIVE data from a web service. Way simpler architecture.

> Do you realize that's the opposite direction of this? In that case you SEND something to a chat app, but this app can RECEIVE data from a web service. Way simpler architecture.

Well, it receives data by hitting a specific endpoint with a specific protocol. It’s “simpler” in that it may be easier to implement, but you still have to modify your service to get that to work. And if you must modify your service, you may want to implement something that’s better supported and doesn’t require users to install (and configure) yet another app.

This would be nice for websites and services users want to get push notifications from. I get that html5 notifications should be able to do this at some point but there is still merit in having another way of getting stuff to your phone and possibly desktop.

Thing is, it does not offer push notifications. It's very opposite of that - the app seems to just poll the service with regular intervals.

Yes, but the interval is set in JSON, which is convenient. Besides, there is a manual mode.

Awesome! I had been doing this via signal-cli, but their aren't great tools for working with Signal's API programatially and my solution was quite hacky. (I used a Google Voice number to register Signal, and parsed the output from `signal-cli --daemon` to recieve messages, and used Python's os.system to call the binary to send messages.)

Amazing idea. For my job, I have all sorts of instrumentations on my web services for monitoring sake. However, I have none for my pet projects, and this is just the right amount of monitoring I need. Now, I can wait for the iOS version or perhaps just roll my own app; It is a very neat idea, which I can see helping me in tons of scenarios.

Help me out because sometimes my reading comprehension is wonky.

So is this just polling an api that responds in JSON and then updating your phone if something in the response changes?

Nice idea. Does it poll though? How do you prevent receiving the same message twice?

I would have done it using server-sent events instead.

I think it does poll, and it's just looking for a diff in the previous message.

Curious, what are some examples of public json files I would "subscribe" to?

My understanding is that you would be making your own. E.g. if you had a site that people could comment on, make a url that serializes those comments as JSON and then point this at that.

As pointed out elsewhere by Andrew_nenakhov, we have that already—it's RSS. Or Atom. Or JSON Feed if you really must have JSON for some reason.

Relevant xkcd: https://xkcd.com/927/

There's no simplicity. REFL.ME is easier. Try it.

  $json_array = array('reflapp'=>true,'message'=>'test');

shouldn't that be

  $json_array = array('refl.me'=>true,'message'=>'test');

It's already been fixed. Thank you.

I think this is basically Slack webhooks without Slack. Not a bad idea, actually.

> The "refl.me" key must always be true, otherwise an error will be called.

Why?

Wish this wasn’t the case, would love to integrate services I don’t control into this.

I won't pretend it's "so easy" or anything like that but could you just proxy those requests and add the key yourself? AWS Lambda comes to mind as a cheap/free way to accomplish this with the benefit of letting you modify the data if needed on the fly (or even check against a DynamoDB for history/other rules).

Wouldn’t this defeat one of the benefits of this software, the need to not host your own server (or any backend infrastructure)?

I think the value-add with this service is the notification sending just by exposing an endpoint. I don't see any mention of not needing backend infrastructure (they even call out "connect your web service").

I think this is aimed at people who already are or can expose data via an HTTP endpoint and want notifications when that changes/updates. This includes the wide array of software developers who work on large OS projects that support sending notifications via services like PushBullet/Pushover/etc. Also, presumably, web services would just add a new endpoint with some auth token to easily send push notifications to their users without managing an app and everything else needed (push tokens, certs, GCM/FCM, apns, it can be a headache, trust me).

Personally I see pushover-type services (I use pushover so I'm only speaking from that experience) as more useful since I can "push" out of my networks much easier than a service can "poll" me. That said pushover has always been a little clunky IMHO with trying to get a service to send me notifications. So a simple "Paste this url into the refl.me app to get notifications from us" is a pretty attractive alternative for certain use cases.

Proxy them, add the key.

[EDIT] damn, beaten to the punch.

I’m curious how many services you want to integrate match the format refl.me expects (namely a JSON object with at least a “message” property).

It seems likely you’d need to proxy most services anyway. With numerous FaaS providers these days that seems trivial.

My guess would me as a sort of "signoff" from the person hosting the JSON that refl.me is approved to use their endpoint. It sidesteps a slew of abuse issues by either knowing it's a feed you own OR a feed you are re-hostings/proxying and so the liability rests with you.

In my mind it's in the same category as google throwing garbage or while(1){} loops at the start of their JSON responses to prevent XSS JSON reflection attacks. I know it's not the same thing at all but idk, thats what comes to mind for me.

EDIT: Of course I think of a better comparison as soon as I hit submit: Sort of like LetsEncrypt looking for a .well-known/acme-challenge to validate your domain.

Yes.

What exactly is new in this ? Isn't this what Android and iOS notification services do ? And all messaging services ? What did I miss

Any web service can integrate with it as long as it responds with a JSON object that adheres to the specification.

It is genius.

Still didn't get the genius part,... This simply keeps polling a list of endpoint and if a new message Id comes, flashes it and stores in the list of flashed messages ? That's all right ?

It's a mobile app that polls an endpoint for json, after the user configures it (seemingly with a QR code), that's all. There's no middle-man server, and it's also not push.

Neat idea, low barrier to entry. Can't tell from docs how authz/authn are handled.

And who maintains the message queue until the client pools next(the service provided by these middle-man servers) ? And what about message by user identifier ?

And client app will eat my battery because one more service is constantly polling now ?

You can configure the refresh interval in the service response: https://refl.me/docs

Would be nice if you could override this client side as well.

True enough, this is in concept the very same as Apple Push Notification (APN) service. Difference is that you are in charge of the server side and handle all the security and uptime yourself. Very cool idea, simple enough to generate a lot of really cool projects.

No. APNS works differently and iOS devices do not poll Apple's server at frequent intervals. I explained it here [1]

[1] https://news.ycombinator.com/item?id=21338838

The difference between push and pull.

Push notifications function on the principle that when a state change happens, the notification is pushed to the device.

Pull notification is when device polls some service for data, detects change, and then emits notification if change has occurred.

There are benefits and drawbacks to both.

Im not getting notifications with the test_crc endpoint. Key is correct, Pixel 2 Android 10 latest version.

Why isn't my device compatible with this version? What versions are they targeting?

Also, this could be just a html page with a few lines of XMLHttpRequest/WebSocket code no?

Is this thing russian just for me? Couldn't find a way to change the language.

Click the "RU" link in the top corner, then click "English".

Or just go here: https://refl.me/?lang=en

No, I mean the app itself.

Update the app. Error corrected.

Thank you! :)

Simple and interesting idea!

Thank!

Reply on news.ycombinator.com