Show HN: An anonymous P2P social network for Android, written in Clojure | Hacker News Reader

Show HN: An anonymous P2P social network for Android, written in Clojure

nightweb.net

257 pointsgw133 comments

Lein-droid author here. A few commenters asked about the performance and maturity of Clojure for Android, so I decided to answer in the top level.

The main problem of Clojure for Android right now is the large size of Clojure runtime. This leads to two issues: a) long loading time (can take a few seconds even for the basic app), b) applications often gets closed by the OS once switched to another, in order to free that memory.

Since ProGuard is no help in optimizing Clojure-compiled code, the issues should be addressed in some other way. One that is possible is to write source-shaking compiler that will strip the final APK of the unused functions. It's pretty hard to do with the current compiler, but if we eventually write Clojure-in-Clojure compiler (there is a proposal for this year's GSoC by the way), it may be feasible to add these optimizations to it.

And finally a shameless plug: if you are interested in trying Clojure for Android, here are the links to get you started: https://github.com/alexander-yakushev/lein-droid https://github.com/alexander-yakushev/neko

I have been waiting for this.

Here's my Nightweb link for anyone interested in experimenting with me: http://nightweb.net/#type=user&userhash=5avfzwgkdzgddsv3...

gw: It would be great to have multiple personas available to me, like an inverse of google+ circles.

http://nightweb.net/#type=user&userhash=fcx2ahb4vuobu7ej...

That's mine ... awww feels like the beginnings of ICQ ;)

But ... we can hack on it ... finally a good reason to learn Clojure.

Exactly, I've been both wanting to learn Clojure and develop for Android (but no Java thanks) for a while.

So here's mine http://nightweb.net/#type=user&userhash=dtvjjjuk7eai42xx...

Here's mine, for science:

http://nightweb.net/#type=user&userhash=tnaohxmxj2eg3fsf...

Here is mine: http://nightweb.net/#type=user&userhash=bicbgdgdxvv5ifxa... kj2voujr7vk

Me too! http://nightweb.net/#type=user&userhash=psvmye7nw7mp53yp...

Jumping on the bandwagon as well! http://nightweb.net/#type=user&userhash=wlvkr6lmjrz7stgf...

And here is mine http://nightweb.net/#type=user&userhash=7whx46v3h2ehiiei...

And mine!http://nightweb.net/#type=user&userhash=trz3t72fcwwqzdi4...

http://nightweb.net/#type=user&userhash=utvlt5hh6o5bjqil...

Hello World/Hackernews

http://nightweb.net/#type=user&userhash=h4e7zymwpdvsdjxs...

Just passing by..

http://nightweb.net/#type=user&userhash=a4ehmznzujb4iiai...

Here's mine: http://nightweb.net/#type=user&userhash=dwo2f2ieza3t2koa...

Mine too: http://nightweb.net/#type=user&userhash=prn3nvnrikl7ewpf...

and mine: http://nightweb.net/#type=user&userhash=prn3nvnrikl7ewpf...

http://nightweb.net/#type=user&userhash=ownwrsbcevb3pjqg...

Although tying these identities to our hacker news IDs basically means throwing anonymity out the window.

Interesting, here is my link. I think this will suit my nature

http://nightweb.net/#type=user&userhash=q3ytnh5ci6cc72gt...

You wanna get things done in Clojure, you're gonna have to touch Java sooner rather than later.

On the other hand, I didn't actually learn Java. Knowing C++ and reading documentation seems to be sufficient for using Java libraries with Clojure.

http://nightweb.net/#type=user&userhash=lj7zda4sk3b7xjtj...

Thanks very much. By the way, currently clicking the link in a browser doesn't seem to bring up the intent chooser to open it in the Nightweb app. I'll try to fix that today. In the meantime, the workaround is to email or text the link to yourself, and click it from there instead.

Yeah I've thought about allowing multiple users on a single device, so I'll look into that in the near future.

EDIT: I fixed the issue with clicking links in browsers. I will issue version 0.0.5 with this fix because I think it's pretty important.

you should define a separate namespace for users/posts, like a subdomain or path, for the app, right now the browser is confused when opening the homepage

I think it deserves it's own protocol nightweb:// . Will also make it easier for th iOS port later ;)

Using a nightweb.net http url on the other hand would allow a read-only web access to the service, increasing conversion rates ;)

But it would undermine the "distributedness". Maybe Nightweb could serve a website on the device itself and one could use pagekite or a dynamic dns to serve it.

There's no reason a third party couldn't host a read-only view into Nightweb, performing something similar to tor2web.org.

The third party would be running their own Nightweb client, then visitors to the website would trigger content downloads on the client, rendered out as a webpage.

This would provide the "public view" without sacrificing the decentralized nature.

Great idea.

In fact, this type of functionality is hopefuly becoming an Internet Standard: http://www.internetsociety.org/articles/moving-toward-censor...

However, Bittorrent is not up to the task, due to NAT issues. UDP NAT puncturing is not present in this stack, right? Thus how is it going through carrier-grade NATs which block all these smartphones?

Disclaimer: working on a similar thing for years Tribler Mobile.

First, the website would only be meant for users without the app, so local hosting would not work. Second, it can only be read-only, unless you upload your private key.

I think it would provide a window into the social network, kind of "Oscar is using this, join him now by downloading the app!"

The implication I see is privacy of people clicking a user link, but the problem already exists now.

I think it's considered bad practice to define your own URI scheme for your app, though it is very common on iOS since you have no other choice. I thought using HTTP links would be nice since (a) they are much more likely to be linkified when pasted in emails and on websites, and (b) clicking them from non-Android devices will take you to the website where you can learn about it.

You could also have a context header on top saying something like "Join [User] on Nightweb" or "Hey, [User] is on Nightweb!" and then your website follows explaining the concept.

I've also been waiting for something like this. Glad to see it's finally here.

http://nightweb.net/#type=user&userhash=aibbyarsuvbdttxc...

Could you write a blog post about building the mobile app in Clojure?

I would love some comments on the maturity and performance of Clojure (speed, load times, memory consumption, executable size...) for Android apps.

While the app is interesting, I am also more interested in Clojure's feasibility for writing Android apps. Last time I looked, Dalvik's GC was so bad it took many seconds for the Clojure runtime to bootstrap.

There is also the possibility of using and wrapping the Apache Cordova (phonegap) API from ClojureScript. Trade-offs galore there, but one advantage is faster (still not comparable with native) startup time.

Things have improved quite a bit recently. Daniel Solano Gomez is the pioneer here, and Alex Yakushev made it a lot simpler by creating the lein-droid plugin for leiningen last summer.

It will impose a few seconds of load time no matter what, so it is best for long-running apps where loading time isn't as important. The cost in runtime speed, memory consumption, and APK size are also present, but aren't a big issue in my experience. The same Clojure optimization rules apply.

I think things will get better and better once more people use Clojure to write Android apps. It will be a signal to the core devs that it's an important platform. Currently, you must use a fork of 1.5 that includes a few necessary modifications.

I've just tried NightWeb and the startup latency is very high, even on my Nexus 4. Sadly I can't think of an app where 5-10secs of latency is acceptable :(

Yeah I will definitely work on this, I apologize for that.

If you manage to get a Clojure Android app to boot up quickly, I would be very interested. For now, the only reasonable ways to not use Java for Android apps that I've found are Scala or Xamarin.

Here goes: http://nightweb.net/#type=user&userhash=4zfhmwiio7kdp5oc...

How does the protocol differ from, say, tent.io?

Just in case anyone has the same issue: on my device (Nexus 4) it took a while for the UI to become responsive when first opening the app. Patience solved this one :)

The word "decentralized" is a pretty loaded term unfortunately. I think tent.io and other similar projects are decentralized in the sense of who controls the servers, but they are still fundamentally client-server protocols. Nightweb (and I2P specifically, which it uses) is fully peer-to-peer, so there is no distinction between clients and servers. I think that's the most important difference.

Sorry about the non-responsive loading, I will work on that in the future.

Does this have a particularly adverse effect on phone battery life, when compared to a well-written app for a centralized social network (aka, not the Facebook Android app; perhaps the G+ app)? I can imagine that a lot of uploading of social network content to your peers would take place.

Also, what is the bandwidth utilization like compared to an app for a centralized network? With a lot of us on tiered data plans now, this is a crucial concern.

I ran the app for most of this morning on my Nexus 4, after adding most of the links in this HN post as favorites. Between 9 and 11 my battery lost 40% of its charge. Ordinary usage tends to yield 14 to 17 hours of battery life from a charge. This kind of impact on battery life will be a deal breaker for most people until resolved.

Spotify's desktop app is P2P, but their mobile app is server-client. Perhaps advanced users could mitigate the battery life issue in the short run by setting up a "proxy" on a server that joins I2P, while presenting the front-end on a mobile device -- like you would with, say, an IRC bouncer.

Fell out of my chair reading those "ordinary usage" numbers. Reading a book for a couple hours drops my N4 by 50%, easy.

I own a Nexus 4 and agree with the parent poster. 14 or more hours mostly idle is normal. I get even more if I dim the screen for normal use.

Fortunately, Android JellyBean provides all the data you need to fix your problem. If you are getting worse battery life, you need to check Settings -> Battery and determine which apps are unreasonably eating up your battery life.

"Nightweb starts its life by generating a DSA-SHA1 key pair"

Just a note for the authors, as of 2010 NIST recommends not using SHA-1 for new digital signature generation and "disallows after 2013". See section 9 here: http://csrc.nist.gov/publications/nistpubs/800-131A/sp800-13...

Yeah I chose that because I2P uses it. The devs are already discussing upgrading, and once they do I will upgrade it in Nightweb as well.

Android 4? That's a pity. But why a mobil app? Why not a .war app for the i2p router? IMO Social networks are something that are mostly used on desktop computers.

Also it seems to me like you want to fork i2p. I think that's a very bad idea. First since you should be very confident in your skills if you roll your own crypto. Second because a split seems not necessary and harmful in the long run for the network. The i2p networks could be bigger aswell, and this could help.

I didn't actually modify the core or router code at all, I only needed to modify the I2PSnark code (i.e. the BitTorrent component). I did not roll my own crypto at all, other than the digital signing of meta torrents which is an application-level concern. It is technically a fork because I have a separate copy of the I2P codebase in my repo, but I will always update it to the latest versions as they come out.

>but it will pay big dividends later if I need to rewrite parts of the I2P backend.

I'm all for clojure, open source, innovation and forking. But i urge you to stay part of i2p and not create incompatible releases. Maintaining your private branch will result in a lot of extra work, and I'm 100% sure that this will result unfixed security holes. The i2p project on the other hand would probably gladly accept your java patches.

Understood; I actually was moreso referring to rewriting I2PSnark, not I2P itself. The BitTorrent codebase is pretty old and I know I can improve it if I had time. I agree that touching the core/router code itself is dicey, but keep in mind the I2P developers themselves have encouraged people to make alternative implementations. I don't plan on that right now, though.

I think the I2P community would be thrilled about I2PSnark patches. I2PSnark is in active development currently, as the I2P update mechanism is being moved into I2PSnark.

Personally, my use of messaging and social apps happens almost exclusively via mobile. I'm writing this reply from the excellent Airlock HN app even.

I doubt I'd give Nightweb the time of day if it necessitated a desktop.

IMO Social networks are something that are mostly used on desktop computers.

50% of Facebook's traffic was mobile one year ago and I'm sure it's even more now.

I wonder how they're going to get around the problem of malicious collectives and Sybil attacks, given the anonymity constraint. You should be able to set up dozens of fake accounts to vouch that you're a "good" person, or that someone else is a "bad" person, use astroturf tactics to wage malicious campaigns against other parties, and basically sow havoc.

So far it is only distributed twitter ... you don't have control about who's following you. The network does not really have a trust component yet.

This app is pretty well polished and from an ideological standpoint very awesome.

If you have any feedback/requests for I2P, I'm sure that zzz would be very interested since he did all of the I2P@Android work. Catch him on zzz.i2p or #i2p-dev on the I2P irc net.

Yep I've spoken with him a few times on his forum. I announced Nightweb there a week ago but I don't think I ever got any replies. I am definitely willing to incorporate any changes he suggests, as he knows the I2P codebase as well as anyone.

http://nightweb.net/#type=user&userhash=6pocouo5pxv6wwnt... is mine, though I can't promise I'll do anything with it. ;P

Here is mine URL http://nightweb.net/#type=user&userhash=eyynvzrb4npqnrrt... I decided to add a photo as well

http://nightweb.net/#type=user&userhash=lzk6fkyifc4zhjls...

Here is mine. Great to see new work and so clean out of the box.

Why do you say it's a "killer app" for Android? I think it's a great idea, but it does not use the fact that it lives on a mobile phone, so it's certainly not a killer app for it.

Can we haz a desktop or web version?

I only saw this on their website:

    > The goal of Nightweb is to be a "killer app" 
    > for anonymous networking

>> > for anonymous networking

It suggests me to share my profile and posts with others(with rest of sharing apps like Twitter). How am I anonymous then?

Besides, after installing the app and creating a profile - I just had to enter a name, how do I make first contact? I searched and it didn't work.

Looks like there's sth that I am not doing right.. ?

You have to find people externally, such as through the links people shared on this page. There is a bug preventing links in a browser from being openable in the app, but I just released an update that fixes this. There is no way to discovered people internally right now, though I did make it auto-add my user so you can at least see some content after it runs for a little while.

I don't think of it as "anonymous" networking, but more as a network that can't be controlled or brought down. Especially by relying on bluetooth and future, improved peer-to-peer connection technology, we could be free from carriers and ISPs.

The main advantage of the application running on phones is that everybody has one (hyperbole intended) and that phones are always on.

For a web version to be fully distributed it would need to be run on a server, so people would have to hook up a RaspberryPi at home, rent a VPS or trust some big Nightweb farm (the diaspora model). It's infinitely easier for most users to just install an app on their phone.

A Desktop version would probably not be "always on", which would limit the usability as a social platform seriously.

I always had the idea to create a distributed social network, I'm looking forward to seeing how this goes :-) Hopefully very well, would love to delete my facebook account.

Great stuff but will it really scale? See secushare.org/2011-FSW-Scalability-Paranoia

Did anyone try to save Nightweb's data directory encrypted in Dropbox or ownCloud?

Would it be possible to add something like a reply option to a post? This would give us twitteresque chat.

Would love to be able to share posts with certain persons only.

PS: Can someone provide a download link for those avoiding Google Play? TIA

It would be way better if the author just provided a download link on it's page, but in the meanwhile: https://www.dropbox.com/s/g98p4nn2j33tr8t/net.nightweb.apk (version 0.05)

You can already reply to posts. Just hit the new post button when viewing a post, and your post will link back to it.

I'm considering adding an APK download link, but my site doesn't use SSL so I didn't feel comfortable doing it there.

Also would be great to have it in F-Droid repo (http://f-droid.org/). They have direct APK download over https.

One thing that wasn't clear in the protocol or explanation page was the favorite-of-a-favorite discovery.

I'm assuming this works by including your personal favorites in your meta torrent, so when people get your new content, they also get a list of who you've favorited?

Does this also mean that the meta update notification gets sent out when I favorite / de-favorite someone?

Yeah, your favorites are stored in your meta torrent, so it will be updated every time you add a favorite user or post.

Thanks, figured that was the case.

Nice take on announce_meta, btw. I had mused over something similar (though not as well fleshed out) to the Nightweb concept but kept bumping my ahead against the notion of how to announce additional content when the bittorrent protocol ties info hashes (and magnet links) to a direct hash of the content - meaning you could never update something already announced for a specific key. And since the hashes are content based, it's not like you could say "my next content will be <<previous hash plus 1>>".

Extending the protocol to include a new announcement method side-steps this very nicely :) Kudos.

Very intriguing. I was just beginning research into a server-less shared state model and this seems to solve that quite nicely.

>How does it avoid spam?

For me a good solution would be to use machine learning, the same way spam filtering works for email.

A decentralized version of Reddit could actually be pretty simple, with a flooding P2P network coupled to some machine learning on each of the nodes.

The beauty is that anyone would be free to implement the content filtering measures the way they like it.

How would you discern what is popular and what is not(that is, the "upvotes" of reddit/HN)? Or did you mean only machine learning?

People could optionally broadcast their pseudonymous votes, using cryptography to avoid impersonation.

That would actually be a lot richer than the current "one vote is one vote" systems, since you could learn which people you tend to agree with.

Could this be a practical tool for groups such as Occupy Wall Street and the various groups of "Arab Spring"?

This is so cool. It makes me want to switch to Android just to use it. Any thoughts on a desktop client?

Ditto. Definitely seems like it has the potential to be the Killer app based on the viral potential of P2P, anonymity and widely adapted OS platform. All the best.

I am definitely considering making a desktop client. I can re-use all the backend code but will have to make a UI from scratch, so it won't necessarily be soon.

Second the motion for a desktop client

I _have_ an Android, but I dislike typing on it. For me, it's a phone, period.

Not yet indexed by Play Store, but can be found by going to the http url. Oops, the problem is: My 2 T-Mobile HTC 4G phones are not compatible with this app. Neither T-Mobile Samsung SGH-T679, and Vodaphone Samsung GT-I9000. Only works on Asus Nexus 7.

It requires version 4.0.3 (Ice Cream Sandwich) and above, so that may be the issue.

Guess it will not turn out as killer app then. This suggests other fixes, such as apk sizes < 50MB: http://stackoverflow.com/questions/10475954/why-does-the-goo...

It's not like it will suddenly see mainstream adoption in any case. The project is young and the early adopters will be the nerds and the geeks who don't mind the bumps. It's always like this. Organically it can spread and by then it could both be a robust app and have prerequisites pretty much anybody fulfills.

> the goal is to have uncensorable, untraceable communication and file-sharing on mobile devices."

This is really interesting. How is this uncensorable? From what I've read, i2p requires defined servers to resolve addresses which themselves can be taken down.

I do not use I2P's domain name system at all. Everything is just fully-qualified base32 hostnames underneath.

What do you mean by base32 hostnames? ips? If you mean ips, then, especially on a phone, wont the ip change often. If you mean some actual hostname (presumably not the reverse lookup) then who translates that hostname? If it is provided by nightlyweb servers, then if that server gets taken down, you have no more network, right?

I mean the hostnames used by I2P itself. They behave like normal hostnames, but they don't require any DNS-style resolution because they contain your public keys inside the hostname itself. They are equivalent to .onion addresses in Tor, except a lot longer.

I understand that, you are saying that you have an id of who you want to talk to. The problem is, where do you start looking? Even with .onion you need an entry point to the tor network. It can be a DNS server, a web server, or whatever; but it must be hard coded (unless you start broadcasting on the local network looking for an entry, which I havent seen an example of). That is the problem. The first entry point can easily be taken down.

You have to bootstrap onto the I2P network initially. There are several IPs hardcoded into the router by the I2P project, which are run by volunteers. If these IPs are blocked in your country, you are out of luck.

There are a few possible solutions to this. First, allow users to add new bootstrap nodes to the app. Second, provide "meshnet" functionality to the app. I am considering both. The latter is much more complex to implement, but Android provides an API called Wi-Fi Direct which I could conceivably use to connect Nightweb nodes together if they are in close proximity. I'm open to other ideas.

That does not make it inherently uncensorable. China can still look at your traffic, go 'this is nightweb traffic' (or even just bittorrent traffic, I doubt they care about that), and block the communication. Bam, censored.

First, all they see is that is I2P traffic(or rather they see it's encrypted, as the protocol has no magic bytes anymore afair). And secondly they haven't done much work in regard to blocking UDP afaik, so unless that has changed, they can't really censor I2P once you got it up and running.

I've joined, here is my link for anyone interested: http://nightweb.net/#type=user&userhash=3c6mdngahryip52n...

Same.

http://nightweb.net/#type=user&userhash=xcrwasje7ojqdf3k...

Wow, this is just awesome! The app is really polished for an early beta release, especially that hidden markdown feature.

Just three things that come to my mind:

1. how long does it take to propagate updates?

2. what is the traffic consumption on mobile?

3. what about statusbar notifications?

Once you've bootstrapped/integrated with the network, it takes a minute or two for a new update to reach you in my testing. The initial time to bootstrap onto the network is around five minutes.

The traffic consumption is pretty high, because it becomes a full peer on the I2P network. Over a month I would expect it to consume around 1GB or more. If this becomes an issue for people, I may add the ability to be a non-contributing peer.

You might want to add a status text to the service notification, displaying how far bootstrapping is..

Please make the non-contributing status the default then, or at least ask the user on installation... capped data plans are still the default in many places... :(

Do android applications have access to their amount of data used? Maybe it would be possible to cap the data usage or lower the bandwith when a limit is reached.

I think a setting to only use WiFi would be valuable for a lot of users.

It might make more sense to switch between the type of peer based on the type of network connection. If the phone is currently connected to wifi, then run as a contributing peer. If the phone is only on 3G/4G/LTE/etc then switch to being a non-contributing peer.

Definitely agree with the option to become non-contributing, or at least introduce some limiting - it's a struggle to get a data cap of higher than 500MB here. Speed doesn't seem too bad, even on 2 year old hardware. Is there an (easy) method for transferring your key to another device, and conversely if there is such a way, is there anything to protect the file if someone was to get access to the device storage?

Yep I already added data export. Just click your profile and there is a button called "Export..." which will ask for a password and create an encrypted zip file.

Does this allow for two devices to be "logged in" to the same profile, or will one become the primary while the other can't assume control at all?

Please bring this to FDroid! In the meantime an APK for download would be nice.

had the same idea a while back and was pondering how I would get past NAT if you are on a mobile network. does it work there as well? I'm assuming it will continue to work as long as your carrier doesnt block the respective ports.

also, don't take this personally - but if you're looking at targeting a broader crowd, you might have to update the UI to make it look more like traditional social apps.

There are IETF standardized NAT traversal mechanisms, see http://en.wikipedia.org/wiki/STUN#NAT_traversal_solutions

I don't know how widespread NAT is on mobile networks, my operator just started using it and I've been meaning to switch.. not a real internet connection anymore IMO. And lots of people are doing self-inflicted NAT on their home wifi. So it might actually be that mobile connections are less likely to be NATted.

Is there country restriction?

It says incompatible for my country

Do you mind telling me what country you live in? I selected every possible country in the dev console so I'm not sure why it would say that.

OT: One thing I don't yet understand is why security-critical software is usually written in C instead of a language like SML.

Anyone else having trouble adding pics? They seem to attach properly but don't show up after I hit send.

I'll look into this.

EDIT: Ugh, yep, a bug I introduced recently. I am issuing another update, 0.0.6.

Great!

Nice! But i have another problem, I upload a picture and this turned one left.

Can you elaborate? I'm not sure what you mean.

Hmm, wait. If I upload a picture file, this rotate 90° left. (See screenshhot) http://s14.directupload.net/images/130401/asanfblz.png You now what I mean?

I can concrete the problem. If i take a picture with my smartphone camera, this pic rotate. If i take a screenshot with my smartphone this pic don't rotate. I use a samsung galaxy s3 mini with android 4.1.2. I hope it helps (Sorry for my bad english) I'am native Kraut speaker ;)

Thank you for narrowing that down for me, I'll try to fix this soon.

No prob, from dev to dev. I like your software, the clear and minimal design and, sorry, why isn't a btc donate code on your project page?

I'll consider that in the future, but I'm not in a rush to solicit donations right now. If I decide to quit my job and work on this full time, I will definitely start begging =)

I tested this behavior a little bit, it does not apply to the profile picture but when you submit an other picture it appears to be rotated 90° left. So I manually rotated my pic 90° right to counter this, unfortunately the picture appears exactly the same way as before so I suspect it to not be just a rotation 90°left

EDIT: here's a screenshot http://i.imgur.com/efngFHN.png (on the picture I uploaded the sky should appear at the top)

I'll look into this; it hasn't happened on my phone but hopefully I'll figure it out when I get home from work tonight.

how do you solve the p2p user_id problem? everyone can be everyone when faking the id on some client.

It's derived from your public key, so only those with the corresponding private key can update the meta torrent for your user.

Definitely looking forward to using it and reading the code. Landing page is profesh too. Well done.

Have you considered Cjdns, an alternative to I2P that does not require the internet?

I wouldn't say it's an alternative, because CJDNS and I2P operate at different levels of the stack. In other words, you could theoretically run I2P on top of CJDNS. I like the project very much, though integration into Nightweb isn't likely because it requires the creation of a TUN device, which is a kernel-level facility and thus requires root permissions.

Anyone having trouble attaching pics to posts? They just vanish after I hit send.

This app looks awesome. Reminds me of mit roof net. Can't wait to try this out

Reply on news.ycombinator.com