MediaGoblin – Self Hosted, Decentralized Alt to YouTube, Flickr, SoundCloud
mediagoblin.org
mediagoblin.org
So far, I installed my own Matrix server (synapse), my own XMPP (ejabberd), yesterday I got semi-happy with my mastodon setup, and now I was just finishing some tests with ownCloud to see if I could replace Dropbox.
MediaGoblin is on my list of services to setup. I ran a basic deployment and checked some other instances of it before, but I didn't put it higher on my priority list because to me it looks like it focus too much on being a "community-driven website" instead of providing a solid service as a media-hosting/publishing/catalog system.
To me it looks like they are shooting for the wrong level of "decentralization granularity". Each instance of these services are aiming for a "community", and think that the people use the mainstream tools because they don't want to/won't manage the server.
The point they seem to miss is that this only creates another type of top-down organization. It would be MUCH easier for them to focus on a "single-user" system, and start from the point that the communication will work when the applications talk with each other.
To me this is why Diaspora failed, and Wordpress is still such a big part of the internet.
Another thing I noticed: the projects that really focused on separating client from server produced much better results in terms of UI/UX. With Matrix, I just had to setup the server, and then I could have the riot app just point to my instance. If by any chance a better client comes around, my instance would be untouched.
- https://github.com/syncthing/syncthing/pull/3527
tl;dr- Syncthing doesn't do a rolling hash implementation. Therefore, lets say you have a 1GB file. And you add a 1 line header at the top of the file, syncthing will download the entire file.
discussion link: https://github.com/syncthing/syncthing/pull/3527
I love this phrase, it really seems to catch the problem with all these decentralised projects.
Sure I could try to set up whatever matrix/status/appnet/tent decentralised service of the month happens to be on the front page of HN but also I could just not do that and use Twitter.
The other thing is they all want to take over the entire domain you use them on. Either you have to use a subdomain (which means you can't use your cool name@example.com address) or you forfeit your domain to whatever advert the software brings with it.
Honestly I've just given up with them. I use Twitter, I use Facebook, I use Dropbox and I do things other than trying to run my own versions of those in my free time
That's not true. Many such projects use /.well-known/ to provide the ability to redirect from the root example.com identity to where you should actually go to send things.
Just in case its not yet on your radar, you may be interested in Sandstorm:
It bills itself as a 'self-hostable web productivity suite', but it's much broader than that, it's really a platform for self-hostable decentralised apps. Sandstorm manages security and app interoperability, so that developers and users can focus on the core functionality of the apps.
Nextcloud is great for easy access to currently important files and automatic uploads of phone pictures, synchronising contacts and calendars. The news app is also an excellent RSS reader.
Syncthing synchronises the bulk of my data between systems I own, it is much faster at this and isn't exposed to the risks you get with public web servers, but I cannot access those files from systems I don't own so anything I am currently using goes in Nextcloud.
This 1000 times. It just kills me to think about how much potential progress has been lost to these major projects spending so much time and energy on social federation protocols instead of working on the actual primary function of the software.
Mediagoblin could have already been THE software go-to for personal media organizing and display, but it seems to be getting stuck like every other potentially great project that runs out of steam when the main developers lose their focus. And now Yahoo has sold assets to Verizon, and millions of Flickr users are wondering what will happen next.
Put together a distro with a complete solution - mail server, web server, media and social servers.
Make it one-click easy to set up. Keep security tight. Maybe do some deals with cheap hosting providers.
Then federate and connect.
I think this is kind of a good point. It's too hard to set up these 'solves-one-thing-well'-suits to work together in the right way. Having collections of services that are super simple to set up and works well together is probably a good way to do it.
Like one media platform, one search engine, one social platform. Or perhaps having a couple of alternatives but having integrations in between up and running in a plug and play way.
Then again, if you want all this to work in a decentralised fashion you probably have to think about how to achieve that early on.
I just moved my calendar from a large email provider to a self-hosted Radicale server on Sandstorm (syncing with Thunderbird, and DAVDroid on my phone). Small steps, but it was surprisingly easy.
The bad news is that I am the only person working on it, and while I am an experienced developer, I am also a grad student.
In other words it is not ready for production at all. I have presented it to quite a few people for the purpose of finding help in the development, and everyone thinks it's a good idea, but everyone also wants to see a finished version of it before seriously considering helping out. Unfortunately, I can only work on Defiant in my free time, so developing the basic functionality is steady, but slow, progress.
Sorry to be that guy, and I know it's not a nice thing to hear. But I'd hate to see you burn a ton of time on something that's born under the cloud of "there are a million other ones just like it and no reason anyone can see why this one is worth investing effort into". I'd rather see you take all that effort and skill, find a project that's already got some momentum behind it, and get behind that and push - everybody saying "I'll build my own!" is half the reason we've got where we are in the first place.
On the other hand, maybe your framework has a genuine USP that I'm just not seeing. In that case, you'll have an easier time getting interest if you find a way to surface that, front and center, so that even the most disinterested professional engineer who's skimming HN by way of procrastination before the next code review can't help but see it.
Either way, hope this helps, and good luck in your future endeavors!
Make it easy for me to stand up a server, upload a video of my kid, then share the link to that video on the internet.
Leave all the community/social/commenting stuff to community/social/commenting software.
I'd rather have both an open-source media server and an open-source equivalent to twitter, than have both of them rolled poorly into one.
You could say that Mediagoblin has 2 stated goals:
Stated Goal 1) Organize and share user media, which is what people actually want it for.
Stated Goal 2) Federated social something mumble, which no real world users actually want but which free software projects seem to get big e-rections over anyway.
I think that Mediagoblin should have 1 stated goal, the first one.
All news for the past several years has been about goal 2, and I just don't think that anyone in the real world actually wants or cares about that. Most people don't even know what the hell federation is supposed to mean. People just want to be able to stop relying on Flickr to organize their damn photos. Chasing this federated social something mumble dragon has seriously gotten in the way of making the thing that people would actually use, and it's frustrating to watch because there was so much potential.
Their official linked-from-the-landing-page "Easy Deployment" page (god, I'm already terrified by the word "deployment") has bold "WARNING: OUT OF DATE" messages all over it.
Their official site for (Not-Easy) "Deployment", right at the very top, says "GNU MediaGoblin is fairly new, and so at the time of writing there aren’t easy package-manager-friendly methods to install it."
The project is now 6 years old. 6 years isn't new anymore. There isn't an installer?
One use case I imagine is to add media files to my local Mediagoblin instance, and then have them show up on my friends's federated social media feeds on various platforms (GNU Social, Mastodon, Pump.io, etc.).
In what universe is that? Certainly not the Instagram I've ever seen. Mediagoblin's core compentency is 10000x more like Flickr than like Instagram. Now maybe that's not what they want to make, but damnit it's what they present themselves as trying to make and failing to make major progress on.
> One use case I imagine is to add media files to my local Mediagoblin instance, and then have them show up on my friends's federated social media feeds on various platforms (GNU Social, Mastodon, Pump.io, etc.).
Look, that would be a great secondary feature to add onto an already functional and installable tool. But you make the darn tool first and try not to invent imaginary problems! Because GNU Social and Mastodon are social networks, and Mediagoblin is a tool. Do you know what you can already use to post images to social networks with tools? An authentication API and hyperlinks. Not a database redesign.
What problem is mediagoblin actually trying to solve? Is it organizing your media or is it being a new social network? Because one of those makes sense and one of them does not.
> So, if you create content with a...
Mediagoblin isn't for creating anything, and it certainly isn't a network client. It is (ugh, should be) a tool for organizing your media.
Know what's a really great technology for sharing media? Hyperlinks. We built a visual web that is literally world-wide using them. Know what people don't have any trouble sharing media to using hyperlinks already? All of their social networks. Know what hyperlinks don't need? Years of effort spent chasing a white whale.
Mediagoblin still has a chance to be OpenFlickr if they would just stop faffing about with things that don't have any connection to it and start focusing on core competencies and important things like installability.
It is very important to content creators that such functionality does exist. In today's world of digital media, this precise feature, which enables easy collaboration, addresses one of the industry's biggest pain points. Even the monolithic media companies struggle to implement useful solutions to this problem.
(I had to look up what you mean by DAM. I assume digital asset manager?) Are there? What easy-to-install, modern, libre, web-based personal Flickr replacements are there? Do the work generically with many different types of media?
I don't know of any. That doesn't mean they don't exist, but I've known about mediagoblin for quite a few years now.
And the product that mediagoblin shows on their website is exactly the product I'm talking about. It's exactly the product that everyone using Flickr but who wants to be able to also include sound files or who would like to be able to migrate away from Flickr to something more free wants to use. Except installing mediagoblin is a big ol' pile of shell commands and text config files, and that's if you somehow navigate your way to something current.
What the hell? That is the main feature required in a social network, after the ability to see each others comments. People want to organise their lives using the graph that their social network provides.
Is anyone aware of a distributed or federated open source social network which provides event management functionality? I would love to hear about it if you have... Gallery functionality would come in handy too.
Most peoples friends and family are on Facebook, so the events management feature is useful there.
I can't convince my friends and family to use a social network if they lose the ability to organise and be made aware of events as part of the move.
And how did Facebook actually solve the chicken-egg problem you just referenced?
I'd love to get some citations for any of the federated social networking platforms-- Diaspora, Gnu Social, Mediagoblin, Mastodon, etc.-- that link to a carefully considered and workable plan for mass adoption that any of them carried out at any point in their history. For reference you can look at the Wikipedia entry for Facebook (hint: it involves universities and enormous amounts of money).
edit: remove redundant word
It was fairly easy to implement, the only problem is that not many of the people I know have software which lets them RSVP too, therefor I also connected it to Facebook.
I'm actually very happy with this - contacts, calendars, news, files - never have to go on the cloud for these things.
The uptime I've been able to manage for the last 2 years is about 95%. Most of the downtime was due to a kernel panic, a shady bios on the intel nuc, and no means of remotely restarting the thing.... had to get a "smart switch" and now my uptime could potentially hit 99%.
DR is next... got to drop my RAID disk + server combo with some friends and family. Probably slap on gluster, or just use lsyncd to synchronize changes.
We should make a freenode channel for this!
Is there something I'm missing?
Also in terms of self hosting, something I'd really like is the ability to point it at a pile of files and have it ingest them in place. Any idea whether that's possible?
If it depends on an index/searching service, how is the network itself any more decentralized than say an FTP server? I wouldn't call FTP decentralized.
I started implementing something like this myself but didn't make it very far. The fear of eventually running out of space or having privacy/security issues on Drive hasn't produced enough pain for me to really do anything yet.
* ownCloud is more stability-focused, used by large corporations, slower release schedule.
* Nextcloud is like more experimental, with new updates every two weeks, adding new features constantly, two major versions in front of ownCloud.
So, I tend to think them as ownCloud being kind of like Ubuntu LTS release, and Nextcloud being a regular six-month release.
They also cooperate a lot and build on top of each other (for example, Nextcloud just makes a skin on top of ownCloud's desktop client).
Also, Nextcloud's app community seems more vibrant than ownCloud's.
Big advantages over own/nextcloud:
* Easy updates that work. When I tried owncloud, nearly every update I had to fiddle around with the database to make it work. Didn't have a single problem when updating Seafile so far.
* Block-level sync. Meaning if you change one line in a 10MB file, it will only update that part. Last time I checked, owncloud did not support this.
* Sync just works. When I tried owncloud, folders disappeared, folders were duplicated. Never had this problem with Seafile.
I've run ownCloud for some time now and I don't see the reliability issues that some people claim. I am running it on TS140 and not something small like a RPi, so maybe that makes the difference.
The first time it happened I shrugged it off as a fluke. The second time I became gunshy and would disable it any time I was working, then enable it after. Not a big inconvenience but a hassle and something I had to remember. After a while I stopped using it. YMMV, of course.
As an alternative, you can just:
# curl -O http://vestacp.com/vst-install.sh # bash vst-install.sh
which would setup the entire LEMP stack for you (along with a full-featured, minimalist hosting admin panel) and then you can install any of the available file sharing tools.
# bash vst-install.sh
vst-install.sh: line 1: syntax error near unexpected token `newline'
vst-install.sh: line 1: `<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">'# curl -O http://vestacp.com/pub/vst-install.sh
# bash vst-install.sh
For cloud services, the only unambiguous win for native over web client is access to platform APIs that the browser may not use.
Someone has to install these web shells too, often not you.
I feel these solutions seem like they should be aimed at the people and groups you mention, but end up seeming like they're designed for tech geeks and programmers. They need to be made to work on cheaper hosting accounts and installable by someone whose experience with 'tech' is clicking buttons in a user interface.
But the average person in many of these small organisations (even the tech team) is probably confused by a simple WordPress install or even the process of uploading files to a web hosting account.
If a "tech team" can't even upload a file to a server, that is really a piss poor tech team and in general shouldn't even be labeled as such, IMO.
So no, not really a tech team, and obviously they'd be absolutely terrible compared to anyone who actually knew what they were doing. But the general point is still the same: these organisations don't usually have a tech savvy member or employee who could set this up.
And if they do find someone to help them out, I suspect a fair few agencies would struggle with anything that wasn't basic HTML/CSS, WordPress or an online service.
Youtube differentiates itself by being a go-to platform, a website that people actually visit to watch videos. Otherwise, call it an alternative to Vimeo, which is de-facto just a video hosting platform.
But when it comes to decentralised services... I suspect the answer is 'whatever instance connects to most of the others and draws the most traffic'. Which raises its own issues, since [BigVideoSiteInstanceNameHere] basically just ends up with the same problems as YouTube. Being able to control what a large percentage of users watch on a de facto level, even if the actual underlying system is decentralised.
It's still under active development, though I've temporarily handed over the reigns to others while I've been busy getting the federation standard we're going to use nicely shaped up through the W3C spec process https://www.w3.org/TR/activitypub/
That's taken up more of my last year (okay, last two years) than I expected. I'll be back in the swing of things within the next few months. We have a federation branch right now, but the standardization process of ActivityPub meant that we're going to have to retool some things before its released in 1.0!
The good news is that ActivityPub is looking to be picked up by projects like NextCloud, Mastodon, Pump.io, and quite possibly GNU Social, Diaspora, postActiv. This means we should be able to have more federation working across the many federated social web projects out there.
If you're interested in ActivityPub and federation generally, I highly recommend checking out the tutorial, which is baked into the spec: https://www.w3.org/TR/activitypub/#Overview
In the meanwhile, Boris Bobrov (breton) has been kindly doing the main work along maintainership while I've been preoccupied. But, I'll be back soon... and it'll be good to be back!
Or client-server communication ?
https://www.w3.org/TR/activitypub/#security-spam
"Spam is a problem in any network, perhaps especially so in federated networks. While no specific mechanism for combating spam is provided in ActivityPub, it is recommended that servers filter incoming content both by local untrusted users and any remote users through some sort of spam filter."
I feel it shares a lot of similarities with the [matrix spec], though obviously a different vocabulary. Also, matrix is focused a lot more on the concept of subscriptions and eventual consistency.
The other recent protocol I'd associate this with (loosely) is [secure scuttlebut]. This uses an append-only log, which users can subscribe to. The protocol lets applications on the same network locate each other and synchronize logs, or they can connect to a pub server which helps with providing more permanence and crossing network boundaries.
The latter example is much closer to what I'd like to see in a decentralized social web. You build a self-hosted stream of public and private messages, and those that are subscribed (individuals or pub servers) replicate a copy of that stream. If individual pub servers go offline, others will have duplicates and users will be able to continue receiving your updates.
Granted, this setup has some serious issues when you consider abuse. Pub servers currently have no concept of filtering and any approach to that would raise concerns about censorship. I'd also like to see the clients move to a browser based, port 80/443 setup, where perhaps someone can run a self-hosted pub server and web client on a cheap vm somewhere. A sort of web proxy that mobile/desktop clients can sync to. The clients in this scenario would not speak scuttlebut directly, but rather post messages to the web server, which would manage injecting those messages into the ssb log.
https://staltz.com/an-off-grid-social-network.html
[matrix spec]: http://matrix.org/docs/spec/ [secure scuttlebut]: http://scuttlebot.io/
[1] https://english.stackexchange.com/questions/153166/what-is-t...
https://apps.sandstorm.io/app/70awyqss6jq2gkz7dwzsnvumzr0725...
Isn't Sandstorm basically dead at this point? I seem to recall them announcing that the commercial side of the project had failed and they were going 100% Open Source.
This is true regardless of which open source license is used, and I am a strong supporter of permissive licensing over copyleft. ;) I see nothing wrong with corporations building their proprietary platforms on top of open source software as long as I'm not required to use it.
Sometimes as an open source and privacy advocate, I feel like I'm in the minority here, along with a couple of other voices I see along the same lines as me. Though, I am pretty anti-copyleft as well. Go figure.
It's AGPL specifically because it's a GNU project. shrug if you want to write a BSD one, that's your right. IMO they're not extremists. I like the BSD license a lot. But I also think that sometimes some companies exploit software developed under BSD licenses and in those cases coders and users alike lose out. For that reason I think it's very appropriate that the GPL and AGPL exist.
Is it just me or are the copyleft digs increasing in number lately? The little jabs seem nearly as immature as outsiders requesting that FOSS maintainers rewrite their projects in different languages.
But the distinction between GPL and AGPL here matters. If mediagoblin were GPL you could use it to host better-than-youtube.com -- and subsequently add a hardware-accelerated-3d-folksonomy feature to your better-than-youtube.com instance. better-than-youtube.com's users would be [effectively] using a work derived from mediagoblin and yet they would not have access to its source because they were never distributed the binary. GNU created the AGPL to solve the problem created by the trend of "Software-as-a-service".
> they have clearly been modified
How do you know? Mediagoblin has lots of settings and prefs to adapt how it works.
Anyway, the source-provision applies whether or not it's modified. It's the responsibility of any instance of Mediagoblin to provide source upon request to visitors of the instance. The source must match what is running. Modified or not is irrelevant.
> unwittingly in violation
Not at all. They will be in violation if you can access their instance and you ask them for the source and they refuse to give it to you.
6c does not apply. No such offer was made, nor were the preconditions met.
6d is the only option that applies, and while those copies had the standard links to upstream, they had no links to downstream.
See https://www.gnu.org/licenses/gpl-faq.html#AnonFTPAndSendSour... and what follows it if you're not willing to read the license.
Also, I don't seem to be able to edit my problematic reply above (which is correct in some regards, not so much in others). But thanks for clarifying here. I regret spreading some misunderstandings.
Nonsense. The sort of people who appreciate the AGPL are the sort who either say or at least acknowledge that others say "it's free software, not open source".
If you think AGPL is unappealing and that's as far as you take it besides asserting it being extreme, you don't understand the basic concept of software freedom, so you can't judge when someone is extreme about it or not.