Copyparty – Turn almost any device into a file server | Hacker News Reader

Copyparty – Turn almost any device into a file server

github.com

1,096 pointssaint11169 comments

The author of this tool uploaded a YouTube video demonstrating it a few days ago: https://www.youtube.com/watch?v=15_-hgsX2V0

At one point in his demo, he uploads a file but terminates the upload more or less halfway. Then he begins downloading the file - which only progresses to the point it had been uploaded, and subsequently stalls indefinitely. And, finally, he finishes uploading the file (which gracefully resumes) and the file download (which is still running) seamlessly completes.

I found that particularly impressive.

It's very impressive, particularly if you remember waking up to a failed download from the night before over dial-up.

I recall we had special apps to queue and schedule our downloads, and resume them where servers supported it. They were a dream compared to the boredom of staring at progress bars.

Anyone remember DAP, Download Accelerator Plus? The colorful bars were nice. A part of my childhood, downloading shareware Windows games through dial-up.

Download Accelerator Plus... wow what a memory.

Finding that piece of software around 2001-2002 was what allowed me to finally download a specific piece of, ahem, 'shareware', that was about 400 MB, zipped, that I would never have been able to finish on a 14.4kbps modem on a single very noisy phone line that usually dropped the call every 2 hours or so. It eventually took three days but the file came across uncorrupted. It wouldn't have been possible without the ability to resume downloads after dropped connections.

And that software download went on to allow me to start the path learning what I wanted to learn about, and that paved the way for my engineering degrees and thus setting me up for the last 20-some years. Wild how little pieces of the puzzle like that drive so much of your life.

Some apps still do the same, eg:

https://www.downthemall.net/

(also a great app to download everything you wanted from a site, regex selections, etc.)

Makes several connections and downloads chunks in parallel, for some sites with limited upload (their, your download) speeds per session it really speeds up the downloads.

Sadly, not much development recently (9 months ago was the last commit)

Wow, that's a throwback.

Download The Mall!

I remember that...

Getright!

The trouble is those special tools also needed downloading. So I could either sacrifice an evening's, ahem, download, or just chance it yet again. I eventually got an FTP client and it was like a superpower. BitTorrent was honestly more impressive to me than AI. Ah, the good old days.

> BitTorrent was honestly more impressive to me than AI. Ah, the good old days.

That’s because BitTorrent was immediately useful and empowering.

The server that has moved countless Petabytes is glFTPd that allows FXP ( clients without bandwidth can initiate to transfer files from server to server ).

That’s a built-in feature of FTP that doesn’t require server support.

Edit: Source: https://en.wikipedia.org/wiki/File_eXchange_Protocol#Technic...

1. You connect to servers A and B.

2. Tell B to receive a PASV transfer. It replies with the IP address and port it's receiving on.

3. Tell A to send to that address and port.

This is documented in RFC 959, starting with

  "In another situation a user might wish to transfer files between two hosts, neither of which is a local host."

In the BBS days there were transfer programs that allowed for interrupted downloads to be resumed:

* https://en.wikipedia.org/wiki/HS/Link

wget will pick up a dl where it left off. i know that "pick up where you left off" capability has been built into HTTP since the mid '90s if not right from scratch

i used to use that feature to run several downloads from several different sites of different sections of linux distro isos when new releases were put up

One of those things of the past even old nostalgic greybeards like me do not miss at all.

Most files were available via FTP which supported resume.

Not most. There was (and still is) so much locked behind HTTP on poor servers

The vast majority of web servers out there¹ support partial download and have done for years. That the most common UA for accessing them (web browsers) don't support the feature² without addons, is not a server-side problem.

Sometimes there are server-side problems: some dynamic responses (i.e. files that are behind a user account so need the right to access checked before sending) are badly designed so that they uneccesarily break sub-range downloads. This could be seen as a “poor server” issue, but I think it is more a “daft dev/admin” or “bad choice of software” problem.

--------

[1] admittedly not all, but…

[2] wget and curl do, though not automatically without a wrapper script

Many sites also had an ftp server behind it. E.g. ftp.id.com and ftp.cdrom.com were two off the top of my head. Another I remember was downloading high resolution images of Tyan motherboards from ftp.tyan.com. Supermicro also had an ftp server you grabbed bios images from. I dont really recall ever having to download anything big via http. Mostly images, pdf's and small zip files.

Obviously I wasn’t using FTP way back then, or I wouldn’t have made the comment I did.

FTP can't restart a PPP or SLIP connection.

No, but FTP and such protocols shouldn't need to be aware of that layer, any more than it should be aware of my VPN that is happens to be connecting through. You can resume an FTP transfer once the PPP or SLIP connection has been restored though.

I remember redownloading Liero over and over again and failing. And then cherishing it once getting a successful download. It would barely fail to fit into a floppy.

Amateur. Use Flashget or Netants which download the file in 8 simultaneous chunks. I used to cheer the threads on last legs of a whopping 5m download. I hated servers which dont allow resume or even report file size.

If I remember correctly, unfortunately the HTTP server hosting Liero back then didn't support requests with the Range header.

But also likely I didn't have FlashGet on my cousin's computer which was the computer we were using to play it.

And you forgot to disable call waiting!

Magic of http 206 ?

I really didn't think I need this software but the video is so good that I'm gonna try hard to find a use case.

Could be useful when launching a Doom shareware release.

“Race the beam”

That’s really cool. I’ve never seen that work before.

Sounds like...BitTorrent.

Or… proper adherence to HTTP RFCs… with some added devx

bittorrent needs to know the complete file at the beginning to make the pieces. This tool doesn't need to know the complete file to start the upload, nor the download...

IIRC webtorrent /can/ do streaming though....

Sound like BitTorrent needs better PR then

You might like NNCP which was written precisely to support severely constrained or even cut down down networks.

Technically, it's trivial. You either use the range header, or do chunked encoding.

It would be even more impressive if he rebooted the server in the meantime.

NNCP supports that.

Copyparty is an amazing piece of software. I recommend watching the recent YouTube video for an overview[0]. The developer is a personal friend and my household is proud to own one of 20 limited edition copyparty disc releases.

[0] https://www.youtube.com/watch?v=15_-hgsX2V0

The video BGM was quite familiar to me, as popularized by a certain former detective. (Those who don't assign that any special significance may nonetheless find it as royalty-free music from Dova-Syndrome: https://www.youtube.com/watch?v=kKg-aXVS1YI)

> into a file server

This is underselling it by at least three orders of magnitude. This is astonishing tool, you have to watch the demo video: https://www.youtube.com/watch?v=15_-hgsX2V0

This is the wet dream of every power-user. It has tons of features on top of the file server. And it also seems developed by a 10x (100x?) developer, I mean, just making/editing the video is a work of art and humor.

If the author is lurking here, are you doing all by yourself? Do you use any LLM/agent?

It really is impressive.

Hey o/

Yup, this is 97% just me hacking away in vscode -- I use pylance and the debugger but have everything else disabled, easier to focus that way. The only time I use any sort of AI/LLM is for translating new strings into Chinese, since it seems decently capable at that :-)

The remaining 2% is friends coming up with new usecases/features, and sometimes finding bugs.

But now that the project got way more attention than I'd anticipated, pullrequests have started appearing, so it doesn't look like those statistics will stay true for much longer! Really cool having more eyes on it spotting the things I overlooked, really enjoying that.

Been using copyparty for 2 years now. It’s awesome tech and just want to say thank you for your work tripflag!

Highly impressive work!

I noticed this which brought a SMB question to mind:

>login doesn't work on winxp, but anonymous access is ok -- remove all accounts from copyparty config for that to work

>win10 onwards does not allow connecting anonymously / without accounts

Is this an intentional limitation in copyparty itself, or maybe just the progressive difference in Windows versions?

What's the remaining 1%? :)

Rounding error :)

This is great stuff.

The only thing I'd like is some way to run it behind a cgnat. I was on starlink and I'm on an 5g device now.

If there was a way to integrate with Google drive mega Dropbox, githubs etc where I could drop a file list request document one of those services, and your server is pinging that (intermediate) storage service, detects the file listing request or file push request, or file upload request doc, and then does it.

I know each of those is an integration headache but man that would be useful.

Ok so GitHub has a built in markdown editor, so the request docs could be markdown templates. Or maybe static html/js files that generate markdown request docs, and file listing responses can be markdown or more static html docs.

Tailscale or Yggdrasil

Does magic-wormhole work from behind CGNAT?

Why don't you relay traffic through another node on say AWS or Hetzner using Wireguard?

Why would I want to give my credit card to a company to access my files behind a CGNAT? So AWS can decide to charge me $10,000 someday arbitrarily? So my servers get summarily yanked with no transparency?

Tailscale: I don't want to install a corporation's client.

Yggdrisil: "The current implementation of Yggdrasil is a lightweight userspace software router which is easy to configure and supported on a wide range of platforms. It provides end-to-end encrypted IPv6 routing between all network participants." Yeah, not what I need and too-much-access.

Install Headscale.

No corporation, fully endorsed by Tailscale.

I noticed on the demo server you had some ansi art on there.

Were you a part of the efnet ansi/ascii scene?

There's still some of us floating around!

Great project btw, nice work!

tripflag, are there any concerns with running this on a VPS. Haven't looked through the docs yet so don't know if you've talked about that..

In the vid author says they started this pre useful LLMs (2019) on their phone.

From https://github.com/9001/copyparty/blob/hovudstraum/docs/vers... (kudos for doing this, BTW!):

> seafile … and nextcloud … their license is problematic

There’s nothing problematic about the AGPL, really. It protects users. It protects developers against someone taking over their projects. The only people it bothers are those who wish to take a free software project and integrate with an unfree one. That seems like a feature to me!

Wow that nasty. I didn't click on the links because I thought it was one of these weird licences that pretend to be open source but are actually sketchy, but it's actually the very normal and very open source AGPL.

It's a huge feature. It won't prevent forks but it will stop corporations from making proprietary enhancements for SaaS. They have to publish the source code.

I even emailed Stallman about it. The only way to make a proprietary version of an AGPLv3 project is to pay the original copyright holders for a special proprietary licensing deal, thereby supporting its development. Forks don't have the same privilege.

In addition to being an awesome piece of software, their self hosted demo server is the fastest web app I have seen in a long time ... and this is while trending on HN !

Amazing.

Now I am wondering, would it be technically possible to build a similar app but based on the syncthing protocol?

I really like syncthing but it would be cool to have a version where you could just easily share specific files with peers.

Ohh that would be cool! Love syncthing but I wish the relay and discovery servers would be part of the same/main syncthing binary.

I've also seen quite a few semi-technical youtubers make videos about it but not mentioning that it uses public relay and discovery servers usually by default (but maybe that depends on the distro). It's not a bad thing but something one should know before using it.

Not any longer, it seems. :/

Never mine, it's back now.

[starts watching video] Ok cool it's a file browser, there's a million of the---s----e

[keeps watching video] what the fuck

I thought I was going to be rick-rolled, but the video is actually very good, and if the functionality is as described, well then hats off to the creator of copy party. Fantastic work!

Absolutely amazing piece of software, the kind that makes you wish you had a use-case for that. Kudos to devs for taking security seriously, too.

By the way, the youtube video showcases this project really well.

Heh... I have one... have always wanted to make a little solar-powered "library" on my front-lawn...

(You know, like the neighbourhood "take-a-book, leave-a-book" little libraries, except for... digital content... It would fly an appropriate "skull + crossbones" flag...)

I've wanted to do something like this, but I live within WiFi range of a school and am concerned someone would put something "harmful" on there so have never done so.

I created a PirateBox on a little GliNet router a while back with the intention of sharing public domain content but didn't do so beyond having a quick play around with it myself.

And like most things nowadays, it would get filled with highly illegal content within hours of you putting it there. The good old (innocent) days are gone and the society we’re living is not mature/educated enough for such ideas.

I don't think the idea is to put it on the global internet; just make it broadcast a wifi SSID.

As others have said - it would be standalone, not connected to the internet.

Have debated making it "read-only", but then I would be culpable for the curation of content...

That and perhaps I just don't want to encourage people loitering around in front of my house for long-transfers...

OTOH - this could be useful for essentially a "dead-drop" independent standalone box for, uh... "civil disobedience" reasons... (or a free alternative to those "prepper-internet-in-a-box" devices they are currently selling...)

Check this out. This was at one point one of the cheapest and smallest Linux computers around. It’s USB powered and this project turns a WiFi device designed to share photos from an SD card over a standalone SSID into a male USB A powered miniature SBC. (Edit: okay it’s two PCBs technically)

https://github.com/Emeryth/openwrt-zsun

https://wiki.hackerspace.pl/projects:zsun-wifi-card-reader

I got them in bulk from China for ~$6 each.

It would still be physically located on your property with potentially illegal content on it. Sounds like a nightmare.

... well, I live in Canada - my understanding is that the maximum lifetime fine for copyright infringement is about $5,000 when files are shared for personal, non-commercial use...

Which sounds like alot, but if we factor in the extended family and cross-media sharing and the number of separate streaming services we all subscribe to across many many years, then this is a "deal"...

OTOH - I don't want to be the first case/person to help determine what precedent will be set if something actually gets taken to the end-state statutory damages..

On the spectrum of illegality, things can get a lot more extreme than a bit of copyright infringement.

True - I mean, one could try and block based on file-extension/MIME-types, but... nothing stopping a malicious user from renaming a file to an allowed extension, with some sort of malicious/secret payload. (Or... spreading some sort of malware/virus/exploit via media file formats, I have never looked into the possibility of that until just now, apparently it can be a thing - https://cyberpress.org/cybercriminals-exploiting-media-files...)

So yeah - this is probably one of those half-baked ideas that just wouldn't be a good one to actually implement "in-the-wild".

Maybe I'm misunderstanding something, but wouldn't this work great (albeit huge overkill) for the extremely common problem of trying to get files from one device to another (especially when one of those devices is a phone)? I see tools that are supposed to do that posted to HN all the time, with the comments usually pointing out one or another problem with any given utility. This seems like it would be pretty great self hosted, open source, solution to that problem?

I have been having a lot of luck with Blip[0] recently regarding phone <-> laptop file transfer. My biggest issue so far is that it does support iOS, Android, MacOS, Windows ... but not Linux.

[0]https://blip.net/

Try https://payload.app/

Resumable, can queue, send directories, drag & drop, LAN (without account) & WAN (hybrid p2p), all transfers + metadata are e2ee. Linux, Windows, Mac, iOS, Android.

Disclaimer: I’m the creator

Termux and python -m http.server. I use that embarrassingly often, except for cases where I can just use scp or rsync (e.g. between two Android devices that both have Termux installed and I have bothered to copy the public ssh key from one to the other).

A very simple shell script:https://github.com/sancoderhost/ShareFiles

If you have not tried "localsend" I would highly recommend.

There is also https://pairdrop.net - same thing, but you don't have to download anything, it works on the web.

$kdeconnect-cli -d somedevice --share somefile

?? Airdrop works well.

Not everyone uses Apple products.

I saw "Python" and was going to comment a completely stand-alone* executable would be simpler, and then I scrolled and holy moly, there are a lot of features there! It would be quite some work to redo it in another lang.

*It already has no deps

Great job there. A nice tool you've made.

Edit: already adressed: https://github.com/9001/copyparty?tab=readme-ov-file#copypar...

I wonder if cosmopolitan libc[0] could be leveraged to create an actually portable executable™ without a rewrite, looks like someone has figured out compiling python 3.11.4 with it[1]

[0] https://github.com/jart/cosmopolitan

[1] https://ahgamut.github.io/2021/07/13/ape-python/

I have a small/insane project of mine, I wrote a compiler for Python (strict and static subset only) to WebAssembly (bc-to-bc approach, 1:1 CPython compat due to walking internals), than I do wasm2c to sandbox it + pledge and compiling with cosmopolitan into a miniature standalone thing (fast as hell). Just because you have zero dependencies and it's a pure Python and properly typed, lemme try next weekend as PoC. No promises, but this message clicked in my heart

on it

> It already has no deps

Yes when I heard it was python I immediately checked the dependencies expecting a ginormous list and found basically nothing. Incredibly impressive, just a few "stolen" libs here: https://github.com/9001/copyparty/tree/hovudstraum/copyparty...

Worth scrolling through util.py to see lots of hand-implemented code: MultipartParser, read_header(), read_socket_chunked(), html_escape(), atomic_move(), killtree ("still racy but i tried"), termsize(), etc

https://github.com/9001/copyparty/blob/hovudstraum/copyparty...

6k loc for http server/client handling transferring files, rss feed, etc. See `def run(self)` for main request parsing.

https://github.com/9001/copyparty/blob/hovudstraum/copyparty...

I have never heard of this before, but watching through their Youtube introduction, this might be one of the best pieces of software I've ever seen. Assuming it works as advertised, this could replace a few things I've been hosting myself.

From the introductory video: "There is no telemetry and their never will be. Not even an auto-updater". And yet this is one of the most feature-full projects I've ever seen. Brilliant.

I use this as a music player for my ancient ipad in my kitchen. It’s ugly but nothing else worked; the dev even changed it a bit to help me out

I remember the horror I had to go through to upload my ripped dvds to iPad for kids to watch. We had to setup an http server for the vlc to download the files to its data storage

Don’t get me wrong, but that one person is part time developing this magnificent peace of software with his smartphone commuting in the bus is another proof that Python is a great programming language.

This is awesome. The readme is fun as heck and I just want to use the software based on that. I see nothing but complaints about nextcloud and others on r/selfhosted. I can't wait to try this out.

Thanks for the tip, I really did enjoy my scroll of the readme. This bit here really tickled me and set expectations so well:

> inverse linux philosophy -- do all the things, and do an okay job

It even plays freaking chiptunes lol

Seriously considering replacing my navidrome/subsonic service and try my music library trough this. I used to play music straight from the directory tree a long time ago anyway, this might feel right at home.

From the readme:

> NOTE: full bidirectional sync, like what nextcloud and syncthing does, will never be supported! Only single-direction sync (server-to-client, or client-to-server) is possible with copyparty

Worth noting that if you're considering using Nextcloud this may be a dealbreaker... But if it's not, I would also recommend not using Nextcloud!

If you think the readme is fun, check out the demo video! https://www.youtube.com/watch?v=15_-hgsX2V0

You weren't kidding. I was amused by the humor in the first few minutes, but then I got to its showcase of what you can do, and am just even more blown away. They weren't kidding about doing _just about everything_ pretty well.

Take a look at the known issues section regarding iPhones. It’s good evidence of apples non competitive behaviour regarding browser support. PWA/websites are not allowed to be good on iPhones.

I am unreasonably happy about the "Activate Windows" watermark on the demo server (https://a.ocv.me/pub/demo/)

Half techie guy here with basic question. Could I run this on an old android phone sitting on a shelf with a large minisd card in the slot and call it a day?

Yup, here's a timestamp to the intro video where the author says it was "made for termux, born in termux": https://youtu.be/15_-hgsX2V0?t=889

Yeah, I'm fairly certain you could run it on an android device using termux.

You'll likely need to root the phone to get the OS not to kill the termux process due to idleness though.

No deps is great, but what I'm looking for is no bugs, authentication and encryption. I want features turned off by default, configs tight as a drum.

This section of the readme really sets the expectation clearly:

> inverse linux philosophy -- do all the things, and do an okay job > - quick drop-in service to get a lot of features in a pinch > - some of the alternatives might be a better fit for you

This includes a link to this doco in the repo which is an incredible source of info: https://github.com/9001/copyparty/blob/hovudstraum/docs/vers...

If you're looking for security and stability, I would personally avoid this.

I took a glance at the code and it's... not great. It's absolutely full of short, meaningless 1-2 letter variable and function names that make it very hard to read and understand if you're not the original author. Wouldn't be surprised if it's full of security holes that will never be found.

According to the author it was mostly written on the train with his phone, that could explain the terse naming.

This is a developer flex if I ever heard one

From the README FAQ section:

> i want to learn python and/or programming and am considering looking at the copyparty source code in that occasion

> do not

FWIW, it just had an XSS vulnerability fixed yesterday: https://github.com/9001/copyparty/security/advisories/GHSA-9...

> No deps is great, but what I'm looking for is no bugs

I don’t think I’ve ever used a piece of software in my life that had no bugs. At least with no deps (vs say a nodejs project with 500 of them) then the bugs will only be in one place, the main software so ideally they can be fixed quickly.

Obviously, that is not this

This is just good software

Besides being useful, this just seems fun as heck after watching the demo video. I'm curious if folks have examples of other similarly fun/whimsical (but still useful!) software. I would submit https://fraidyc.at/ to the list.

I really like this idea; I’ve made crappy versions myself a few times…. Maybe this one will stick.

(For others, it’s a method to follow people across multiple services without being a normal feed. A person who updates only shows up once.)

I pop this up at an annual lan party amongst friends and its always a hit. Easy to use, easy to run, jam-packed with features

I love seeing the term file server being used as opposed to "NAS". Its a server that serves files so its a file server!

Bj-skys.s3z - i havent heard in years. Excellent choice in the demo video and thanks for serving up on your nuc server! https://a.ocv.me/pub/demo/music/chiptunes/#af-bd94d434

I'm planning to use copyparty with Apple Shortcuts to sync my clipboards across devices - should make life a lot easier.

From the "'frequently' asked questions

> i want to learn python and/or programming and am considering looking at the copyparty source code

> do not

As someone who has tried and failed to upload 2gb files on mobile data, only for the upload to fail at the last minute, this genuinely makes me want to play with it. There aren’t a lot of good drop-in solutions like this.

FTP/SFTP support resuming file uploads

'rsync' used via a terminal emulator like Termux or Android's own Linux VM is also good for resuming partial file transfers.

Really cool! I wish it also had some better support for books so one could replace calibre web.

For example, access over OPDS, which one could then configure as the store backend on Kobo eReaders (yes, that's possible).

This is awesome! The demo is really fun, nostalgic 90s vibes, it even lets me play chiptunes :). Keeping all dependencies optional and just making it a single file is great too!

I also like SendMe [0]. I’ve used it to share files between local PC and server sometimes. And also to send large files to friends without cloud links. It’s written in Rust and based on iroh p2p library [1].

[0]: https://github.com/n0-computer/sendme

[1]: https://www.iroh.computer/

I've wanted to have local image and video hosting for a long time, this came at the right time. I saw the video on my YouTube front-page yesterday and enjoyed it a lot. Absolutely amazing project, my hat's off to the creator

Is anyone else have their antivirus alarms go off on visiting this repo's URL?

The main distribution file for this project consists of a python file that embeds multiple python files inside itself encoded as a binary bitstream in a bespoke archive format attached to the end of the file. Maybe this is triggering your AV heuristics?

No, visiting their repo url triggers a port scan.

Visiting a Github repo "triggers a port scan" is breaking news level stuff.

Didn't happen.

<somewhere around 2000> ZoneAlarm clearly popped a RED window saying the server is attacking me with PING requests. I am calling Internet Police.

This made me laugh and now I feel bad. Somehow I didn't expect HN users to be using antivirus software.

When your corporate overlords insist on such things, one runs such things. And then some sites throw alarms.

With a VPN on (Mullvad), https://a.ocv.me/pub/demo/ returns 403 Forbidden nginx

I have a decent library of ebooks and music albums I (legally!) purchased from myriad sources. This sounds like the perfect solution to get that properly onto my LAN in an accessible form.

This is Awesome!

I had been using serverultimate on my phone for a few years, and recently limitations made the port binding not work anymore. Copyparty using termux on Android worked amazing!

Wow sick loading animations https://a.ocv.me/pub/demo/ !!!

I'll take this out for a ride with my friends at the Assembly Demoparty, let's see if it holds up!

Saw the demo video on YouTube today. Almost made me rent a cheap vps with lots of storage just to run it

This is amazing, just amazing. I'm sure this fills important use-cases in my personal projects.

This is amazing, and somehow the "wapkthrough" video is super fun. Super kudos.

Just learnt about this awesome tool. Now it's on all my systems. :-)

Thanks!

Damn wow I really need to make up a reason to use this.

This is much more than just a file server. It looks pretty neat.

It will be useful for me, thank you for the info.

Coincidentally, I started to do media server similar to this 3 days ago. I just want to serve any folder, then be able to download a file and watch videos with a decent video player.

Now I feel like crap seeing how amazing this project is.

Don't let others stop you. Pretend you never saw it

Thank you!

he probably did this with 500 other file servers already in the wild.

..and with RSS feeds, very smart!

Yes! Bring back RSS

this guy protocols

How does it compare to Caddy?

Looks like another reinvention of the torrent.

No, it looks nothing like a reinvention of torrent.

Reply on news.ycombinator.com