HNHacker News
TopNewBestAskShowJobs

josephernest

601 karma · joined December 9, 2014

submissionscomments
josephernest··on Show HN: Swap.js – a JavaScript micro-framework (HTML fragments over the wire)
Thanks for looking at the lib!

Polling is typically the kind of thing I usually prefer to handle fully manually in JS. That's what I do for my own web apps.

Even if I added a specific HTML tag attribute for polling, at the end, there will always be a missing something, and I'll end up writing JS anyway. So for this kind of thing, I don't think a lib is necessary: you just setTimeout / setInterval in the script that runs when the view is loaded:

    Swap.loaders[".my_view_with_polling"] = () => {
        update_task = setTimeout(do_polling, 1000);
        ...
        return () => {
            // unloader function when we quit this view
            clearTimeout(update_task);
        };
    };
For a real example, see my email client demo: when you open an email, there is a timer. Here it's done: https://github.com/josephernest/Swap/blob/main/demo3-email/a...
josephernest··on Show HN: Swap.js – a JavaScript micro-framework (HTML fragments over the wire)
Demo showing the library in action: https://afewthingz.com/swap-library/email/

PS: some music I made if you want a musical background:) https://afewthingz.com/since

josephernest··on Show HN: Swap.js – a JavaScript micro-framework (HTML fragments over the wire)
Thanks for your detailed comment, the in-depth reading, and for this insightful advice. I'm going to think about all these points!
josephernest··on Show HN: Swap.js – a JavaScript micro-framework (HTML fragments over the wire)
Thanks for the comment!

These 2 other libraries are really great.

For something as important as the choice of a web framework, I realized I really wanted to 100% understand the internals, then I starting working on Swap.

The key things for me are:

- no tool required (just a text editor, no compiler, no bundler/packer, no minification even needed: I just wanted to write JS and that's all).

- be able to re-read the source code in 10 minutes and 100% understand how it works, which is impossible with most frameworks

- even if simple, I wanted that non-trivial features are included (such as browser history, when using previous / forward browser buttons), because I don't want to have to think about this when creating a new web appliation

My first prototypes took a few hundreds of lines of code (for the same result). I spent some good time redesigning it, and finally I was surprised I can cover 80% of my navigation needs (when doing web applications), with only 50 to 100 LoC.

For the rest (features not enhanced by Swap, like form submission), of course, I just write normal JS, but I think I would have done it anyway whatever the framework.

TL;DR: in my lib I just focused on swapping HTML content (parts of the page) when navigating, thus the name "Swap" :) + I wanted the lib to do the boring stuff for me (make sure the browser history still works, etc.)

josephernest··on Show HN: Swap.js – a JavaScript micro-framework (HTML fragments over the wire)
Hi, author here, I just woke up and see all these comments :) I'm reading and I'll answer!
josephernest··on Show HN: Reduce SQLite database size by up to 80% with transparent compression
Will this mode require a separate file along the data.db Sqlite file, to store the compression dictionary?

From your last link:

> The result of this training is stored in a file called "dictionary", which must be loaded before compression and decompression.

Also, what happens if a column has been compressed based on 100 typical values, and then later we insert thousands of new rows with 500 new frequent values. Does the compression dictionary get automatically updated? Then do old rows need to be rewritten?

PS what is compact mode?

josephernest··on Show HN: Reduce SQLite database size by up to 80% with transparent compression
Let's say I have a web visits analytics log database, with a column "user-agent" and a column "url" that have most of the time the same values (99.9% of rows have one of 100 typical values).

Would this work well with compression here?

ie. from 50 bytes on average for these columns... to a few bytes only?

josephernest··on Ask HN: Dropbox uses Python a lot, is it also the case for their Windows client?
Oh this is interesting, I would be super interested to learn about how they embedded Python in their .exe. (Cython can do it, but there are also other ways).

I started reading the blog, but not so much about the packaging of their code into .exe distribution.

josephernest··on Ask HN: C/C++ web framework with routes (like Node.js, Python Flask)
thanks!

i just looked, it doesn't seem to have this easy "route('/hello/world')" system, at least not in the main examples, is it included?

josephernest··on QOI: Lossless Image Compression in O(n) Time
Very cool and impressive!

Do you have benchmarks on what it does in the worst case? Let's say a 1000x1000 pixels image of RGB noise?

The bitmap should be 24 bit * 1Mpixels = 3MB. What's the size, compressed as PNG, and compressed as QOI?

josephernest··on A WYSIWG Markdown web app that lets you edit your locally saved files
Cool projet!

I did something similar a few years ago: https://josephernest.github.io/writing

LaTeX (with Mathjax) support as well, click on the ? button to open local files.

josephernest··on WindowTabs: Browser-style tabbed window management on the desktop
Please add a screenshot to see how it looks like.
josephernest··on Show HN: 128-bit, roughly-ordered, URL-safe UUIDs
Just to see the big picture, what are the main differences between this and Snowflake, or ULID (https://github.com/ulid/spec)?
josephernest··on Encrypted Backup Shootout
I created nFreezer ("encrypted freezer") for this purpose at the end of 2020: https://github.com/josephernest/nfreezer.

Main features:

* encrypted at rest locally (the data is never decrypted on the remote server)

* handles file renames / moves (no data is re-sent, it can save GB of transfer!)

* single Python file of ~ 250 lines of code. It is quick to read the code and decide if you like/trust it or not.

josephernest··on Ask HN: Would you pay for ARM-based dedicated instances?
You're right, all these aspects are important. If you add all these costs, such as electricty cost (3W * 24hr * 31day / 1000 * 0.15$/kwh ~ 0.33$/mo), it's hard to be both profitable and competitive. I'd pay maybe 2 or 2.50$ / mo, but I'm probably not a good example because I like running my own RPi at home and be able to tune things. Nice project anyway, good luck for staring your service!
josephernest··on Ask HN: Cloud providers as personal cold backups
If your provider uses SFTP, you can use nFreezer (http://github.com/josephernest/nfreezer) which I built exactly for the same purpose than you. (I first thought about 7z too but having to reupload the whole archive even if only a few files are modified made me look for another solution).

In short, you can do

    nfreezer backup my_documents/ user@11.22.33.44:/backup/ 
and the backup will be done or updated (if only 1MB have changed since last backup, only 1MB will be uploaded and not 100 GB).

Everything is encrypted locally (including the filenames), and never decrypted on remote. You can't use the files on the remote server, it's only cold storage - the only thing you can do with it is: restore to your local computer (if one day you need it), and decrypt it locally.

It's 250 lines of code in one single source file, so you can read it quickly to see if it's ok or not for your use case.

josephernest··on Ask HN: Would you pay for ARM-based dedicated instances?
Yes and no. Not at this price: for 5$/mo you can have a real dedicated server (not a VPS) by kimsufi provider for example. Also 1 year of rental at 3-4$/mo = you have spent the same price as buying and owning one.
josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
To clarify: if you start from a given nonce and key:

    cipher = AES.new(key, AES.MODE_GCM, nonce)
    while True:
        block = f.read(16*1024*1024)
        if not block:
            break
        out.write(cipher.encrypt(block))
you get exactly the same result as if you do (with a big RAM, bigger than your file) it in one pass:

        cipher = AES.new(key, AES.MODE_GCM, nonce)
        out.write(cipher.encrypt(f.read()))
Please try it with pycryptodome, you will see it is.

You might find this unforunate in the naming, and .init(), .update(), etc. might have been better names to emphasize this.

So this shows that, in its current state, the chunking is just a "RAM-efficient" way to encrypt, but it writes exactly the same encrypted content, as if you did encrypt(...) in one pass. So as long as the file is under ~2^39 bits, it is fine (see https://csrc.nist.gov/publications/detail/sp/800-38d/final).

___

Then, there is another layer of chunking that would be possible, and that would add many benefits: even better deduplication, avoid to reencrypt a whole 10GB file if only a few bytes have changed. This would be an interesting addition, it's on the Todo list, and I know some other programs do it, of course.

But to clarify: this "content-chunking" is independent to the "RAM-efficient" one I use here.

If you want to continue the discussion (more convenient than here), you're very welcome to post a Github issue.

Thanks for your remarks, I appreciate it.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
I'll have a look too.

When using with SFTP, do Restic binaries need to be installed on both local and remote, or just local?

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool

    except ValueError:
        print('Incorrect key or file corrupted.')
So if there is a tag problem, this is clearly logged, and you know something is wrong. The good thing is that you can easily edit it and have instead (pass the filename fn to the function, to be able to log):

     print('Incorrect key or file corrupted, will be deleted:', fn)
     os.remove(fn)
     exit(...)     
Real question: is it possible to `.verify(tag)` before having decrypt(...) the whole file? I doubt it is possible. So an option could be to write the file in a temporary place, and then, only when tag is verified, move it to the right place. Delete it if it is not verified. Another option would be to do a first pass of decrypt(), without writing anything to disk, then get the tag, verify it, and then if ok, redo the whole decryption with writing on disk this time. The latter might be a bit too extreme and halfs the performance.
josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
> Am missing something?

about duplicity and rclone, they don't track file renames/moves in encrypted mode (https://forum.rclone.org/t/can-not-use-track-renames-when-sy...).

i haven't tested the two others (duplicacy, borg), but by reading other comments, it seems the second one has this feature.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
Do you think there is a way to avoid to retransfer files that have been renamed/moved with this method?

This is the use case I mentioned: I often move files from one dir to another when working on audio- or video-production projects, and it can be dozains of gigabytes.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
As I discover it, borg seems to be pretty good for this use case indeed.

But still there is one thing (at least) that can be of interest to some people with nFreezer: it is very simple, it does the job in only 249 lines of code. You can then read the FULL source code in a few hours, to see if you trust it or not. See here: https://github.com/josephernest/nfreezer/blob/master/nfreeze...

If I want to do this with the source code of the tool you mentioned, I would have to spend at least one full week. (this is normal: this program has 100 times more features).

The key point is: if you're looking for a solution for which you don't want to trust a remote server, then you probably don't want to trust the backup tool of a random internet person either. And you probably want to read the source code of the program.

So having only < 300 lines of code to read in a single .py can be an advantage.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
Thanks for asking the question, it's right, there are thousands of backup programs already.

The "key thing" that lead me to code my own is that with nearly all the solutions I tried, data was resent over network when a big file is moved to another dir+renamed, WHEN used in encrypted-at-rest mode.

It's the case for rsync (the --fuzzy only helps when renamed but stays in same dir), duplicity, and even rclone when in encrypted mode (see https://forum.rclone.org/t/can-not-use-track-renames-when-sy...: "Can not use –track-renames when syncing to and from Crypt remotes").

So I wanted to solve this problem, and it works. See point #3 of https://github.com/josephernest/nfreezer#features.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
I will add it.

You will probably be surprised, but I wasn't aware of this one when starting my project! In a way, this is fortunate, because it was an interesting journey to code this :)

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
nFreezer is different to RClone and Duplicity on a very important point (for me because I often move big multimedia projects): if you move `/path/to/10GB_file` to `/anotherpath/subdir/10GB_file_renamed`, no data will be re-transferred over the network, thus saving 10 GB of data transfer.

Indeed, Rclone does not handle renames when in encrypted mode, see https://forum.rclone.org/t/can-not-use-track-renames-when-sy...

Idem for duplicity (tests done).

TL;DR: see 3rd point of https://github.com/josephernest/nfreezer#features

___

Another interesting feature is that you can read/audit the full source code of nFreezer quickly: it's only 249 source lines of code, as of today.

josephernest··on Show HN: NFreezer – Encrypted-at-rest remote backup tool
Hi there! Author here.

I created nFreezer initially for my needs, because when doing remote backups (especially on servers on which we never have physical access), it's hard to 100% trust the destination server.

Being curious: how do you usually do remote backups of your important files?

--> With usual solutions, even if you use SSH/SFTP and an encrypted partition on destination, there will be a short time during which the data will be unencrypted on the remote server, just before being written to disk and before arriving to the encrypted filesystem layer.

Thus this software nFreezer: the data is never decrypted on the remote server.

How do you work with this?

josephernest··on Show HN: Tragopan – Minimal, dependency-free pan/zoom JavaScript library
Having infinite pan/zoom is a very exciting, you always have free room to write new things! I made BigPicture a few years ago for this purpose. Live demo: https://josephernest.github.io/bigpicture.js/index.html

Note: I never did a mobile version, so it only works on desktop; same for the website https://bigpictu.re/demo

josephernest··on I won’t buy ebooks anymore
Are you me? I could have written the exact same blog article! I have the same experience this month, I ended spending hours to be able to use an e-book I bought, and I also finally needed to convert it multiple times, install shady software, etc.

I didn't find an online shop meeting these criteria:

* really big catalogue (of the size of Fnac.com, etc.)

* EPUB with no DRM

I also sent an email to the (French) publisher/editor of the book to narrate the awful user-experience. Maybe you can do the same ;) If enough readers contact the publishers to say they won't buy ebook anymore with such a poor user experience, it might help.

josephernest··on TLDR pages: Simplified, community-driven man pages
1) Great project!

Is there a simple way to install it without having to install Node.js or Haskell or Ruby? (I don't want to install Node.js just for it)

Is there a

    apt-get install tldr
solution on recent Debian / Ubuntu distros without installing another additional package manager?

I mean even the bash solution:

    bpkg ... 
does not work out of the box.

(bpkg or brew or npm aren't installed by default)

2) Is TLDR linked with the project cht.sh? e.g curl cht.sh/tar? Does it share a common source of documentation are they two totally different projects?

3) Few people won't remember tldr.ostera.io (I won't)... Ok we have bookmarks, but still, it would be more convenient if we could just do:

    tldr.sh/tar
    tldr.sh/git_push
instead of having to remember ostera.tldr.io oh no ostera.tldr.sh oh no tldr.ostera.sh no...
← PreviousPage 3 of 6Next →