According to this article, ZFS is sub-optimal for databases due to fragmentation of the db files. Could you please comment? Thx. http://bartsjerps.wordpress.com/2013/02/26/zfs-ora-database-...
29 karma · joined October 6, 2013
There is a sharing feature to solve this problem. :)
But I agree with your point.
I actually looked into WARC earlier but didn't have the bandwidth to do it my first version. When I implement the ability to download your data, I'll try hard to use WARC. Unless there's some brain damage in the format: I hope not! :)
Major points:
* Automatically saves an exact copy of pages (no need to explicitly bookmark) in the background.
* Data is encrypted with your password before being sync'd to cloud.
* Search through your pages.
* Works as a Chrome browser extension. No need for a native app.
You can list all pages you've currently saved by using adding '?n=10000' to the list page URL. Let me know if the 300 number is incorrect.
Thx for the feedback on pricing. Will take it into account.
Copy and paste the same thing into "Autoindex exclude rules" and "Autosave exclude rules".
Could you expand on your feedback on pricing? Can't tell if you're saying $5 is too high/too low. If you wish you can also email me (I couldn't find your email on your site).
Logout if extension not installed is related to the same reasons why saved passwords don't work yet.
The timer is supposed to be a debug feature that I thought was off by default! :) It shows the amount of time you've spent in that tab. I'll turn it off by default in the next update. For now, go into the extension options and uncheck "show page view timer" (near the bottom of the options page).
wrt taking quite some time: the first time you save a page from a site, it's likely to take some time, since things like static assets will also be uploaded. Subsequent saves should be much faster due to dedup'ing which works well on static assets. Let me know if this is not the case.
With PurpleRails, you will rarely be typing your password once you successfully login. It works kinda like Gmail/Facebook etc: you remain logged in for months at a time.
This page explains why password saving doesn't work yet: https://www.purplerails.com/blog/saving-passwords-why-we-hav...
I'll see if I can do something fancier than what I do now to allow saved passwords to work. I suspect that using a Javascript-based AJAX authentication system instead of a plain-old HTML form, I can achieve the privacy goals as well as ease of use.
I hope you will hang in there and use the current version until I figure out a way to fix this. Thanks!
Time limitations are what is preventing me from doing this.
Thanks for your feedback! Hope you will use Purplerails. :)
The basic algo is to generate a HMAC of the plaintext and compare it against a table of previously uploaded blobs' HMACs. HMAC is keyed with a key derived from the user's password. When a blob is uploaded, the ciphertext and HMAC of the plain text are both uploaded.
I anticipate that PurpleRails will be used on multiple computers and over several years. Which is why pages are sync'd to the cloud.
I've adopted the current architecture because I feel that the energy barrier that needs to be overcome to persuade somebody to install an app is much higher than the one to install an extension.
Please see my reply to idlewords. It isn't literally everything; there are some heuristics to detect what was interesting to you.
I understand you might find this excessive. I routinely find that useful. :) See also "As We May Think" by Vannevar Bush.
> and it is terribly inefficient to store say, 30,000 examples of the exact same article ...
You're right: no deduping is and can be done across users (HMACs chained up the user's password are used create dedup hashes). Storage is sufficiently cheap that I feel it's a acceptable tradeoff vs. privacy (i.e., server being unable to confirm that two users have saved the same page).
There are a couple of heuristics to avoid wastefully uploading pages: the full page is uploaded only if the reader expresses "sufficient interest" in the page. Currently the heuristic is 90 seconds of continuous reading of a page, or scrolling to the bottom. If a page is read for a minimum of 10 continuous seconds then only the text of the page is uploaded.
Static assets like JS files benefit from deduping: they take time to upload the first time, but subsequently processing them is much faster.
Typically, people read multiple pages in a browsing session: I rapidly open many tabs and then read each one for multiple seconds. There's a debug mode in Purple Rails in which a timer counts up when I switch to a tab. I find that typically spend 100+ (usually much more) seconds on a page that I read through to the end. This is usually enough time on a residential broadband connection (I have Sonic DSL) to finish uploading a page. I also use Purple Rails on a tethered 4G connection almost everyday: uploading is slower than DSL but it works.
Basically, by the time you finish reading a tab, the previous tab you read would have finished saving.
Like I said, nothing too fancy.
2) privacy-first architecture (e.g., plaintext is never uploaded, plaintext URLs are never uploaded etc)
The main idea is to use a browser extension to automatically save pages that you read to the cloud (including the images, stylesheets etc) in the background. Saved pages are searchable and sharable.
I tried address the privacy angle from day 1. Data is encrypted and only ciphertext is sent to the cloud. Index is stored only on your own machine. Searching occurs on your own machine.
A couple of cool features are that it also saves an exact HTML copy of the page including images and stuff if you read a page "long enough" (currently 90 seconds).
Been in beta for a while. Thanks for feedback.
(1) Saves an exact copy of the page also.
(2) Indexes the text.
(3) Encrypted (search occurs on your computer).
Been in beta for a while. Thanks for feedback.
A comment in the code about why it's OK in your case to use the same key for MAC and encrypt would be useful. I think you're fine. See here: http://security.stackexchange.com/questions/37880/why-cant-i...
I needed to implement deduplication in my system. Since I controlled the server, I developed a slightly more elaborate system which doesn't have the limitation of a predictable IV (predictable from the encryption key).
So in my system, I derive two keys from the same passphrase (PBKDF2 with different salts). I encrypt as usual with unpredictable IVs. When uploading, the HMAC of the plaintext and SHA-256 of the ciphertext are both loaded.
To check for duplication, the client asks if a certain HMAC is already present. And it's an error (at the server) to upload multiple ciphertexts with the same HMAC.
The ability to sign out of all other sessions.
Without this a user who forgot to logout from a library would be out of luck until the session expired.
i..e, why couldn't you have written:
if (e.origin !== 'https://clef.io') {