ctur

There is a good book about the evolution of written communication as the Internet evolved. "Because Internet" by Gretchen McCulloch goes into some of the subtle implications of using punctuation such as periods ending sentences, capitalization, etc. Text makes it difficult to convey emotions and subtlety so even using punctuation can add signal to the communication channel. It turns out there is relatively powerful tonal communication through those deviations on "standard" English grammar rules. Similarly, Emoji also acts as a replacement for gesturing.

Very interesting book and worth reading to understand how to tailor communication for the medium.

Even better -- use this script, paste anything into stdin. It will come back out quotable and pasteable into a shell.

  #!/bin/bash
  
  printf '%q' "$(cat)"
  echo

This truly will lead to the next generation of unicorns where engineering is combined with marketing to innovate entirely new business models from the to-be-realized latent synergies of the previously divergent disciplines.

This post just seems to be to draw semi-sensational attention to a self-help book. The site isn't generally referencing primary sources or otherwise giving signals of why it should be trusted. It also doesn't identify the authors.

It's hard to take the site seriously when it equates gaming to being an alcoholic (while later in the same post backpedaling it): https://www.deprocrastination.co/blog/should-i-quit-video-ga...

Seconding Cronometer as a good tool; it’s the best I’ve found, supports data export, and isn’t full of “social” features that obscure the key feature: easy and accurate calorie tracking. In particular it’s database is more accurate than the crowd sourced ones like MyFitnessPal.

Don’t just walk past; give them some cash or carry gift cards for grocery store or pharmacies to hand out. Then also donate to a local shelter.

Give them cash. Give them dignity by making eye contact. Give the benefit of the doubt they need it and will use it responsibly. Give.

For those who can't upgrade yet, there is an open source utility, Unshaky (https://unshaky.nestederror.com/), that can help with double presses. It's sad and gross that it is necessary but it works well on my aging MBP.

Reading the other news coverage and not just Povey’s screed, it sounds more like white fragility than genuine injustice.

I think XARs are much simpler. snapcraft (and AppImage and other tools) seem to be much more invasive to the development flow and more opinionated about what the contents of the image look like (directory structures, yaml/xml/json files for metadata, etc).

XAR is much simpler. All it needs is a shell script (or raw executable) to run inside the XAR. Everything else is up to the bootstrapping. Note this isn't always a good thing -- opinionated software has its place. For what we use XAR for, though, having it "just work" with standard Python tooling without expecting open source libraries or modules or build tools behave differently is a strong plus.

I am not an expert in AppImage but I think the main thing is it is pretty easy to make a XAR of a Python script and that the overhead is probably less (especially for repeated invocations).

I think XAR is simpler than the other alternatives; they have more complex specifications, being aimed at distributing full GUI applications (though I bet they work fine for these kinds of use cases, modulo perhaps quick and easy conversion of a Python program into a XAR). I suspect, but haven't measured, that XAR has lower execution overhead since it will re-use a mount point if a tool is recently invoked, rather than remounting every time.

One thing XAR doesn't really try to do is work cross-platform. It will rely on the system Python, for instance, rather than embed the interpreter and all libraries inside (it will embed the libraries your tool depends on that aren't part of Python itself). This is a pro in some cases (lower overhead), but a con if you want something you can carry across wildly different systems.

XAR development began in 2014 and hit production use in an early form in 2015; this was before some other options were out there, or before, I believe, AppImage went the squashfuse route.

XAR doesn't aim to, say, represent a sandboxed filesystem or provide some isolation or provide a specification for GUI apps to seamlessly integrate with your desktop. Instead, the main idea is to just get a single file that, when run, mounts a filesystem and runs a command inside it. In my mind, at least, it's a simpler, smaller primitive you could build other systems on top of (like we have with deploying Python, Lua, and Node).

I agree it is a bummer that FUSE doesn't directly work on Windows, but it should be doable -- we would love for someone to figure out the best way to do this on Windows. Happy to collaborate with anyone who'd like to make this a reality.

Not related, but at first glance it looks similar. However, it appears that ruby-packer uses a library to access the squashfs image rather than actually mounting it. This makes it difficult (probably impossible) to load .so files from the squashfs file without having to copy them externally or to carry other helper executables along inside the image without also having to copy them externally to invoke.

A big goal for XAR is to be utterly transparent and not require hooks into Python, Node, or other runtimes to function.

We generally deploy XARs like a normal executable (delivered via RPMs or other native packaging). We also have an internal packaging format we use to deploy them as well. The big thing for us is we treat them like normal executables, so any standard deployment model for an executable works for XARs.

We actually started with using "real" squashfs files. This had three main disadvantages:

- We had to maintain our own setuid executable to perform the loopback setup and mount (rather than relying on the far more tested and secure open source fusermount setuid binary that all FUSE file systems rely on) - Getting loopback devices to behave inside of containers (generally cgroup and mount namespace containers) was a little tricky at times in some of our environments - We didn't want to have a huge number of extra loopback devices on every host in our fleet

In fact, after implementing the loopback-based filesystem version, we almost abandoned XAR as the downside of the security considerations and in-container behavior wasn't ideal. The open source squashfuse FUSE filesystem really is what made it possible.

Another side benefit is we could iterate far faster with squashfuse -- this let us fix some performance issues, add idle unmounting, and implement zstd-based squashfs files, and then deploy that to our fleet, faster than we could deploy a kernel to 100% of hosts.

Yep, it's similar, but squashfs is more optimized than zip files for random access like a filesystem (rather than an archive). Also when using zstd-based squashfs files, there is much less overhead for the decompression itself which effectively becomes free.

One syllable, rhymes with "car" but with a "z" sound ("zar").

FUSE isn't generally lighter weight than a filesystem but it can be relatively competitive for simple use cases like a read-only filesystem. Additionally, squashfs lets you pack metadata and data very tightly, and since it is a readonly filesystem, has some optimizations normal filesystems can't (how data is placed, overhead of managing metadata operations, etc). Also squashfs lets you choose how the files are laid out and compressed so that all files of a certain type, such as all .pyc files, are close together, which increases compression ratio and reduces overhead for subsequent file accesses (i.e., can reduce random disk or flash IO).

In practice the timings of XAR vs filesystem are close enough to be "in the noise" -- it's when compared to PEX or PARs that the difference is quite large.

Generally XARs are lighter weight than a container. While you can (and sometimes we do) use XARs to deploy, say, a self-contained service like a website, often they are used to replace command line tooling.

Container isolation (cgroups, namespaces, etc) would make it difficult to do some of the system level tasks we use such tools for, such as configuration changes or monitoring.

Likewise, we often are replacing a PAR or C++ tool with a new XAR version, and it is nice to simply replace the executable and not have to change how it is invoked. In this regard, invoking a XAR is identical to running any normal executable or shell script.

Hi, I'm Chip, one of the authors of the blog post and XAR itself. Happy to answer any questions anyone may have about how XARs work, the way we use them, or the motivations that drove their development.

This rings true to anyone who has owned a Tesla, especially if they purchased it when the model was new. Repeated trips to get minor things fixed is not uncommon.

Nonetheless, they are excellent cars and worth the early adopter frustration (IMO).

I've used it for the same purpose, but one thing to bear in mind is the problems are incredibly focused on a small domain, so while you can pick up some bits of a language, it is nowhere near a thorough introduction.

It often is, especially if the tarball is compressed (especially if you use anything but Zstandard) or on a network store rather than fast local ssd. Typically a rule of thumb is a hard drive with a spindle may five around 100mb/sec throughput, so that is likely the bigger bottleneck, followed by cpu if using slower compression algorithms.

Zstandard at a high compression level gives a good tradeoff of decompression speed vs underlying physical throughput, be it network or spindle.

Even so, yes, the sequential nature of tar is not great for this reason.

Squashfs files, which can be used on Linux and Mac (using fuse for OS X), are a very good alternative for archIves, though without some of the features described in the linked article.

I definitely understand reluctance to trust new data formats. I will say though that we have been using Zstandard in production at Facebook since well before 1.0 and trust it completely for both data at rest and data in flight. Since 1.0, we've widened that even further to many different parts of the company.

To that end, every release (and pretty much every commit) undergoes significant testing. We also fuzz test heavily and have significant regression tests in place constantly looking at corner cases and verifying known prior bugs stay fixed. It's impossible to prove there are no issues, but so far we've put a lot of effort into being as reliable as possible and seen that play out well in our production environments. We trust it.

We currently are exploring a seekable format for Zstandard right now. What use case do you have in mind -- network or at rest? If you want to file a github PR describing how you'd like it to work, that will help guide our implementation. We have a few in mind based on internal Facebook use cases, but the more different needs we know about, the more general purpose the result will be.

The Fourth Estate is doing its job. Whatever the truth is, the pressure from respected institutions like the Washington Post and New York Times will be a big part of why we eventually find it. I am proud that the press is holding the government accountable.

I'm not sure when videos will be posted -- hopefully soon.

All benchmarks today are single threaded. The algorithm itself is single threaded, but can be parallelized across cores. We will soon release a pzstd command line utility to demonstrate this, similar to pigz, which accelerates both compression and decompression.

Zstandard uses both huff0 and FSE together when it compresses -- it doesn't switch between them based on the input.

Yann will be giving a talk on Zstandard at today's @Scale 2016 conference, and the video will be posted. He can answer the most technical questions about Zstandard, but I may be able to answer some as well; we both work on compression at Facebook.

The format itself is documented (https://github.com/facebook/zstd/blob/master/zstd_compressio...) with the intention of other implementations and language bindings being readily available. We also have a zlib-compatible API for easier porting to applications already using Zlib. Our hope is that Zstandard is both easy to use and easy to contribute to.

I would love to see Kraken taken for a spin by objective third parties in a fully transparent way that compared it to the available alternatives on equal footing.

Sadly, a compression benchmark that doesn't detail the compiler, hardware, corpus, or even version of the libraries used is not very compelling. If others cannot reproduce your results, the results aren't of much value beyond marketing.