gajjanag

> Oh, that must be it. Indian people care less than people of other nationalities, and it has nothing to do with... say, drastic disparities in the quality of public infrastructure.

Public infrastructure in India ultimately derives from the people. India has had 60+ years of independent, democratic rule. Blame for the lack of public infrastructure can't even be shifted entirely onto the colonial rule: India had some of the earliest hydroelectric power stations and pretty good rail infrastructure set up during that period.

Different cultures assign different values to various things. India collectively has not assigned a huge value to public hygiene. This is evidenced by things like the paan chewing and spitting mentioned in this discussion. This is socially acceptable in most parts of the country.

Slowly, things may change.

Same in India, there are signs and rules against such usage.

But like many such things in India, very few actually care: http://www.thehindu.com/news/national/karnataka/you-may-be-f....

> I know of folks that own Audis and Mercs, but live next to a dumping yard. Now what is the point of having all that money, when you get see/smell only filth most of your life.

The point is that one gets to compare it against the general population and thereby satisfy one's own ego. Generally, cars are a nice instrument for showing off wealth: they are highly visible. Homes are another excellent example.

You highlight very well the absurdity of valuing such monetary wealth in a context where it makes little sense.

Aren't there problems where large scale data sets are not available, even for the big players?

With small amounts of data and/or unconventional learning situations, there should be options for incorporating domain expertise in interesting ways. I might be wrong, but I think the fraction of problems that can be solved with deep learning/large data set analysis is a small fraction of the set of AI problems as a whole.

For example, at a statistics symposium at MIT last year Stanford Professor Robert Tibshirani mentioned a healthcare (cancer diagnosis) problem, done in collaboration with healthcare people: http://www-stat.stanford.edu/~tibs/ftp/canc.pdf (these slides are different from the ones used at MIT, the content is similar though). He mentioned how they talked to some deep learning experts at Google, who told him that the data set was too small for effective deep learning usage. Another reason is mentioned on pg 30 of the above, basically sparsity being more important than nonlinearity here.

In healthcare, people care a lot about "interpretable" models. The term is ill-defined, but in a sense it is something where the trained model is easily explainable, and easily written down. A linear model is a simple example of such a thing, though usually insufficient on its own. Deep neural nets arguably do not meet this, hence a reluctance (among other reasons) to shift in certain fields.

Your point may still be valid though - I do not know whether there are enough viable economic opportunities. Computer vision in particular is dominated by DNN/large scale analysis these days, and I sympathize with your difficulties here.

The problem is not necessarily the expense - there are a whole bunch of steps that need to be taken in order to ensure that the big hosts don't flag your mails as spam - https://support.google.com/mail/answer/81126.

And even if it seems to be working fine at the moment, there are no practical guarantees that this will not be subject to change: https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA. Even someone as high profile as Linus Torvalds had to resort to the classic method of public shaming in order to get a prompt response.

Lastly, there is not much incentive even if I cared about privacy - my typical recipient is anyway a Gmail account, so Google can definitely construct a shadow profile for the sender even if the sender does not use Gmail.

Again, it is a classic problem of a heavily centralized internet.

That is good intuition, but note that one of the general drawbacks of Cramer Rao is that it applies for the unbiased case only (technically no, there is a version for the biased case, I have never seen it used though), and that too only well under certain regularity conditions. Indeed, there is a term for estimators where equality is met in CR, they are called "efficient" estimators.

There is significant research effort these days on going beyond unbiasedness and asymptotics, which in many cases has been beaten to death and is "classical", though there are still plenty of questions.

Here is a simple illustration of unbiasedness/going beyond it. Consider n iid (independent and identically distributed) coin flips of coin of P(heads) = p, and based on the observations, we want to estimate p, call the estimate p'. Optimum estimate in the sense of minimax - min (over estimators) max (over p in [0, 1]) E[(p-p')^2] is not the simple maximum likelihood which is the sample average. Sample average yields a score of (1/4)(1/n), while optimum is (1/4)(1/(sqrt(n)+1)^2), which is slightly lower. It is natural to wonder why people care, typically the motivation is in high dim statistics where there is dependence on the dimension as well. In such a case fine grained characterization of performance sometimes affects the scaling in terms of the target loss amount and dimension, i.e the asymptotics now has at least 2 parameters, sometimes more.

My personal unhappiness with these problem formulations is their lack of robustness to transformations - suppose I now want to estimate p^3 instead of p, the optimum min-max will change in most cases, and definitely does here. Plugging in the sample average and then cubing is far more intuitive, and often works fine even though it is not optimal. There are more extreme examples of this, but that will take the thread too far afield.

If one is interested in the topic of biased estimators, I suggest reading about James-Stein estimators and more generally shrinkage estimators.

There is a beautiful paper by Bin Yu on three common methods for getting minimax lower bounds on risk title "Assouad, Fano, and Le Cam" - https://link.springer.com/chapter/10.1007/978-1-4612-1880-7_....

If one is interested in a take on this problem by an anthropologist/archaeologist turned epic fantasy writer, touching upon some of the thoughts in the comments here, see: http://www.steven-erikson.com/index.php/commentary-endgame-v....

On the downside this is a rather long essay, on the plus side it does not require familiarity with the author's work.

I agree with these sentiments, but for slightly different reasons.

One of the main selling points is that commuters can save time (the focus on self driving cars, not trucks/rail). However, long commutes are mostly a constraint imposed by suburban sprawl and associated planning failures. The same applies to automated delivery mechanisms from shops, again of limited use in sufficiently dense communities. In such a scenario, this feels a lot like optimizing 1% of execution time in a program.

For large scale transport of goods or people, the same situation applies for the economics - the labor of the driver should be a small fraction of the operating costs. There could be some safety benefits, but this remains uncertain. I found https://www.bloomberg.com/view/articles/2015-05-28/cars-will... quite interesting in this respect.

Seeing what happened with the invention of the automobile, I won't be surprised if self driving cars become the norm somehow. In the most extreme case there might be an outright ban on non autonomous cars at some point in the future. This is something I certainly don't look forward to.

You are right in general, but note that for instance there are still many 2 stroke auto rickshaws: http://www.autocarpro.in/news-national/karnataka-ban-polluti....

Typically, the game goes as follows: new laws get drafted, local groups such as auto rickshaw unions get together and demand time/cash incentives to move to more expensive, cleaner equipment, local governments grant extensions, and the cycle repeats.

Then there are the general issues plaguing enforcement in India, which are listed in the wikipedia article.

> This is even possible because auto rickshaws and buses have gotten expensive. Public transport of any kind is expensive, unpredictable and not worth when you look at the overall comfort and economics of having our own scooter.

When I lived in Bangalore, and whenever I visit there, BMTC (the public bus transport provider) has buses at least every few minutes along the main corridors during the day.

They are often crowded, but I don't think I could call the service "unpredictable", beyond the general variations that apply to all means of vehicular transport in Bangalore. Last mile connectivity though is often painful.

The service is indeed expensive, especially when one is sharing a commute with another person.

This said, I really like your summary of Bangalore.

Per http://hariharan-ramesh.com/sports.html, it seems like your options are sadly limited, though you could give the linked Respro a shot.

> Google is teaching their car how to navigate city streets, deal with human drivers and pedestrians, navigate intersections, handle construction zones, deal with emergency vehicles and cops directing traffic, etc. All with a much better sensor package than in a Tesla Model S.

The interesting thing is that no matter how much one anticipates situations a priori and builds them into the AI/model, there will likely exist situations not anticipated by the engineers.

For example, it is not clear to me how exactly a situation like https://en.wikipedia.org/wiki/Driving_direction#Sweden will be handled. Will this be "patchable" (never mind the logistics and security issues of providing updates to a nation's fleet), or will it require a full retraining of the AI? Or more generally, how robust/modular will the designed AI be to these kinds of situations?

The reason I like this example is because it is a classic instance where humans have inertia and thus have difficulty in consistently applying the "switch driving side" rule - I have seen international travellers requiring at least a few minutes at the wheel to reorient themselves.

In principle one would hope that a modular software solution can handle things more consistently.

A link that focuses on higher level issues regarding I/O streams in C: https://www.irill.org/videos/GNU_Hackers_Meeting/jim-meyerin....

In particular, I am not too happy with the title ;).

All fine, but it is left conveniently unspecified what is meant by "reduce your rights". In fact, I don't even know my rights as I was not explicitly presented them in the form of a list in the first place, making the "reduction" ambiguous.

Admittedly it is not as easy to spin this as a more explicit ToS, but I am sure that with a sufficient number of legal experts, this has considerable leeway in terms of interpretation.

Some general thoughts.

I would guess that the first step is acknowledging that the problem exists and is not made up by a few disgruntled people.

The next step is understanding, for which there are plenty of resources on the web as a start. I believe that living in another country for at least a year is helpful. Note that it is very easy to fall into the trap of insulating oneself from the local culture by simply trying to copy over one's main country of residence. I have seen it happen all too often with many NRI's in India.

Then comes the really hard part, which is creating solutions based on such understanding.

How about being able to build taller buildings in order to support higher population densities in major cities?

Most cities in the US have all kinds of zoning regulations that prevent cities from reaching higher population densities: https://news.ycombinator.com/item?id=11918230. For example, there are quite a few livable cities in the world supporting > 15,000 per sq km, most cities in the US hardly reach half that amount.

This is definitely not a free market phenomenon; I am pretty sure that in a free market (or even a market with rationally determined, not NIMBY based regulations), there would be increases in population density of major cities in the US.

> That reduces, but doesn't eliminate, the amount of information you're leaking.

All cryptographic systems save the one time pad do leak some information, in the sense that they are not information theoretically secure. For instance we know that the output of standard block or stream ciphers (with fixed plaintext) is distributed over an exponentially small fraction of the possible output space.

So really the question here is whether one can pad to the point where it is computationally infeasible to launch this kind of attack, and whether this padding amount is so large as to defeat the compression entirely.

For example, two normal distributions with variance 1, means ~ 2^(-k) away from each other to each other can require ~ 2^(2k) trials for a constant probability of hypothesis test success (say 1/3).

Has there been any work analyzing this rigorously, taking the length leakage as side information available to the attacker?

Exactly. I have Signal installed, but none of my contacts have it installed on their phones.

And Signal's "helpful" banner at the top that allows easy invitation of contacts will (rightly) be treated as spam by most of my contacts - these secure/encrypted chat apps keep coming and going, with some people saying "just use Whatsapp, it is now encrypted", others complaining that it is closed source, etc. I can't keep changing my recommendations every few months and expect my contacts to do the same. In fact, it is only through hn that I even heard of yet another protocol: https://matrix.org/.

Practically speaking, WhatsApp works best wrt security for me due to these network issues - essentially all of my Indian contacts have it for some reason, likely again some positive feedback loop due to network effects in India. And there is no such dominant app among my US contacts, hence I fall back to plain old SMS.

As a learning exercise/fun project for the author, I think this is ok.

But for any serious study, I fail to see what this offers over the completely free, very readable, more carefully written, and more thorough "Information Theory, Inference, and Learning Algorithms" by David MacKay: http://www.inference.phy.cam.ac.uk/itila/book.html.

As an example of the problems with the pdf in its current stage, Theorem 2 (asymptotic source coding) includes the term "negligible loss" without even defining what "loss" means in source coding. Lossless and lossy coding are very different things, all the preceding stuff is really discussing the lossless coding problem. Pedagogically, these need decoupling.

> This is what's wrong with Linux kernel development. We have pluggable subsystems like I/O scheduling or CPU scheduling where BFQ or BFS are not mainlined and have to live as external patchsets. Then we have more than a few competing tracing/probing frameworks all somehow magically in torvalds/linux.git despite their overlap and unfinished (partly experimental) state.

> I used to think that Linux kernel development is about who comes up with good, working code, but more and more I'm starting to get the impression it's that you need to have the right connections for a competing implementation to land.

I think this is true of all sufficiently large organizations with human interactions: politics is an integral part of them.

It also reminds me of Con Kolivas's reasons for no longer working on the mainline kernel: http://forum.digit.in/chit-chat/81361-why-i-quit-kernel-deve....

Generally, I just hope that server developments and improvements trickle down to my personal computing devices.

Indeed.

I am not familiar with the technical details of XP, so these are mostly observations from a user and not developer perspective:

1. On my old Pentium 4 box with 256 MB of RAM any release after XP hardly met the OS requirements. Yes, hardware has come a long way since then, but IMHO there is no technical reason why a kernel with a bare bones desktop needs more than this amount. I never understood why Microsoft's recommendations for RAM went up to 1 GB and beyond. My pretty standard Arch Linux setup in 2016 does not take more than this right after boot.

2. XP had a rather long lifespan as an OS. It was only later that Microsoft got into the 2-4 year upgrade cycle, hoping customers would purchase a lot of the upgrades. What happened actually in many cases was a simple skip of alternate upgrades, concretely Vista and 8.

XP definitely had some pretty bad problems, like the fact that it was not really ready for the 64-bit era, manifested in things like the 2 GB limit on memory per process. I don't know about what they did with the 64 bit XP version; it looked a lot like a stop gap solution.

I used the EOL announced for Windows XP support as a nice excuse to get myself to use a GNU/Linux system. I have not looked back since then.

As an aside, there are still some holdouts of XP usage, such as some lab equipment software written many years back. In fact, Windows XP embedded was supported all the way till January 2016.

I have not implemented a compiler so I can't really judge on that aspect.

Isn't it also true that undefined behavior can stem from varying hardware/platform considerations? In particular, leaving some things judiciously undefined allows flexibility in implementations and thus can give good performance across a variety of platforms. Sure, one could force some precise semantics, but that might result in unnecessary "emulation" code on some platforms.

So at a programming language level, by having such emulation/fixed behavior, the semantics are simpler. But at a low level, the semantics are more complicated as the machine instructions may differ significantly, thus losing transparency. This is a concern in quite a few resource constrained applications.

For a simple example, see http://blog.llvm.org/2011/05/what-every-c-programmer-should-... and the section on oversized shift amounts.

I like your answer, but I was really referring to widely used languages.

> Not when you take the arcane undefined behavior rules into account. C semantics are anything but simple.

Maybe, but note that simplicity should be viewed in a relative sense.

I am curious as to which language you think has simpler semantics than C. For example, I have not found Python, Julia, MATLAB, C++, Verilog, or shell script simpler than C. Same goes for my initial explorations of Rust.

Even if one includes the arcane corners, the spec is < 200 pages (excluding the stdlib).

On what basis can "notably most secure browser" be asserted?

I think both aspects of this statement are definitely not clear - certainly not notably (reference?), and comparisons based on some searching seem to go one of numerous ways.

I would be happy to see a security expert's view on this though.

See also this link on Chrome devs essentially telling users that upgrading their kernel from 3.16 (not even > 7 months old at the time): https://news.ycombinator.com/item?id=9164251. Thankfully better sense prevailed and it was resolved fast. So even in terms of OS support it is not unambiguously better than others.

> People forget that Linux is the dominant kernel/os because it dominates nearly every market that isn't the desktop (server/mobile/IoT/toasters).

This is mostly correct while referring to the kernel. Nevertheless, a kernel by itself is just one component of the low level system. And I personally definitely do not consider the Android experience a "GNU/Linux" one - even the base libc is replaced with a quite broken (e.g wrt POSIX) Bionic. This is leaving aside all the firmware issues, lack of administrator control without "rooting", etc.

So in other words: it is natural to "forget" that Linux is widespread on mobile devices, simply because the experience is extremely different.

Exactly. What I have found is that when I examine deeply enough, most DRM content is not even worth it, and even for purely entertainment purposes, there exist plenty of non DRM solutions: movie theaters, libraries, outdoor activities to name just a few. I found Michael Niedermayer's quote very nice in this regard: "Breaking DRM is a little like attempting to break through a door even though the window is wide open and the only thing in the house is a bunch of things you dont want and which you would get tomorrow for free anyway" (https://ffmpeg.org/pipermail/ffmpeg-devel/2016-January/18824...).

And if you really don't even want to accidentally support or open such DRM content, you can configure the browser appropriately (media.eme.enabled and --disable-eme in Firefox).

On most Linux distributions, Firefox is built from source and then shipped via a package manager. At least on Arch, --disable-eme is not passed as a configure flag. Yes, one could build Firefox oneself, but it is a beast to compile. It is also ridiculous that this needs to be done at build time.

A perhaps easier approach that should work based on the name (but have not confirmed) is to lock via the user.js method user_pref/lockPref("media.eme.enabled", false). If this works as intended, then simply make this config file part of the deployment should be enough.

+1. As someone who has worked in the field of information theory, I strongly dislike the abuse of its usage here.

Funny that the term "information theoretic" is used with not even one "theorem" in the linked document.