timtadh

It looks like v2.1 has fixed that.

Also, +1 this is awesome.

@dang if you are here: HN should do this natively!

In response to several threads here: it is important to distinguish when scientists are self critical vs. when non-scientists are critical of the scientific method. For instance, there is a long history of scientists criticizing how the scientific process is currently conducted for the purposes of improving the scientific endeavor. That work is sometimes used by non-scientists who question the overall scientific method. However, such use is invalid as the scientific self-criticism

1. assumes the validity of the scientific method

2. relies on the scientific method as its critical lens

Whereas those who critique science as a whole:

1. assume that the scientific method does not work and does not arrive at "truth"

2. then use scientists being self critical to prove #1.

Such a "proof" does not work as there is its uses the assumption "the scientific method arrives at truth" to derive the contradiction "the scientific method does not arrive at truth". See for instance comment: https://news.ycombinator.com/item?id=16859200

In reality, work on reproducibility is about improving the practice of science overall. It does not in itself show that science is inherently untrustworthy. What it does show is that scientific discovery is difficult and it takes a lot of effort and new findings should be treated critically. What does critically mean in this context? It means with in the boundaries of science analyzing the theoretical basis, hypothesis, method, and experimental results for potential flaws. It does not mean to be skeptical as a default because science "doesn't work."

I too wish there was more standard containers available in Go's standard library. However, I don't think there will be a collections package unless and until generics make it into the language. That said, there are some pretty good libraries out there:

- https://github.com/timtadh/data-structures ## I wrote this one.

- https://github.com/Workiva/go-datastructures ## this one is also popular.

- https://github.com/golang/go/wiki/Projects#data-structures ## big list here.

Most people who criticize the Dragon (Compilers etc... by Aho et al.) book seem to focus on chapters 3 and 4 which are the chapters on lexical analysis and parsing. The book has 12 chapters. Whatever your feelings on the parsing techniques, the book covers WAY more than that. It has a really good introduction to code generation, syntax directed translation, control flow analysis, dataflow analysis, and local, global and whole program optimizations.

As someone who has quite a few books on compilers, program analysis, type theory, etc... I find the Dragon book an irreplaceable reference to this day. It has a breadth of content shared by very few other books. For instance, Muchnick's classic "Advanced Compiler Design and Implementation" is really good for analysis and optimization but neglects all front end topics. The only area where I believe the Dragon book is inadequate in is type theory (I recommend Types and Programming languages [TAPL] by Pierce and Semantics with Application by Nielson for a gentler intro).

As to parsing, its chapter on parsing (4) is not as "hip" has some people want. However, it is solid and will teach you how to do parsing. There are newer and fancier techniques not covered in Chapter 4 but in general most people would benefit just having a solid understanding of recursive descent parsing!

Where I live $80k is a significant portion of the cost of most houses. The price will be a significant barrier to entry for houses outside of hot real estate markets.

> And there goes Shakespeare. On average there's 15,000 words per play. I bet most readers do not have the education to know every single word in that 15,000.

In English (as I assume in Chinese) you can usually figure out what is being said in Shakespeare even if you don't know the exact definition of the word. You also can usually pronounce it correctly (at least for the modern pronunciation).

@suryabhupa How similar is this work to the Grammatical Inference field? There has been a lot of work over the years in specification inference which feels similar. Many of the studies in specification inference learn automata representations of object interactions. I know there have been other application grammatical inference in Software Engineering as well.

I case people didn't click through it is an awesome comment by the original author of the IE5 DOM Tree explaining how it was implemented.

Then again Steve Cook who proved the existence of the class NP-Complete was awarded the Turing Award only 11 years after his paper was published: http://amturing.acm.org/award_winners/cook_n991950.cfm

I guess I live in a small world because I have no idea how to type an umlaut on a US Qwerty keyboard but I can easily type Paul Er\"{o}s.

+1 for admitting to snobbery it is an important step!

Different problems (and people) have different solution domains. For instance, the Union-Find algorithm [1] is straight forward to implement in an imperative language with mutation. It is significantly harder to achieve an optimal immutable version as demonstrated by a paper from 2007 [2].

There are real trade-offs between features in terms of what is easy to express. Your favorite way to program may be more difficult in Go. You think Go's features are in "poor taste." However, it is totally reasonable that different people with different problems might actually like the language. I myself have programmed in many other languages including SML and Scala and believe that for my current problems Go is a good fit.

That said, whenever I have to do even a little bit of numerical work (as I currently am doing) I miss a language with good numerical options (Python, R, Matlab, Julia, ...). Go stinks for numerical work and none of your criticisms have anything to do with why it stinks. Not having a nil pointer would not suddenly make Go a great language for numerical work.

Language choice is once again about trade-offs. I will happily take the trade-off of poor numerical support (less than 1% of my code) for concurrency primitives, compilation to native code, easy C integration, memory safety, and garbage collection. There are things to like about go and things to hate. I do hate the way errors are dealt with, poor support for writing collections, etc... But, just because I don't like those things doesn't mean it can't "get the job done."

[1] (https://en.wikipedia.org/wiki/Disjoint-set_data_structure)

[2] (https://www.lri.fr/~filliatr/ftp/publis/puf-wml07.pdf)

I have do (academic) work [1, 2] on finding semantic code duplication. Two points that I have learned about code duplication:

1. There is are a lot, A LOT, of code regions that share similar constructions when analyzed in terms of dependencies. Dependencies only consider data flow and control dependencies. Where a statement X is control dependent on another statement Y (usually a if-condition or loop-condition) if Y decides whether X executes.

In my studies I have found modestly sized Java programs (~ 75 KLOC) have > 500 million patterns representing duplication in their dependence graphs.

2. Not all dependence structures which are "duplicate" would be considered duplicated by a human programmer [2]. It takes discernment by someone familiar with the code base to decide whether or not regions are actually duplicated.

I would argue you can draw similarities using automated metrics between disparate code bases. Those similarities are not evidence of copying. To decide whether similar regions are actually copied you would need to do further and subjective analysis. Without directly evidence of copying it would be very difficult to make a solid claim one way or the other. But, given the vast amount of similar code regions that exist (and assuming most code is not copied) I believe it should be given the benefit of the doubt.

Note: I have not studied density of duplicated code between different projects. The above is merely an conjecture based on my experience.

[1] http://hackthology.com/rethinking-dependence-clones.html [2] http://hackthology.com/sampling-code-clones-from-program-dep...

A good general method for comparing ASTs is Tree Edit Distance. I have an implementation in python https://github.com/timtadh/zhang-shasha . If computing the exact distance is too slow you can use an approximate algorithm https://github.com/timtadh/pygram .

It is in the poster track. That is a different (much, much lower) bar than research track papers.

That is a reason to use the AGPL when working on free software. Using a BSD license gives you (the creator) zero protection from bad commercial actors who want to use the software without contributing back. The AGPL is not the right choice for every project but I do believe it is a good choice for software which can be trivially setup as a service. For instance, an open source web application, an open source database, an open source irc server, etc... I think Linus did the right thing going with the GPL for Linux and sticking with version 2.

He does have 10 publications from his time as a PhD Student. https://dl.acm.org/author_page.cfm?id=81100544101

Linux laptops: buy pre-installed. If you want linux and you want to be sure everything is functional, AND you don't want to devote lots of time to a "project" buy a pre-install. Dell Developer Edition laptops (XPS 13, 15) make great laptops. I hear good things about system 76 and purism as well.

Save time. Support having good laptops and good drivers. Buy pre-installed. Paying the "windows tax" and installing a Linux on a windows laptop isn't just more work, it is bad for the ecosystem.

Going from $500 - $700 to $100 is a pretty big depreciation in my book. I guess it is better than going to $0 if that is what you are comparing it to. However, if you compare it to a durable good I do not imagine it compares well. (Example durable good: well built wood furniture)

From a purely practical standpoint it means it is more tractable to solve large instances exactly. You can always wait for an answer but if you need more memory than you have you are out of luck. For instance, let's say before you had a problem of size 64. 2^64 bytes in gigabytes is 10^10 gigabytes. That is more ram than I have. However, a constant multiple (say 1,000,000,000) gives only 64 gigabytes. That takes an intractable problem to a tractable problem on a server with a large amount of memory. Yes, you still have to wait for the solution (maybe a long time) but at least you would have a hope of actually solving it.

It is actually called Delta Debugging and was pioneered by Andreas Zeller (https://www.st.cs.uni-saarland.de/dd/). His first paper on it ("Yesterday, my program worked. Today, it does not. Why?") used essentially the technique above to isolate the failure inducing change. Surprisingly, techniques like this work very well and doing better is sometimes very difficult especially for large programs. For instance, in theory, using "Concolic Execution" which combines symbolic execution, constraint solving, and concrete execution one can do better. However, such systems have their own limitations with respect to program complexity. See also the tremendous effectiveness of AFL (American Fuzzy Lop) http://lcamtuf.coredump.cx/afl/ . AFL uses a straight forward technique to mutate code. It doesn't use any fancy program analysis and has been able to find lots of bugs in real programs.

That is true and a good use case for fail2ban. Useless was probably a strong word, what I really meant was of limited utility in increasing the security of the SSH service.

I'm confused, how is SSH an example of defense in depth? It is an access method. You should absolutely harden your SSH configuration. Fail2Ban is useless on a properly configured SSH server (no root, no passwords, no kerberos, only keys). Managing the keys at scale, well that is a different story.

I agree with you that ASLR, NX, and CFI are the most important system level defenses to employ.

Honestly, applying to speak at industry conferences is a very frustrating experience. I have given lots of talks at my university, I have spoken at local meetups, and at one local free conference where the organizers knew me. Students and other attendees at my talks often tell me that my talks are good. However, trying to break into the "conference" circuit has been difficult as I have never gotten any direct feedback from anywhere I have submitted to and often wonder if they read my proposal at all. They sometimes even forget to inform you that your talk has not been accepted! It is a bit ridiculous.

In contrast, while I don't want to paint a rosy picture of academic conferences, you always get detailed feedback on your paper/talk that you submit. It may be biased, it may be frustrating, but at least you can tell that someone at least looked at your paper/talk and gave you some feedback. Industry conferences never do this.

As I said, it doesn't happen to everyone, it doesn't happen to most. But is does happen. Here is someone who it happened to in Cleveland: http://www.ideastream.org/news/be-well-why-do-babies-die-in-... . Note, not on drugs, not high, not drunk, doing the "right things." Had researched co-sleeping safely etc... I head her talk on the radio about it. The cause of death was determined to be her breathing on her child's face while they were both asleep. This just isn't the kind of risk I would be willing to take.

It is very dangerous to sleep with your baby. I know that many people in many cultures do sleep with their babies but you should not. There are several ways it can result in the baby dying:

1. You can suffocate the baby by breathing on their face while you are asleep. The baby will not cry and will not wake while this is occurring.

2. You can crush your baby by rolling on to them while you are asleep.

3. Your baby can suffocate from their nose and mouth being covered by a blanket or pillow or even the soft mattress if they are get rolled over.

These very sad infant deaths happen frequently even in the US where co-sleeping is not as common as elsewhere in the world. Here are some recent news articles: http://www.nola.com/health/index.ssf/2016/04/co-sleeping_dea... , http://woodtv.com/2015/06/05/mom-hopes-babys-co-sleeping-dea...

Co-sleeping advocates will tell you "as long as you do it safely you can sleep with your newborn." This is false. There is no safe way to sleep with a newborn. Newborns cannot turn their heads away if you breath on them. They cannot move if you get too close. While, many parents sleep with their babies and the babies do not die that does not mean that it is safe.

Update: The mayo-clinics prevention guide for SIDS: http://www.mayoclinic.org/diseases-conditions/sudden-infant-... . TLDR; Newborn babies should sleep by themselves on their back in a crib with no stuffed animals, pillows, or "crib pads."

Another note: http://thescientificparent.org/crib-notes-is-cosleeping-real...

I think the point is: his salary while not high by SF was high in nearly any other location in the US. In any other location, he and his family could live how they wanted. In any other location, have a $40,000 car instead of a $10,000 or less car is a perfectly reasonable. In any other location, eating out more than once a month would be perfectly reasonable. Having a expensive hobby racing cars is reasonable, just not in SF.

The point of the article wasn't that I am poor you should feel sad for me. The point was: look the Bay Area is so expensive that it is impossible to afford unless you are making an extremely high salary. A good salary isn't enough, a high salary isn't enough, it has to be extreme otherwise you will need to make life style sacrifices to live in the Bay Area.

If you like these notes you might also like my article on the subject: http://hackthology.com/object-oriented-inheritance-in-go.htm...

In general, using struct embedding to simulate sharing code and data is pretty limited. I rarely use this feature in my own code. However, having structs which automatically implement interfaces is awesome because you can "say what you need." You can also compose the interfaces together by embedding them. For example of that, checkout the Map type in my data-structures repository: https://godoc.org/github.com/timtadh/data-structures/types#M...

The transcript is very nice thank you for putting it together. You are right though, I hate the bold face. Luckily a quick trip to the developer tools took care of it. A better way to index large bodies of free form text is a topic index that points to the correct areas of the text for phrases and topics. These are sometimes known as concordances. That way the reader can decide what is important to them rather than letting you decide.

My favorite is you have all these pun titles and then:

> The Humble Programmer - E. W. Dijkstra

Which is actually the title of that essay by Dijkstra! http://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/EW... I guess this guy really doesn't think much of that essay?

One advantage is it is easier to have a single persistent connection that the client can interleave in multiple requests on. That way you don't have to pay the price of constant opening and closing TCP sockets. Just open one once and use it until it dies and then open a new one. There are many reasons a socket can die so you client does have to deal with that situation.