feelin_googley

Interesting that this statement did not trigger the usual mindless, meme-like HN syllogisms:

"Software is never finished."

"All software has bugs."

They try to discredit the truth that such small projects exist and might have something to offer.

I am not sure about others but I have never considered what is reported at http://www.netbsd.org/support/security/ as comprehensive.

I like to think the majority of NetBSD users either track -current, compile their own custom kernels or can learn to do as much. The project certainly makes it easy enough.

The flipside of the "many eyes" argument is that the Linux kernel has substantial financial backing and teams of paid developers behind it. NetBSD is relatively small and most contributors are not paid. I think they do an admirable job considering the number of active contributors and maintainers they have. (That is really an understatement but I do not want to appear too biased.) This is not the first time I have seen them immediately fix stuff when it is reported1.

1 http://bulk.fefe.de/scalability/

Unfortunately, even though the presenter mentioned loc several times, he did not try to calculate a bugs:loc ratio. Having a highly educational "history of BSD UNIX" and a large quantity of source code that allows hobbyists to run old hardware in their tree makes NetBSD not only unique and useful but also an easy target for any contemporary security researcher.

I only wish more would take on the challenge. Because sometimes things get fixed fast when this happens.

But comparing one project that has SVR4 compatibilty code to another project that has no such compat code, is that really an apt comparison? The old code is there for a reason. IMO, it does not imply many users will ever need it, that every user is actively using it, or that any user should have it in their kernel if they do not need it.

It is very easy for NetBSD users to compile kernels without code that will not be used, e.g., compatibility code. Easier than in any other OS I have tried. IMO, by making custom kernels so effortless to create (incl. best cross-compilation experience, IMO), it encourages users to compile smaller kernels without the drivers, compat code and other stuff they do not need.

But the presenter here was focused only on the assumption of a "default" kernel config. Not the potential to create small custom kernels with minimal attack surface.

Also: When the presenter was discussing filesystems, I wondered if he knew about rump. I think NetBSD was the first to have a such an innovative, safe way to mount filesystems using kernel drivers in userspace, without risk of crashing the kernel.

Depending on how one reads the RFC, it appears that the "sophomoric"1 compression scheme, which has been the source of numerous exploits/vulnerabilities in DNS software, also related to this thought about other classes besides Internet (IN). In the section on compression, RFC 1035 says that "pointers can be used where the format of the RR is not class specific."

1 https://news.ycombinator.com/item?id=503120

But in practice, are the RRs in todays DNS packets class nonspecific? Other than IN and CH (sometimes used for testing), are there other classes that todays DNS software can be expected to handle? Do todays queries almost always fall into the same type (IN)? How many query classes are there?2

2 https://www.iana.org/assignments/dns-parameters/dns-paramete...

Note even support for the "ANY" class has been the source of controversy.

Compression is what makes writing DNS software more difficult than it needs to be. However, according to one authors reading of the RFC, compression is only "required" in recursive DNS servers, not authoritative ones.3

3 https://cr.yp.to/djbdns/notes.html

"DNS, however, was not designed by experienced programmers."

This is another reason I prefer using local and remote authoritative servers versus open shared resolvers/recursive caches, such as the ones run by third parties such as Google, Cisco or various ISPs.

No (sophomoric) compression needed. Whatever the reason was for having a method of compression in DNS packets4, I personally do not need it.

4 The blog post: "QDCOUNT: An unsigned 16 bit integer specifying the number of entries in the question section. We'll be sending 1 question."

Take a guess what would happen if he sent more than one question. The RFC just says the qdcount is "usually" one.

RFC 1035 is written for a future that never happened. (For better or worse.)

If I were a user concerned with privacy and tracking I might wonder why the authors of the HTTP/2 standard did not make any efforts to address this type of fingerprinting.

Question for the author. It reminded me of something else, a number of years ago.

Did you ever consider this:

http://wiki.rumpkernel.org/Repo:-drv-netif-dpdk

It is ESP8266 and can be programmed without grahical IDE, using only GCC toolchain and Lua?

Why cant they just use a HOSTS file entry? What is the need for DNS?

If there is no need for DNS then why use the public, commercial CA system? That system relies on domainname registration credentials as "authentication".

Not sure I understand the design of this software. What is the purpose of the web server?

s/DNS server/third party open resolver/

IME, querying an authoritative server for the desired name triggers no such limitations.

One does not even need to use DNS to get the IP addresses for those authoritative servers, if the zone file is made available for free to the public as most are, under the ICANN rules.

I have thought about building a database of robots.txt many times. IMO, robots.txt has an important role besides thwarting "bots". It can thwart humans as well. It can be used to make entire websites "disappear" from the Internet Archive Wayback Machine.

Perhaps others are making mirrors of the IA.

However, I have thought it could be useful to monitor the robots.txt of important websites on a more frequent basis than IA, in order to (if possible) preemptively archive the IA's collections if robots.txt changes are ever detected that would effectively "erase" them from the IA.

Perhaps the greatest thing about robots.txt is that it is "plain text". This "rule" seems to be ubiquitously honoured. Did the author ever find any html, css, javascript or other surprises in any robots.txt file?

Then use UNIX filters to transform it into the format one prefers.

For example, NYT has a jsonp feed I have posted before.

   # replace "world" with whatever NYT section you prefer
   curl -o 1.jsonp https://static01.nyt.com/services/json/sectionfronts/world/index.jsonp   

    exec sed '/\"guid\" :/!d;s/\",//;s/.*\"//' 1.jsonp

As an experiment to enjoy text-only versions of popular news websites, convert the rss to html for a "text-only" version.

The author of radare2 wrote a filter called "rss2html" some years ago and it works well enough for this experiment, so there is no work involved here.[1] rss2html will transform a page of rss xml to html, txt, etc. Build it with libcurl and it can fetch pages too.

    curl -o 1.xml http://www.bbc.co.uk/news/rss.xml
    rss2html 1.xml > 1.htm 
    browser file://1.htm

    http://www.bbc.co.uk/news/rss.xml
    http://feeds.reuters.com/Reuters/worldNews
    http://www.latimes.com/world/rss2.0.xml
    http://www.wsj.com/xml/rss/3_7085.xml
    http://rssfeeds.usatoday.com/UsatodaycomWorld-TopStories
    http://feeds.washingtonpost.com/rss/world

Plan9 "fixes" many of the flaws/deficiencies of the original UNIX, such as that one.

Whats cool about this Plan9 project compared to the original Plan9 is one does not need to use the Plan9 compiler to compile it. One can use gcc, clang or icc.

   echo nmu360.se|exec nc -vvn 91.226.37.83 43

While Grub is highly well-known, popular, sometimes innovative and of course very flexible, it is also more complicated than I would like and so I am not a user. The primary reason is because of its strategy to locate parts of the bootloader on the media in a way that can vary from one installation to another. The alternative, "standard" approach being to locate all parts of the bootloader in some pre-defined area, usually near the start of the "disk".

The Grub approach obviously has its justifications and advantages. Maybe some see the Grub approach as an advantage because e.g. if one accidentally loses access to an area near the start of the media where most bootloaders historically reside, then one can still potentially recover.

However, if understanding the boot process is a goal, I see the Grub approach as a disadvantage. It makes locating and verifying the bootloader more complex when it is fragmented into parts that are potentially scattered around the media instead of located in a pre-defined area near the start of the disk. If one makes changes such as the formatting or partitioning of the disk, are all the fragments from a prior Grub install accounted for? I would prefer not to have to consider that question. Whereas I know how to completely remove/reinstall the bootloader if I always know where it will be located.

If the goal is to understand the bootloader one is using on a lower level than simply using it, e.g., verifying it is there on the media, I think there are more simple alternatives than Grub.

Alas, last time I checked the Linux kernel is still not compliant with the "Multiboot Specification". This is unfortunate for me because the non-Grub alternative I prefer supports it and it really works well for multibooting. I would love to use it to try booting a Linux kernel. I suspect there is probably a workaround but making the easy choice to use Grub probably obstructs people from experimenting with alternatives.

While experimenting years ago, if I remember correctly, I booted a NetBSD kernel using the FreeBSD bootloader (the part written in Forth). I seem to remember this was possible because of compliance with the Multiboot Specification.

"One bootloader, many kernels." (Without chainloading.)

It is not worth debating whether this impressive work is part of a plan by a corporation or a conspiracy among multiple corporations to steer/takeover "open" standards. (From past experience, it can be assumed that corporations will try.) The important point is the effect that this work is having, irrespective of intent.

The author uses Microsoft or Apple as an analogy. Question: If something goes wrong at the corporation -- e.g., as we see today MS is not what it used to be, at least not in the browser market -- then what happens to the corporate-driven "standard"? Will Chrome always be the #1 browser?

It should not matter.

If people believe it does matter, then we know people are not aiming at complying with a open, immortal standard, they are aiming at being compatible with a present-day successful corporation, hoping to ride on their coattails.

I submit to the reader that whatever "standards" the corporation creates may well die out if the success of the corporation ever begins to fade. Keep in mind also that these "features" that the corporation has introduced as "standards" may have been introduced primarily to benefit that corporations business (and business model). They are the largest/primary implementer. Maybe their software is the only software that ever used the feature. If the software "disappears" from use because the corporations business falters they stop developing it, then what good is the standard (e.g. if it is too difficult, not well-enough understood or for whatever reason that other software authors did not implement it)?

I leave it to the reader to draw some examples of where this may have happened in the past from the internet and web archives. It could be exhausting if one were to try to be comprehensive.

1. This is only personal opinion but when it takes a minimum of two hours to compile a "browser" (as suggested by one corporation that sponsors one of these monstrosities), then clearly that is a problem. It should cause people to question. Instead it effectively stops people from compiling -- they give up and just accept the binary distribution. By comparison, IME (please do not assume I am referring to Linux) it only takes a fraction of an hour even on modest hardware to compile a complex kernel. Kernel development may not be for everyone, but there is no reason I can imagine why every users "web browser" needs to be a single program of that size and complexity. Personal opinion. Others may disagree. I respect everyones opinion but am entitled to have one that differs.

Why not create a tmpfs directory over some existing directory? Why try to write to disk when the disk is known to be malfunctioning? Then send the static binaries needed to the tmpfs directory, e.g., using netcat. Perhaps I have misunderstood the problem.

Many years ago, I once tried a pre-packaged, well-regarded, "system rescue live CD". When I tried to use it on a computer with a malfunctioning disk, the rescue CD would fail to boot. Apparently it tried to access the computers HDD during boot. I switched to using another less-hyped OS to boot from USB stick into RAM and encountered no problems rescuing the malfunctiong HDD. Oh well.

Why try to use use the disk if it is not working?

Bonus: aligned columns[1]

(tested on 80x25 and only with text from blog, not actual /proc/net/tcp)

[1] Hack. Probably there is a better way to do alignment using only printf; alas I only know a subset of printf features.

   #! /bin/sh
   printf '%s\t\t\t%s\n' Local Remote
   sed '
     /: /!d;
     s/.*: //;
     s/ /-/;
     s/ .*//;
     s/[0-9A-F][0-9A-F]/0x& /g;
     s/ 0x//4;
     s/ 0x//7;
     s/://g;
     s/-//;
    ' /proc/net/tcp \
    |while read a b c d e g h i j k;do
     s=$(printf '%d.%d.%d.%d:%d %d.%d.%d.%d:%-22d\n' \
                 $d $c $b $a $e $j $i $h $g $k);
     l=${s%% *};r=${s#* };
     if test ${#s} -lt 45;then 
     printf '%s\t\t%s\n' $l $r;else
     printf '%s\t%s\n' $l $r;fi;
     done

   sed -n '/<\/br>/!{
   /\./s/.*/.&\
   =&:255.255.255.255:1\
   =*.&:255.255.255.255:1/;};/\./p'

Assumption is that user wants page to load faster but does not object to tracking.

What if user wants page to load faster and objects to tracking?

Source of Firefox tracking protection is list at disconnect.me?

credit: eco https://news.ycombinator.com/item?id=15964393

Basic gethostbyname()->HOSTS file blocking:

   #!/bin/sh
   exec curl https://disconnect.me/trackerprotection/blocked \
   |sed -n '/<\/br>/!{/\./s/^/255.255.255.255 /;};/\./p' >> /etc/hosts;

   User may want to add these too:
   https://disconnect.me/trackerprotection/unblocked

   #!/bin/sh
   curl https://disconnect.me/trackerprotection/blocked \
   |sed -n '/<\/br>/!{/\./s/.*/.&\
   =&:255.255.255.255:1/;};/\./p' >> _root.zone/root/data;
   if cd _root.zone/root/data;then exec tinydns-data;fi

   #!/bin/sh
   curl https://disconnect.me/trackerprotection/blocked \
   |sed -n '/<\/br>/!{
   /\./s/.*/echo 255.255.255.255 > _dnscache\/root\/servers\/& /;};
   /\./p' > block.sh;
   if sh -c ./block.sh;then exec rm block.sh;fi

It is not Javascript per se, but the so-called "modern" browser that creates problems for so many users. The problem is not even that this program exists. The problem is that users are coerced to use it and it is controlled by a third party that needs to sell something, either to advertisers or to users.

Again, there is nothing inherently threatening about Javascript. There are many languages to choose from and anyone can choose not to use it. It is not tied to the web browser anymore. Unfortunately it is from that association that it gains a stigma like the one the author suggests with his "war on Javascript" comment.

Today, there are a variety of standlone JS interpreters, of all sizes, and the language is routinely used outside of the browser. Peer-to-peer projects use it. Tiny interpreters for microcontrollers use it. Javascript can live on without the corporate web browser.

The problem is the irrational assumption one must use a corporate sponsored web browser to do anything interesting with a computer. This program automatically runs code from "anonymous", commercial third parties, which today happens to be JS, using the user's computer resources. The user today does not get to choose the language, but more importantly, given the level of coercion and absence of alternatives, she does not get to choose whether or not to run the third party code.

The general tone of the NPRM seems to be that an ISP can and will block or throttle any non-web or non-email traffic. That could include any peer-to-peer innovations that seek to restore the original functionality of the internet, such as those mentioned by the author.

On the contrary, the dissent by Commissioner Clyburn specifically mentions Skype as an example of internet usage. He believes the traditional notion of "permission-less innovation" is under threat from the Chairman's proposed approach.

https://apps.fcc.gov/edocs_public/attachmatch/FCC-17-60A1.pd...

The author highlights the importance of distinguishing "the web" from the internet. Perhaps nothing is more important. The internet has more value than the web. The web is severely limited in functionality. The internet, still underexploited in its potential, does not suffer from the same limitations.

The solutions I write using homegrown utilities are both more elegant and faster than any framework or library I have ever seen posted to HN or recommended elsewhere. Not to mention smaller and more agile. IMHO.

All frameworks and libraries that I have seen will all fail given the right input i.e. fuzzing.

I think parsing and transforming the content from webpages is just viewed as work that no one wants to do because, for whatever reason, webpages are still unpredictable.

Yes.

Interestingly Mozilla Foundation is not asking for donations.

Mozilla Corporation is selling traffic to ad-supported search engines, and profiting handsomely.

Instead they are asking for user cooperation in their experiments.

Why not create a different breed of browser that does not expose users to advertising. Profit motive?

Let users support it with donations.

This should be the mission.

Do I need javascript to accomplish any of these tasks? No. I am not interested in graphical web pages. I am interested in retrieving information, images, documents, sound files, video files, etc. I never need javascript to get these files, nor to read, edit or view them.

Simple example: I do not need a webpage full of javascript to view a YouTube video. I only need a video player (compiled without networking code).

Years ago, links added a javascript engine. Later it was removed. There is no point. A script can still consume inordinate amounts of RAM, even when there is no GUI. And for what? What is that javascript doing?

In the distant past, I recall browsers used to hand off media files to other programs, based on Content-Type (see article on MIME posted earlier today). Today, these external programs have been subsumed by the "modern" browser.

Perhaps modern browsers can be useful as offline image viewers, document viewers and media players. As I am in text-mode, the graphical browser is on another computer, connected via crossover cable or LAN. After inspecting their contents in text-mode, I transfer the documents and media files to a fileserver.

The decision to forgo using the so-called modern browser comes down to how important web design is to the user. If the user is interested in how fonts look, how a page of text is arranged, icons, styles, etc. and wants each and every website to look different, then the "modern" browser may be unavoidable. Probably javascript is needed to share in the web designers "vision".

The text-only browser OTOH makes all websites look more or less the same, regular, and if in text mode there is only one font, easy to read, IMO less eye strain on black background. It is perhaps better suited for the user that wants fast information retrieval, reliable, efficient file retrieval and cares little about graphical web design.

I happen to like text-only browsers for viewing HTML (e.g HTML tables), tcpclients like netcat for making TCP connections, and my own software for generating HTTP requests. Almost all websites work[FN1], with zero "loading time" as one may experience when using "modern" browsers to do these tasks. I can easily get the content I want (text, with option to download images, PDF, video, etc.) and skip the stuff I dont want. No autoloading of resources. I choose what I want.

Surprisingly, the web is actually getting more, not less text-friendly. Today I can often get text encapsulated in JSON, Markdown, etc. instead of wrapped in HTML, making parsing even easier.

There is heaps of Javascript written by others available on the web today but as a user I have little interest in running it. I would rather write my own.

FN1. "work" means I get the body the page that contains the content.

This is an article about the LinkedIn v hiQ case at AdWeek.

  curl --user-agent INSERT_ANYTHING_HERE http://www.adweek.com/digital/rami-essaid-distil-networks-guest-post-linkedin-hiq-labs/

How do they do it?

Pattern match against the User-Agent string.

Effective shibboleth.^W engineering.

Clarification: If a user, not a "bot", makes the "wrong" choice of user-agent string (e.g. in the browser settings), then they will be labeled a "bad bot", even if their behavior is no different than other users who are not labeled "bad bots". For example, they make one HTTP GET request just like any other user. There are databases of "acceptable" user-agent strings available to anyone. If still unsure about the point I am making, see this post from several days ago: https://www.sigbus.info/software-compatibility-and-our-own-u...

As for cjdns,

https://github.com/cjdelisle/cjdns/blob/master/doc/notes/cry...

From only a cursory review, I see cjdns as a network where one does not need a third party to: issue a name, issue a key, check a key against a name or a name against a key. Each user creates their own private key, their own public key and finally their own address from their own public key. Unless I am mistaken, users get public keys and addresses from other users, not from a third party.

Alternate:

http://ix.io/D2F (html)