HBGary planned to "blow the balls off Nmap" | Hacker News Reader

HBGary planned to "blow the balls off Nmap"

seclists.org

168 pointsdesigooner101 comments

Open articleView on HN

I was reading this and thinking 'I am pretty sure nmap already does this', and as the first reply confirms, it does:

http://seclists.org/nmap-dev/2011/q1/768

all these leaked emails just remind me why I left the netsec industry so long ago

Just reading this email and knowing these types of people I already know how this came about:

1. random surfing on the web and finds out about LFSR

2. thinks 'this would be awesome for a port scanner'

3. sends email with 'lets do something like nmap but better and faster using this thing i just read about on reddit'

4. 'i done my bit, GO TEAM!'

5. add 'software architect' to email signature

"There's this thing called an LFSR; it's math-y, but very cool" in a 2009 message from someone pretty well plugged-in to the vulnerability research scene tells you a lot about where we stand with crypto vuln research.

I think that's slightly unfair; Greg's expertise isn't in crypto, but in Windows NT kernel internals.

I'm making a comment about the field, not about Greg.

"Greg's expertise isn't."

ftfy

Do you work in the field? I don't know that you actually "fixed" that.

I don't know the credentials of this 'Greg' in any field, but he comes across as an inexperienced blowhard, that makes me doubt he could possibly have any expertise in any area. A modicum of expertise in some area and experience in developing software should ensure someone has more respect for something like nmap. It's a nontrivial piece of software and this email comes across as "I can build Reddit in a weekend".

Let's definitely debate Greg Hoglund --- someone you admittedly don't know anything about, but are apparently happy to comment about --- rather than talking about the thing this comment thread is ostensibly about.

I'm debating his email, which is exactly what this entire topic is about?

It is frustrating that, since Hoglund appeared in the Anonymous storyline, that's all we get to talk about --- because people, like (I don't know you, so don't take too much offense) you, come out from the woodwork and hang superficial barbs like so many ornaments on an ugly, boring Christmas tree.

You are commenting about this guy's personal, private email. You don't have to like or respect him. But your contribution to this thread --- "This guy sounds like a total asshole in his emails!" --- degrades the discussion. I'm calling you on it. Please, don't be such an asshole.

I think he has the arrogance of someone who is excellent in one area, but foolishly thinks it will qualify him to be excellent in other areas. Let's not forget we're building a huge psychoanalysis on a throwaway email here.

If you don't know enough to have something of an idea what you don't know, then you don't know anything worth knowing.

I wonder what he'll write when he discovers the Mersenne twister?

In reference to what he want's you can't compare a LFSR with a Mersenne twister. His goal is not primarily to produce a good pseudo random number (->Mersenne twister) but to enumerate enumerate all IPs of net range without repetition an. When you wand to be fast you want to avoid repetition.

He was probably looking at a data structure white papers/research. LFSR is good to use as a primary key for DB's./Linked lists(sequential numbers, no so great) I was working on an API along while ago using C.

I think this could be a mission for Redis and Node! Anyone want to hack together a little API/framework for LFSR Generation?

What sort of LRSRs are you looking for? And what language? I put together a LFSR generator as a test demo for a bit vector I homebrewed a while ago -

https://bitbucket.org/pnathan/logic-vector/src/598a6cddb080/...

I'm going to use Javascript and Redis. I'm working on a few little projects to learn both systems a little bit better.

Any particular reason it needs to be an LFSR?

A question I posted on SO some time back may be of interest, it solves a similar problem using an LCG (generating a shuffled range of numbers with no repeats):

http://stackoverflow.com/questions/464476/generating-shuffle...

It's really simple to program in ASM or using ISE/Verilog. going to a low tech approach it can be done on graph paper.

No reason why it's specific..Simple, 'easy to port'. I was doing a little bit of research about a theory I had but when I figured out enough I never fully completed that little project.+2 years prior.

> Anyone want to hack together a little API/framework for LFSR Generation?

I'm kinda interested. But I'm opposed to blowing the balls off anything.

Related HBGary, Anonymous post: http://news.ycombinator.com/item?id=2311067

(What we learn about Cyber War from the incident)

  > It should be FAST AS SHIT

Last I checked, fecal motility was pretty low. Maybe he should get that checked out...

[update]

http://seclists.org/nmap-dev/2011/q1/771 :

  > I've already had my brain cranking on what
  > elite networking code I could now write in
  > the kernel and I've always wanted to write
  > a badass portscanner too.

And now he can program those both over wifi with his laptop while in his tree fort with the "no parents allowed" sign hung off the side. Afterwards he can rescue his girlfriend from ninjas with is excellent karate skills...

Both of those emails seriously boggle my mind...

Everything about it is mindboggling; if you're going to put effort into building something in a place as inhospitable as the kernel, a portscanner? Who's going to pay for that?

The harder it is to write an application, the better the application must be. Putting stuff in the kernel means it will be harder to get right, so clearly the app will be better.

A security hole in the kernel is a small price to pay for a 0% speed increase.

System calls are expensive for a variety of reasons (including, but not limited to, the cost of extra data copies and the cost of switching security "rings"). For system-call heavy applications (in particular, networked services), moving things into the kernel provide huge speed benefits. [0] [1]

You are right that it's generally a stupid choice to make, but you're dead wrong in assuming that it's got no benefit at all.

[0] http://read.cs.ucla.edu/click/click

[1] http://www.research.ibm.com/afpa/ "All three components are implemented in the operating system kernel for maximum efficiency"

Dude! I so want this scanner as it will clearly be the best.

I am deeply sceptical of any technology where the builders think up the name first and then try to find a reason it should be called that. It just seems too much like stuff I did when I was 12.

or people who register a domain name first

and then do nothing with it

How do you feel about GNU?

GNU is not a technology, it's a movement. And to get all Marxist up on this joint, the building blocks for a new movement is created long before the leader is conscious of the need for one; the status-quo automatically provides an anti- checklist to motivate mobilization. One only has to scream "down with the government!" in a poor neighborhood to prove this; no further deliberation is needed, as the audience comes equipped with its own reasons to revolt.

Sounds more like the Hegelian dialectic than Marxist.

And you would be absolutely right! Marx was a Young Hegelian and applied it political thinking and his theories of society.

My impression is that this part of the email sounds like some business/sales guy that has a great idea where the technical parts are 'easy' and can be 'implemented by monkeys.'

"I have a great idea, let's make a product that can make us a lot of money. We'll work out the details later, that's the easy part."

It doesn't say they came up with the name first.

"I would like to call it "B.E.S.T. Scanner" so people kind of get stuck calling it "the best scanner". We can figure out what BEST means later."

This characterizes his entire attitude better than anything I've seen so far: "too much like stuff I did when I was 12". That, and some of my early Web customers in the 90's. I cringe just to read this stuff.

where the builders think up the name first and then try to find a reason it should be called that

How about federal legislation?

> I would like to call it "B.E.S.T. Scanner" so people kind of get stuck calling it "the best scanner". We can figure out what BEST means later.

Epic.

Certainly blew my balls off.

Are you sure it didn't just cause them to shrink to unrecognisable size through its ultimate suckitude?

Ball Erupting Scanner Technology

I almost bust a nut loling out loud at that.

Mental note: come up with a new site "B.E.S.T. Social Network"

You just did! Time to send it off to the developers. Your work here is done.

Man, I didn't realize being an Idea Guy was so easy! I might have to consider a career change…

Bowel Express Speed Technology. Remember, it has to be FAST AS SHIT.

Most programmers I know get the "I just discovered a silver bullet, damn I'm awesome" nonsense schooled out of them somewhere between entering college to a few years after, but apparently HBGary is an entire organization built on the exception.

Yeah, I remember the day when my default action on thinking up an "awesome game-changing idea" became immediately looking up who had already tried it and why it didn't work.

Older, sadder, wiser.

It gets better, eventually you think of an idea that some tried that did work and feel a little better.

Yeah so these are ideas that date back to the '90s, so it is genuinely weird to see Hoglund talking about them in 2009. But for whatever it's worth: if ever a tool needed its balls blown off, it's nmap. I have nothing but respect for Fyodor, but that space badly needs real competition.

There is, for what it's worth, very little reason why the best scanner need be written in C. Scanners aren't even I/O bound in their most common use case; they're timer-bound.

> There is, for what it's worth, very little reason why the best scanner need be written in C.

What about all the packet manipulation code? Bit twiddling tends to be difficult in other languages.

Bit twiddling is much easier in other languages than in C. And virtually every mainstream language has a pcap binding for the packet reading and injection (although a good straight-socket scanner could still give nmap a run for its money).

I give you https://code.google.com/p/bitstring/

What kind of things are wrong with nmap (I've never used it). Fundamentally flawed, poorly implemented, neglected project, documentation, etc?

Nmap isn't fundamentally flawed. It's ambitious and sprawling in ways that detract from its basic purpose. It wants to be a general-purpose network surveying tool, but is constrained to the UX of a port scanner.

Someone else should write the best possible port scanner and free nmap up to be the giant network profiling system it clearly wants to be.

"It wants to be a general-purpose network surveying tool, but is constrained to the UX of a port scanner."

What exactly does that mean? Nmap certainly isn't constrained to the UX of a port scanner. In fact, their expansion wasn't even in the form of bundled build, you can get all the tools separated, like: nmap, nping, ncat, ncrack...

So actually, it really isn't detracting from its basic purpose, you can still run simple network scans. But, if you got the knowledge, skills and the other tools installed, you can then also do much than advanced network scans, like vulnerability discovery and exploitation, passwords bruteforcing, etc...

Your comment isn't making much sense to me, please help me understand.

It has N different kinds of "stealth" scans that all produce the same kernel of valuable information, it has an elaborate (and, against most corporate networks from external vantage points, not particularly accurate or helpful) OS detection scheme, it runs scripts, it produces stdout and file output in a variety of complicated formats (it really just wants to be a service hooked up to a database), it runs scripts, IT RUNS SCRIPTS, come on. This isn't a serious argument. You just like nmap. That's fine; I think nmap is impressive too.

Oh, and it takes _for_ _ever_ to run. I'll pastie the 50 line EventMachine scripts I race it with when people on my team get stuck waiting for 12-hour nmap runs.

I'd like to see those scripts when they are posted, thanks.

Sure. You know how simple this task is when you have an event loop, right? Here, I'll write a port scanner right now:

  class HostProbe 
    module Probe
      attr_accessor :bp
      def connection_completed
        @win = true
      end

      def unbind
        self.bp.closed(self, @win)
      end
    end

    def closed(obj, won) 
      @inflight_now -= 1
      port = @inflight_q.delete(obj)
      if won                 
        puts "%TARGET-LISTENING: #{ @target }:#{ port }"
      end
      fill()
    end

    def sweep
      Generator.new do |g|
        (1..65535).each {|port| g.yield port}
      end
    end

    def fill
      @inflight_now.upto(@inflight_max) {
        @inflight_now += 1
        if(port = @sweep.next) 
          EventMachine::connect(@target, port, Probe) do |c|
            c.bp = self
            @inflight_q[c] = port
          end
        end 
      }
    end

    def initialize(host, opts={}) 
      @target = host
      @sweep = sweep
      @inflight_q = {}
      @inflight_now = 0
      @inflight_max = opts[:inflight_max] || 10
      fill()
    end
  end

  EventMachine::run { 
    HostProbe.new(ARGV[0])
  }

I think that may be it? I'm not even going to bother seeing if it evaluates. I write that stupid script once a month or so. There is probably a set of nmap options that crushes it for speed, but with the default options and a firewalled target network (ie: most professional nmap targets), I lap nmap with it.

(Ok, you also have to disconnect when you hit a listening socket. Oops.)

> Someone else should write the best possible port scanner

Best Possible Port Scanner... BEST... sorry, it doesn't fit. We could give you say the Best Egregious Scanning Tool, would that work?

If you feel the need to justify nmap being writting in C, which you shouldn't, just remember: The people who wrote nmap know C, and managed to write fucking nmap in it.

If you want a better one written in something else, then get cracking...

I don't even know what this is supposed to mean. Are you an avid C programmer? Or do you just really like nmap? I am asking seriously.

Not sure why I'm bothering to respond since you're being rather trolly.. but neither. I rarely program in C, and I rarely have a use for nmap.

Rather I just intensely dislike when people pick language choice to focus on. You pick your language based on what your team knows, and if you can ship with it. Picking it for any other reason just makes you come off in a PHB type of way. If you have real criticisms and suggestions to make, then do so and make the world a better place in the process.

I'm not the one downvoting you; I just upvoted you. But you should re-read my comments. I don't have a problem with C.

The issue is that "very little reason why the best scanner need be written in C" is a rather meaningless statement that verges on plain old flamebate.

I think he's wrong here too. I can remember using nmap back in the really late 90s. Which wipes out a lot of languages that aren't C.

C was and is a good choice. I'd be a lot more surprised if it was written in Java or Perl.

C is a bad choice for a new port scanner started in 2011. I've already written comments here justifying that statement. If you really think I'm wrong, it might be fun to hash out why. (I've got a fairly decent background in nmap's problem domain, for what it's worth.)

I don't have a problem with C, although if I had to do it I would use C++ for object-oriented support and more or less guaranteed portability. Possibly also to compile in a lua interpreter. That said, I'm not a domain expert and was responding to the question of whether nmap should have been coded in C rather than what the best language would be for something started NOW.

I don't know what the best alternatives would be and while I doubt I can contribute much to the discussion, I've read your other posts on security with interest and would be interested to hear which language you'd pick and why.

Personally, I think Lisp is a fantastic language to write a scanner in; you can express async state machines in it naturally, and most Lisps have very good foreign interfaces, which would allow you to just call into libpcap. Lisp is also good for DSLs --- look how Seibel did the little expression language for binary formats in his Lisp book; that's exactly the problem statement that drove me to make my own dumb language in '97 to express packet formats.

Virtually any language will work for this (which is why you should avoid C). About the only thing I'd steer you clear of is a JVM language. The JVM is ordinarily a win, but here, where you want the path-of-least-resistance to getting pcap into your program, it's a bit of a pain. So scratch Scala from the list. Other than that, have fun.

But use Lisp. It's a great first Lisp project.

Thanks!

I am saying something very simple that seems to have caused your head to explode. It is: you could write n+1map in Python using Twisted, or Ruby using EventMachine, or Erlang, and probably end up faster (not because of the language, but because you'd design for speed and the language doesn't matter) than nmap is in C.

This isn't a controversial statement. Nobody thinks nmap is the fastest possible scanner. To argue that n+1map should be written in C, you have to make a case for what aspect of a port scanner benefits from C. Which is why I pointed out: nmap doesn't even look I/O bound; it looks timer-bound. And in most cases, neither I/O-bound nor timer-bound programs benefit greatly from locality, cheaper copies, or (within reason) optimized memory allocation and layout.

I believe the problem is that you seem to be saying c should only be used when performance matters. They think the language choice is irrelevant and shouldn't have even been mentioned. Or so I believe.

For my part, I am an avid C programmer; it's my first language, and virtually the only one I used through the '90s. And I will say: you should avoid writing in C unless you need to. The cases I can come up with off the top of my head where it's necessary:

* When performance matters and can be gained through writing C code --- for instance, if you're compute-bound, or if you need fast access to data structures.

* When building incremental improvements to large C codebases --- ie, writing a loadable kernel module.

* When you're deploying in small-footprint environments.

What are the other cases where C makes sense?

I brought up the C thing not because I want to take potshots at what is probably my favorite language, but because a majority of the developers on HN don't write C, and you'd hate for them not to take a crack at competing with nmap because they had a faulty belief that they'd need to use C.

He's just saying they used what they knew best. For example, there's no reason Git has to be written in C[1]. Mercurial, which is Python, is doing just fine. C is what Linus and the kernel hackers knew best.

[1] Yeah, I know it was mostly hacked together using Bash and Perl, but that was a long time ago. Now it's mostly C. I'm suspecting they rewrote the performance-critical stuff in C, and left everything else alone. Not sure, though; I don't follow Git development.

I'm not criticizing Fyodor for writing nmap in C!

What's terrifying about this is that this company somehow scored a contract with the FBI.

How many other HBGary Federals are there out there?

Why is wanting to compete with nmap a terrifying attribute of an FBI contractor? Technology within the FBI pretty much begins and ends with EnCase.

While the FBI isn't known for being on the technical bleeding edge, and used to bleed a lot for technical screwups, the RCFL's - Regional Computer Forensics Labs - are very much out there on the leading edge of computer forensics technology.

---

There are some amazing technical innovations in EnCase and the products built on it. Unfortunately, the company has management issues that result in some very poor decisions in many areas, including deployment of software development resources.

I've been using EnCase for about a decade (including a large portion of that working with the FBI), and I honestly can't think of any amazing technical innovations.

The support for reading .pst files? The fact that it can finally handle filesystems besides FAT32 and NTFS? Certainly not it's stability.

EnCase is very good at what it was designed for, which is basically an idiot-proof way for an investigator to comb through a hard drive looking for search terms.

I would argue that technically EnCase isn't even as good as FTK, which is something that pretty much every third-party evaluation of the two products has shown.

I won't disagree that there are some smart folks working at the FBI, and that's not even counting the ones who spend all their time developing "zoom and enhance" software for photo analysis.

One who immediately springs to mind is Dennis Montgomery. Here is a link that is NSFW: http://www.playboy.com/articles/the-man-who-conned-the-penta...

In all of these cases one of the main problems seems to be that non-technical people have too much decision making power in technical areas. (I'm not going to address this huge problem here.)

I've lost count of the number of times I've been in a meeting listening to someone talk about something like the "Linear Feedback Shift Register" mentioned in the article.

Unfortunately, technology seems like magic (or snake oil) to people who don't know anything about it...

Greg and Aaron, two of the guys involved with HBGary, are incredibly smart and capable. They were very well respected in the industry for their intelligence, products, and services.

---

How well would you stand up in the court of public opinion if several years of the internal email messages for your startup were leaked?

I'm very concerned about the rise of the cybersecurity/infosec complex ala the military-industrial complex, but one of the reasons for my concern is that they have very bright, very motivated people working for them who know how to play the D.C. political games.

Well, that email is a piece of relatively ignorant crap that I could have written when I was 17. The last time I read something like that was a Phrack interview....

The longer you're involved the more you'll find that people in positions of authority and prominence rarely deserve them.

Please, please, please STOP referring to networks as class a, b, c, etc. The 1980s called, they want their networking jargon back.

I doubt HBGary reads HN. And if he does, it's only for reconnaissance missions.

Apparently there are people I insulted that still use that terminology. It really makes you sound, umm, managerial and pointy haired when you use those.

What are you talking about? It is a critical part of Layer 3 of the OSI Model!!

What is the correct terminology?

http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing

Could you enlighten me? What is a better way to refer to /24 /16 /8 networks?

/16 obviously. Say it aloud a few times and just feel how it flows off the tongue. "slash sixteen, slash sixteen." So much smoother than getting tongue tied saying "class B" out loud...

OK, I thought there was a more technical reason for not using Class A/B/C.

What would you call 192/8? I would call it 192/8, but I don't even know if it's a valid network in the classful world.

Good point.

Everyone says stupid things on emails. Specially on PRIVATE emails.

It is funny, sounds stupid what he is saying, but how many stupid things we say on email, IM etc on a daily basis? Don't judge just because of it...

"We can find source code for such things on the net to help us write it. It's just a few lines of code."

Is this a case of greed and a road to fast fame?!

This coming out from someone as respected (at least in the past) and renowned researcher like Hoglund doesn't give a ringing endorsement to the industry. Was he that desperate for fame and fortune?!

I think a similar effect with LFSR you can have with a De Bruijn sequence http://en.wikipedia.org/wiki/De_Bruijn_sequence.

So this is really what all the commercially oriented security researches are like?

It seems that the people I truly consider hackers rather commit a patch.

It's lame that this is on HN. Basically it's just finger pointing and laughing at someone for being stupid on the internet (in a private mail, it seems).

Not being stupid on the Internet could be considered relevant for Internet entrepreneurs

Reply on news.ycombinator.com