Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Honestly, as someone who has been playing around with/working with AI in some form (previously bayes nets etc...) since the 90s and trying since 2012 to build our computer vision company - every keynote from AFGAM makes me want to give up a little more each time.

It's like even with a world class AI team the majors just blow past everyone with their on demand scale, access to data, hardware, people and distribution.

In the past three weeks the products that were shown at conferences (GTC, Build, I/O) would have been 100 different independent companies, with very highly trained, specialized PhD level researchers and developers only 5 years ago. Now this stuff is just baked in to the top platforms.

This has been happening for a while, and I've been saying it for a while too. No clue where to go from here honestly.

edit: The point I didn't make here but is subtext is that, IMO ML/AI is the last frontier for technology (an IMO humanity) so if a few dominate it, its kind of game over for the existence of smaller players generally.

View on HN

I've been wondering for a long time whether machine learning will break the long-time tech industry lifecycle we've all come to regard as some kind of immutable law of nature.

Traditionally, 1) we start with big, established companies in tech. 2) A few people, dissatisfied with the trammels that big companies place on their employees, leave, found a new company, and then, 3) through superior grit, gumption, execution, flexibility, and speed, steal market share from the big companies. As the new company grows, 4) it accumulates organizational scar tissue and is slowly infiltrated by incompetent careerists, and finally, 5) becomes the very company that the founders set out to create.

We've been going through this cycle since the late 1950s, when the "traitorous eight" left Shockley Semiconductor Laboratory to found "Fairchild Semiconductor".

Machine learning might stop this cycle by breaking step #3. Big companies have access to huge data sets that no group of human developers, no matter how talented, can match. Even slow, hidebound, and barely-competent use of these data sets can squash upstart smaller companies.

It'll be like a planet, full of water and life, with a core that cools to the point where it can no longer sustain plate tectonics. Without constant replenishment, the crust seizes up, the oceans boil off, and the life dies. The tech industry will become Mars.

Machine learning might stop this cycle by breaking step #3.

Yea that's exactly what I think is happening, and in fact that's one of the core promises of being first into AI - you use it to move faster than anyone else can because it gives you better intelligence and prediction.

Even more so with AGI, which Deepmind (Alphabet company) has explictly stated they are trying to create.

On the other hand, #3 might in this case be exactly that: Small companies creating accurate & precise modeling techniques using only a limited amount of data, while giants continue to rely on the sheer and blunt volume of it.

>Small companies creating accurate & precise modeling techniques using only a limited amount of data, while giants continue to rely on the sheer and blunt volume of it.

until of course sheer and blunt volume allows to build more accurate and precise models. To me this is exactly at the core of the current AI wave - the models emerging inside deep representational layers are more complex, accurate and precise way beyond whatever human analytical thinking could have built or even just come up with. My friend just couple days ago was talking about one specific multi-year effort few years ago to constructively build a practical model of sentiment analysis which i in turn contrasted with that recent result from OpenAI of "sentiment neuron" emerging from [unsupervised] learning of next character in Amazon reviews.

You may be right for the time being, but I don't think we're permanently entering a brave new world where machine learning startups are doomed to fail to the big tech companies. It's well known that existing machine learning algorithms are extremely inefficient learners, which is why you need enormous datasets and computing resources to be competitive currently. Right now, for instance, a computer would need to see thousands of different images of a new animal to recognize it consistently, whereas a human only needs to see a single hand-drawn sketch. If someone can develop machine learning techniques that make computers generalize as flexibly as humans, that would be a huge game changer and would completely upend the advantages that the big tech companies have.

If someone can develop machine learning techniques that make computers generalize as flexibly as humans

Funny you should mention that...

Deepmind (acquired by Google in 2014) goal: "DeepMind Technologies' goal is to "solve intelligence" Demis has gone one to say elsewhere their goal is to create AGI

Also, the data you need to get there is owned by a few companies. Also those companies hire effectively every great ML PhD on the planet.

It's well known that existing machine learning algorithms are extremely inefficient learners

Right, which is why the heads of research at AFGAM are the pre-eminent minds in the field (Lecun, Hinton etc...) and are actively working on this along with unsupervised learning. These companies are either funding university labs that are developing this stuff or doing it in house.

>Right now, for instance, a computer would need to see thousands of different images of a new animal to recognize it consistently, whereas a human only needs to see a single hand-drawn sketch.

valid question here would be - would a fresh human mind perform that well too? Or may be such performance is conditioned upon having at least several years of experience of living on planet Earth and thus having the models of animals and everything else around been built using billions of images that human eyes would have generated during those years?

> It's well known that existing machine learning algorithms are extremely inefficient learners, which is why you need enormous datasets and computing resources to be competitive currently.

when put into perspective, i.e. compared with the cardinality of the above mentioned set of images human eyes generate and computational performance of the brain - 40K TFlops - computers don't look that inefficient learners to me.

> valid question here would be - would a fresh human mind perform that well too?

I think it's questions like this that will point the way forward. Humans definitely don't come working at 100% as soon as we're born. Lots of interaction with the environment is necessary to become a functioning person. But at the same time, we still seem to have a real ability to generalize hard-wired in. A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice.

It seems to me that lots of data will be necessary to get things off the ground --- but I think the sort of "big data" needed for this will be qualitatively different than the big datasets used to train modern state of the art models. My hunch is that it will involve an agent's interactions with its environment (so it will look something like reinforcement learning).

For a playful yet pretty interesting take on "psychogenesis" I recommend reading Diaspora by Greg Egan.

Also, we are born with very good models (for faces, cognition of human languages + speaking and hearing them, seeing things on Earth, movement coordination, other cognitive fundamentals, like counting, categorization, and so on) plus all of that really well packaged as a starter kit, a do-it-yourself general intelligence, with amazing supervisory framework (parents, peers, society, reflective optimization about one's actions) and reinforcement (emotions, memory coding is waaay too drastically modulated by emotions).

And we're still not sure what else is in there, how all this communicates. What's the operating system of consciousness? (We "know" that consciousness is just a "program", but is it the main thread, is it a scheduler, what's the right mental model for understanding the interaction of brain faculties, components of cognition and consciousness itself?)

> A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice.

It does seems pretty well established that some humans overfit on the particular aspects of verbal communication that we generally call "accent" (not to mention actual lects), even when given access to a much larger and more diverse set of individuals and their utterances to generalise from.

Generally agree with OP that ML is one of the last viable frontiers for the current generation of computer tech. We are basically taking the data generated by machines invented decades ago, and using statistics to optimize a few % of productivity points more out of them every year. This optimization applies to humans to - one day you'll be required to have an OBD dongle monitoring your driving data to get car insurance, for instance. Many facets of your life will be monitored and fed into a company's individualized production function or pricing equation.

When the economic cake is growing slowly, every last drop of productivity enhancement counts, and ML is doing a good job at shaving a few basis points off basic costs, such as energy (see DeepMind's work on Google's datacenters), transportation (autonomous cars), and human capital (making some simple jobs redundant). That's why everyone is piling into IoT/ML/AI right now.

What ML is NOT doing is creating a new paradigm/market/computing platform. It's optimizing old markets and technologies. It's not baking new cakes.

We'll see a new platform soon. The cycle of human technological progress has been chugging along steadily for a long time and has sped up exponentially in the past 200 years. I'm sure some new tech will come along soon that will allow us to bake more cakes instead of spreading icing ever-thinner on the ones we already have.

Generally agree that ML is often shaving off a few % after all the low hanging fruits that can be achieved without it have been exploited, but...

> What ML is NOT doing is creating a new paradigm/market/computing platform. It's optimizing old markets and technologies. It's not baking new cakes.

This is simply not true: there are many products/markets (such as automatic translation, speech recognition) that are simply not possible without this progress in ML.

What's the paradigm?

In translate it replaces a naive translator's job. In speech recognition it gives you a new-ish interface to systems.

In prediction and personalization it gives you what you likely want. (Local weather, local traffic info, local news, in the language it knows you understand, in the format you prefer, at the time it think you most likely want it, etc..).

It puts some "smart" into things. But translate is just a smart dictionary, it's not a real translator, for that we need stronger AI (something like the Jeopardy playing Watson + Google Knowledge Graph / FreeBase + language translation + it should ask questions if it doesn't understand something).

ML is amazing, but it's just a slow march toward more and more adaptive smarts (general intelligence) in a box (hence artificial). And there's probably a tipping point for that. When it can start to learn, or program itself, blablabla... ( https://intelligence.org/2013/04/29/intelligence-explosion-m... )

If we look at things from that point of view, then nothing is new.

What are computers good for? They are just replacing a calculator, which was just replacing some manual calculation machines, which were just replacing mental calculus.

A "naive translator's job" cannot translate arbitrary sentences for free and instantaneously. We are not far from real-time (and quality) translation that will make it possible to talk to somebody in a foreign language and have everything translated on the go (there's already a feature that replaces text in a foreign language in the image you're viewing).

I don't know :) I think I'm quite satisfied with the level of disruption ML is bringing to the world. And I feel people are constantly pushing away against this ("This is not real AI!") every time we start to understand how these things work.

Reinforcement Learning is the case of a program teaching itself (without training data or instructions) but I still, some will argue it is not AI because "it's just maths and engineering hacks", I guess.

I understand how ML/AI works, and there are great things like collaborative filtering, self-driving whatevers, "expert systems" (IBM Watson), magical image processing (which just shows how much detail our own brains just imagines and fills in for us in non-ideal conditions).

I'm not saying it's not real AI, I'm saying I can't wait for the time when these separate components/models can be "synergized", when one big system can be trained for multiple tasks, and when it can train itself tasks.

>one day you'll be required to have an OBD dongle monitoring your driving data to get car insurance, for instance

FYI there is a company in Australia that already does this, well, you get discounts if your device shows that you're a safe driver or something...

I think there's a real danger to underestimating small % improvements over time, depending on how you're measuring.

Especially if you're looking at error rates; 99% accuracy is very different to 95% accuracy is very different to 90% etc.

I've been wondering the same thing. It is why I don't like the fact that we need increasingly specialized types of computers to do ML. This is also why I am very interested in research that is not based on deep neural networks, which seem to be especially hungry in terms of training data and hardware power.

I think this depends on what you see as the downfall of these previous companies. If you agree with Christensen's theory of disruptive innovation, the issue was that large companies would undervalue small markets that could grow to become large markets.

When seen in this light, it would be surprising if ML were a disruptive technology, since it falls more into the "sustaining" category of innovation.

Most of the large tech companies were built on technologies that no-one expected to become world-defining companies at the time.

With all this focus on ML/AI, it would be very surprising if a small ML/AI shop became huge, rather then getting bought or replicated.

Almost by definition, the big companies of the next era will come from the spaces that large companies are ignoring.

the big companies of the next era will come from the spaces that large companies are ignoring.

My entire point is that they aren't ignoring any spaces, they have all future spaces accounted for by having their tentacles in literally everything possible that could be future tech.

Bldinsides come from different industries, so they need to cover their bases from all directions, so that's what they all have done with their projects, products, investments in...AI, Energy, Biology, Transportation, Infrastructure, Waste Disposal, Agriculture the list goes on.

The time of the non-ubiquitous firm is over.

AFGAM aren't software companies, they are innovation conglomerates. I mean why do you think google changed to Alphabet?

Right, but this has less to do with the fundamentals of ML as a technology, rather than the less sexy explanation of better management practices.

They are inextricably linked. If you organize your company around ML, as Google, Facebook, Amazon, Microsoft and Apple have stated they are doing, then yes it does have to do with the fundamentals of ML.

Advances and implementations of ML is driving the creation and iteration of products for these companies, as well as helping them run their companies and evaluate where to go next - follow the data as it were.

So it is a critical point that every one of their CEOs has outlined in clear detail over the past few years.

Look at the flipside - for the people who want to focus on applications of ML instead of futzing with platforms and implementations this sort of environment is a godsend. Compare what it took to keep a server/website/database up in the late 90's compared to something turnkey like Heroku today. It frees up a lot of developer time and energy for building the next layer of innovative things!

The "flipside" is that some of the best minds in AI basically have only 2 choices coming out of school:

- AFGAM

- Military

The period of being able to build your own competitive AI startup was...what, like 3 years between 2010 and 2013?

As of 2016 investors were still throwing money at 'ML for x' startups.

Yeah, why not, Google will buy them if they are okay.

If AI is the high end of programming, then it's the equivalent of saying that a recent manufacturing grad can't build their own car assembly line and has to go to work for Toyota, Honda, et al.

Barriers to entry at the "top" of industries are generally higher than lower down.

I'm sympathetic to the sentiment.

Consumers might benefit from the conglomeration more than from a lot of independent groups each reinventing the wheel or spending resources on duplicative infrastructure.

There are trade offs of course. In particular, I really don't want one single ML platform to have all my data and know me that well.

The metaphor would be that no combination of people will ever be able to compete with Toyota, Honda etc... and if they tried would be acquired or beaten to death by Honda because they are leveraging their massive scale.

But what about Tesla? It managed somehow to compete with Toyota, Honda, etc, although it had to create a new market (market of electric cars) to do it.

Korean cars manufacturers created now mature brands from scratch. All it took was billions of dollars and a couple decades of time. Can a startup do it? No. Is the market locked up? No as well.

Tesla isn't competing with Toyota... They are competing with Porsche and Rolls, etc. Further, they are a mess financially and have not proven to have staying power.

Despite the hype and genuinely great technology and growth it's still nowhere near competitive with the big automotive firms.

So while Tesla has definitely driven innovation here, there are still MAJOR hurdles to them being globally competitive.

Further it took Billions and someone like Elon Musk to be able to get even this far, so in that context if even Musk and tons of government dollars can't get there then it's safe to say nobody can.

Frankly, I think your belief is due to lack of knowledge. As a current PhD student who knows several ML researchers at Google, there is fair bit of marketing involved in creating perception of Google supremacy. while no doubt Google is a leader in AI, their advantage is not as great as it would appear by reading the comments here.

At the same time other companies, Apple /Salesforce / Amazon / Oracle and several other smaller ones are well aware of threat posed by Google and are always on the lookout for ML/CV startup that can fill the gap in products relative to Googles offering. E.g. Amazon acquired Orbeus around same time Google launched Cloud Vision.

We fund a ML research arm at the University of Alberta, give me a break with this.

I'm not talking about just Google. Its AFGAM: Apple, Facebook, Google, Amazon, Microsoft

You can add Baidu in there too probably. That's it though.

always on the lookout for ML/CV startup that can fill the gap in products relative to Googles offering

Right, acquire. Not compete. That's the point. You can't actually compete with these companies, like they did in the past with HP, Dell, Nokia etc... the best you can hope for is to be acquired.

Funding is not knowing.

There are always new markets which get ignored. Apple, Facebook, Google, Amazon, Microsoft did not compete with HP, Dell, Nokia they essentially entered a new market. Most ML startups don't offer a new product but rather just enable features and acquisition is often the intended end goal.

You're missing the point.

We expect that DNNs will a fundamental thread to computing going forward. AFGAM has effectively locked up all of the talent and data in the broadest markets already. Not only that they are building and deploying the frameworks that new products will be built on in those other markets. So they don't need to create the product, they just need YOU to use Tensorflow or CUDA or whatever they come up with that runs on GoogleCloudGPU or Azure Compute etc...

What you're missing is that it's not about making a few million as a startup. It's that they are so far ahead in what is the most fundamental game changing, final human technology that I don't see a future where they are unseated.

> final human technology

Can you share your reasoning behind this opinion? Humans have invented a lot of tools and technological platforms over the years, what makes you think ML is the final frontier?

A lot of people have written about this over the years. I'll just let Yoshua Bengio go into it:

[1]https://www.youtube.com/watch?v=ZHYXp3gJCaI

While deep learning and related technologies are exciting and extremely powerful, they have not yet proven that they will achieve the "final human technology" status that you are proposing. We may be centuries away from that, and at this point trying to speculate is foolish. However, even if they don't live up to those lofty outlandish prospects, they will still prove very valuable and useful.

What is "proven" to you? I mean, the tech displayed today is basically unmatched by most humans. Curating the best shots in Google Photos, all the integration with Google Home, Lens that can identify almost any object in meat space... the list goes on and on. It is not going to regress somehow... I mean, it is probably exponentially increasing in power by the day.

There is no AI competition against a company that has Tensor Processing Units among the absolutely incomprehensible data store that Google has amassed.... Especially if you are not some world class mathematician or computer scientist. The best you can hope for is to be acquired if you are in the ML/AI space as a stand-alone entity. In most cases they can probably safely ignore the bulk of AI startups.

That's still only computer vision and speech recognition. Google still does piss-poor at answering your questions using snippets.

It seems like picking the best photos, at least, is a pretty forgiving task. It's subjective, and I expect people will accept any set of pretty good shots.

Also, those TPU's are available for rent.

Definitely forgiving, but I think you are missing the point. The fact is, Google owns the whole process from top to bottom.

Nice that TPU's are available for rent, but to what end? If I run something truly innovative on their hardware (TPU), with their software platform(Tensor)...is that actually innovation, will they have the means to just copy it? How do I compete with that?

Would you say the same thing of Go or Chrome? These are tools anyone can use. Building something higher in the stack isn't cheating, it's just reuse. (Snapchat is built on Google Cloud, for example.)

As always you want to be careful about dependencies, but that doesn't mean rewriting everything.

Yes, I would, as there is no way for me to truly compete with Google if I am using their tools. The premise is so flawed that I have trouble even formulating possible counter arguments you may suggest.

I use chrome and little snitch is always telling on it. It calls home so much that I have just given up in order to have a decent experience and gave it full connection rights. I don't know how much of the browser Chrome owns, but I guess it is not insignificant. That is a lot of data that I will never have access to... let alone for my family nor on at Google scale.

Netflix uses AWS and competes with Amazon Prime... I get it. I do think that no one knows the whole story.

Getting back to OP... it has to be damn tiring/frustrating to even contemplate competing in AI/ML space with these big mega corps. I simply would not, that ship has sailed.

That's the age old question - what if Facebook copies what you did, in a week? Like they did with Twitter or Snapchat?

By the time they copy you, you're already big. I don't really see the problem as long as the tech companies don't hoard the information and spread the knowledge - which they are doing with initiatives like OpenAI and many more.

The big question is, what happens when they invent AI that builds its own programs and exponentially accelerates? Who needs startups at all then?

Bull. Instagram is eating Snapchat for lunch right now, for starters. Twitter is going down in flames soon enough as well. There is a massive consolidation taking place in social, and I think with tech in general. We are at a point where automation, AI/ML and resources are the next battle grounds. Sure there will be wins here and there, but like the app stores in 2014... the gold rush looks like it's over.

So in your opinion having accurate visual recognition/identification is the final human technology? We can stop now?

No, you didn't read what I wrote: It [Google AI efforts] are not going to regress somehow... I mean, it is probably exponentially increasing in power by the day.

They not only have custom silicon (TPU), they have the software too. Do you have extraordinary insight|foresight|genius or have you discovered/developed a major breakthrough in AI/ML? If not, good luck competing with these companies. It is a resource problem at this stage, as consolidation happened and there is no going back.

> that I don't see a future where they are unseated.

The way I see it, consumers (at least right now) are free not to buy into the hype and consume whatever it is that these companies produce. So if prices are too high or quality too low, a new company can enter in.

The real danger I see is the possibility that essentially all consumers will be forced to pay for this stuff through some regulatory capture, in a similar fashion to the state of the healthcare industry in the US.

I don't know - maybe I'm just irrationally optimistic but I don't believe anyone has "locked up all of the data".

You can download the whole Wikipedia data dump at under 100GB uncompressed (text only - with media is around a TB). The entire common crawl with 3bn pages is only around 250 TB. While the Wikipedia dataset is too large to fit in ram for most people and the common crawl is too big to fit on a single disk, you can process these in your own local cluster quite easily and relatively cheaply.

Honestly I think the real breakthroughs to be made will be algorithmic and I don't believe those are out of reach for "civilians" outside of the tech giants.

While I think OP is a bit too pessimistic, in a way s/he is right. Facebook can design a simple experiment, implement the changes in their whole platform, roll out, gather data, and iterate on it.

You are stuck with 25TBs of wikipedia for the rest of forever. And "academic datasets".

Because the important data is probably not inferable from web links or otherwise semi-passive Internet structures.

That said, I think the missing component is still a good mind theory.

Humans don't run split tests on alternate realities but they can learn from wikipedia alone - I think we just don't know quite how yet.

We need a lot of data currently because algorithms don't generalize like people do.

I'm not convinced you need to be a tech giant to be able to make that breakthrough - I think it's a problem of approach, not a lack of data.

People ingest a lot of data. Especially when they are young. They touch everything, look at everything in funny ways, chew on things, and so on. Try to interact with everything to gain data.

It's just not textual.

Great, so who are the people going to make those breakthroughs?

Last I checked they all work for...AFGAM (or are teaching)

Bless you for defining "AFGAM" - search engines weren't helping, and I was trying to put things like ML and AI in for context.

Ha, sorry about that. It's also been called FAMGA[1].

[1]https://www.google.com/search?q=FAMGA

OK. I'll coin the phrase: Afgamistan.

This is an excellent initialismanteau, I will have to use it.

I agree with this point. One can view this is as an extension of the current trend towards platform/infrastructure outsourcing. If AI and ML are deemed essential to future apps, we should be just as comfortable consuming it via a PaaS as we are with leveraging other parts of a cloud infrastructure.

A valid concern, as pointed out, is when the large platform providers begin moving into domain-specific markets, leveraging their scale and closing out start-ups/small business. However, I believe nimbler startups will always be able to carve out a successful niche as long as they are closer to the customers.

> However, I believe nimbler startups will always be able to carve out a successful niche as long as they are closer to the customers.

The difference is where the majority of the "idle wealth" is created. Sure, there will be great business doing consulting work leveraging cloud providers - you can make a good business simply consulting on such things.

However in the end you're simply enriching a large company. Whenever an "amazon cloud consultant" transitions a closet of servers to AWS - they made a bit of money, but the large bulk of the value of that customer is now in a giant corporation's hands vs. being spread out how it was traditionally (either in house, or via much smaller infrastructure providers that generally competed on service without much lock-in).

The lack of competition in the web space is astounding to me, and will be incredibly damaging to innovation. How quickly people forget why decentralization used to be considered such an important goal. Yet it appears the entire industry is rushing head long into this model.

One thing though is that although 1:1 specialized and individual service is always superior, the providers arent well paid. This means that startups, though better than large companies in a certain respect, can no longer become rich because they no longer are capable of creating a barrier to entry. A butler provides good service to you individually yes, but he can also be replaced fairly easily. (Relearning your quirks and stuff)

Startups then return to what they are in any other countries. Just a way to make a living rather than the potential path to riches it is in the bay area.

As a side effect you get smaller elites controlling more of the pie. That has other implications as well.

The stupid-money from spying-heavy web/mobile advertising is suppressing other business models that would be friendlier to smaller companies. Until it's gone (likely require a law—please oh please) we're (entrepreneurs and consumers alike) stuck with a handful of free-but-we-spy-on-you solutions, either provided by one of the big tech companies or soon-to-be acquired by them, and little else.

I think you need to pick a different niche. I don't know what your computer vision focuses on but try to pick something the big players aren't focusing or has less mass appeal. It also says to me that if you become a domain expert for this niche and develop the models, etc. that you are ripe to get acquired by one of the major players as automation (and software) continues to eat the world.

It's called commoditization. It's a pretty common phenomenon that follows the same well known cycle of first disrupting and then ushering the whole industry into a new era.

Nothing to worry about, unless you are trying to create these commoditized items, which is pretty much a doomed undertaking.

Not really, deep learning is still a research subject. It's not commodity for small companies, it's probably commodity for google, (e.g. they can use deep learning to design optimal deep learning systems)

Here's the central issue for someone outside the CS industry: I have large scale, high value data in a competitive, highly regulated, exciting market, something that bends GDP: defense, aerospace, healthcare, transportation. It's not like I got a bunch bored tweens in the Valley to share pics of their food. I have to sell my organization, which may be looking at a decade or more of data, that my CS collaborator can a) handle the data at scale, b) secure the data, c) hold on long enough to get to market. Because a couple years won't even get you to the FDA's or FAA's doorstep. NASA, DoD? FRA, NHTSA? You're looking at a 10 year lead time. You're looking at hosting tens of millions, hundreds of millions of dollars of data, running an equal amount of compute, and doing it for the next 10 years. Not the last 10 years. The next 10.

I can't, cannot, convince the org to bet the future on the ability and moral grounding of two random guys in a garage. They wouldn't even be able to wrap their heads around it, the difference in scales is too big: "You want to give our 10 PB of data to two guys in a garage? What? Get out. In fact, go get a new job, you're done here."

These deals are being signed by people who worry about capitalized words, like Gross National Product or Personal Health Information or Mars. We're talking orgs with tens of thousands of people who have committed their lives to a thing. We're not trying to use Dutch actions to steer 20-somethings in Orange County to the right sushi restaurant. Which is a sweet trick, don't get me wrong. But it's a bit different when we're trying to protect nations and their economies, and maybe just save the world or the human race.

Do you know what the US Government defines as "a large firm" for contracting purposes? 200 people. That's about right. You and I both know that those same two guys working for AFGAM are still the same two guys. But I need to contract with an entity wherein, if those guys I'm collaborating with go south, I have recourse with the Inc to keep the project on track. They can fire the guys, bring in new guys, subcontract some guys, whatever. But I've got to convince serious people that I can maintain a 10 year trajectory in a white hot, globally competitive market.

Someone convinced the CIA to drop a big fraction of a billion dollars on AWS, like 3-4 years ago. So it appears to be within the realm of feasibility.

Right, AWS. Not a couple of guys building out a cluster in their garage. That's my whole point.

I'm not entirely clear what you are complaining about, but I can't, cannot, convince the org to bet the future on the ability and moral grounding of two random guys in a garage. sounds like a pretty reasonable stance for an organization to take?

Enterprise sales takes lots of demos, trials on subsets of data, support arrangements, relationship building etc.

NASA, DoD? FRA, NHTSA? You're looking at a 10 year lead time.

DARPA programs have a fixed, 2 year span, and they aren't allowed to be renewed. It's deliver something in 2 years or fail.

You and I both know that those same two guys working for AFGAM are still the same two guys.

But it's not. Or at least those two people don't deliver a product, they deliver research outcomes, which then takes double the number of people to turn into reliable software, which then takes double that number of people to make deployable and run at scale, which then takes double that number of people to support. And then you need a sales team.

I'm not complaining, and I didn't say anything about DARPA. The parent was complaining that it's going to get harder to break into ML as a start-up. My point was to illustrate some of the reasons that big players want to play with other big players.

DARPA is DoD.

DARPA is a small fraction of DoD: it represents 3.6% of the R&D budget, and less than 0.5% of the total DoD budget.

Could you set yourself up as an independent unit under a larger company?

Sure, the US is not a fast mover in goverment innovation, and it has reasons to be conservative. (For example it's really big and complex [federal republic], and so on.)

But there are smaller orgs (let's say Mexico, or Spain, or Greece, or just a town in Germany), they might try whatever they think helps them.

Keep in mind you're watching a marketing demo.

I know this to be the case. It's all careful shown for the moment.

The nice thing that saves ML from being only another way for AFGAM to dominate and force all the innovation to a highly concentrated group of geniuses is transfer learning! You can download Imagenet trained brains and get them to classify stuff just as good as in the competitions and then adapt them to various different applications. Google has been nice and releasing some of these models and they'll form the basis of open robot brains, etc. in the future. There will probably be a Open Robot distro pre-packaged with a bunch of these brains and applications in the future.

Little companies can adapt these brains to niches that are too small for the big guys.

Transfer learning is great, but is still pretty limited; the main success stories are retraining CNNs from ImageNet to other datasets and word2vec/GloVE.

In the space of all the things you would want to do with ML, this is pretty limited.

[EDIT]: I stumbled upon a transfer learning for NER paper a second ago: https://arxiv.org/abs/1705.06273v1 And the results are not conclusive. You can can improve your results in truly data impoverished regimes, but you get diminishing returns as you add more data, so the only way to catch up is to get more data.

So, you may be able to get reasonable performance with fewer samples, but you will still get better performance with more samples, and the more samples you get, the less benefit you get from transfer learning.

This may not always hold, and I wonder if active learning paired with transfer learning would do better, but at the moment it's not convincing.

It's always been about the data. Algorithms talent and effort is at best a 2nd order effect - and this has always been true.

As a result, you would success in this area is mostly driven by the data. In other words: this is rolling out exactly how you would expect, no?

Oh it totally is, it's just one of those things that I think most of us expected to be more distributed rather than relegated to a few silos of data.

That's not to say there aren't opportunities, it's just that if anyone makes significant progress in one of those other areas, it's either on the back of AFGAM or will eventually be absorbed by them through acquisition/targeting.

> No clue where to go from here honestly

Would you be opposed to joining one of their AI teams? Clearly you believe they are building great stuff...

Not the GP, but many people in this situation cannot move to one of the few places the AFGAMs have offices (let alone the fewer places they've located their AI/ML teams), and working remotely doesn't seem to really be an option.

Except, their AI is simultaneously too stupid to preempt problems like the story we had with the Firebase customer. To generalize this a bit more, their AI is certainly too stupid to preempt similar problems with over-billing in general (though you can certainly create a little conspiracy around that one). And then to generalize even one layer higher, their AI is also too stupid to identify candidates who might get frustrated with their customer support and turn to the HN bully pulpit. And lastly, come on, they can't even predict the election winner! Otherwise why would they throw their weight behind the losing candidate? :-)

Because that was not AI, and that was not a problem for them they set out to solve.

You can out compete Google by simply keeping your services not deprecate themselves and make them available and having good customer service . Look at Clarfai when benchmarked it against Google's API and Microsoft Clarifai was more accurate .

I'm glad to see this comment at the top and setting the direction of the discussion. A year or two ago, this thread would have been dominated by ML is 'just' this or that, ML can only do XYZ, it's not AI, it's certainly not a path to AGI, and nothing is.

"IMO ML/AI is the last frontier for technology (an IMO humanity)"

This is too extreme. It is the last frontier of some set of problems, but it just introduces a new set of problems. This always happens. Even the implausible singularity still introduces more problems to solve.

I think the point is that our role in solving those problems diminishes. Certainly some, and maybe a lot.

no, this ML level still statistical inference. next step is about structure and generalization.

To balance out my other comment, check out Indica Labs. It's one guy in New Mexico: http://www.indicalab.com/

To be honest, your sentiment comes from selfishness. You want a fair playing field so that you can rise up but at the stake of slowing down progress for the entire world. Some fields are meant to be dominated and that's part of life man.

There are many reasons monopolies/duopolies are bad, it's not some selfish game.

That's not really the case here but let's say even it is, these big guys are producing most of the work for everyone to lean on. Look at all the open source initiatives out there. Don't stop progress for the sake of money.

I feel for you man. Same stuff is happening in IoT.

I think IoT is even worse, since the actual tech isn't new or innovative, just being applied everywhere because it has reached a certain price point now. It's also a badly integrated insecure mess currently that has a lot of bugs. Which screams for a big player to take over.

Yes, access to data is everything.

Reply on news.ycombinator.com