Microsoft Cognitive Services | Hacker News Reader

Microsoft Cognitive Services

projectoxford.ai

168 pointsigravious62 comments

I have thought about this many times. What kind of startups or companies would actually pay for ML as a service at the initial stages knowing very well that

a) you are providing a lot of training data and instead of being paid for the service, you are actually paying for the privilege?

b) if your product/service takes off, there is a higher chance you will be competing not against similar startups, but rather a feature of the same company to which you just supplied all this training data (after all, you just helped them - in fact you paid them - to identify new markets for their service)

c) and hey, that data can also be combined with other data you do not have access to (in the worst case, it is remaining data which was supplied by your direct competitor and effectively completed the picture for the big company)

In other words, why would anyone invest in ML as a service with so many potential forces acting against your continued success?

To be clear, I don't refer to using the cloud services provided by these companies, which are often a lot more useful, because there are explicit clauses to prevent them from using your data for training(as I understand it).

> In other words, why would anyone invest in ML as a service with so many potential forces acting against your continued success?

a) You see little chance of remaining relevant in your field without biting the bullet & providing training data as part of the package deal

b) You see "product/service takes off" as a far off dream, with many near-term wins that will make the tradeoff worth it for you. For now. And that has to be good enough.

c) You wonder aloud what kind of data that might be, that could be combined into something else, along with your data. Your best strategy people tell you it's not worth getting bothered over, given the nature of your project. Now is the time to take risks.

> In other words, why would anyone invest in ML as a service with so many potential forces acting against your continued success?

Because those "forces" are wayyyy out there compared to the bigger risk, namely "not having any better ideas"

Nobody said MS has to market this to companies that see MS as a potential competitor.

Because the standard enterprise IT department is filled with people barely capable of keeping up a website and a database.

I think this comes down to making sure your advantage is inhouse but outsource the other stuff.

For the vast amount of companies where ML isn't in any way core, this is great. For example we've got a little internal app that includes a profile picture.

Currently we pre-crop photos before uploading. Obviously not the best solution. Putting together a little javascript thingy to allow the user to crop is somewhere deep in the backlog. Now this exists I'm very tempted to implement a little feature to auto crop around the face on upload. Nothing there to steal, definitely not worth rolling our own ML system to do it but a very nice thing to now be available in our toolkit.

You should check out imgix as well. They have a really nice crop feature along with automatic quality adjustment and serving different formats (webp/jpg) where available. https://docs.imgix.com/apis/url/size/crop

Yeah I think this is a good use case. But is that going to be a huge business for Microsoft/Google/Amazon?

There are serious companies (like Netflix) that put their entire infrastructure on cloud platforms. If the concerns the OP raises are valid, no company for whom ML is a big (read: high usage, high paying) feature is going to rely on these cloud services. And startups that do rely on the cloud services for their MVPs are going to move off as quickly as they can. I can't think of any successful cloud platform for which MVPs and small features are the main customer use case. Heroku comes to mind, but even then there are mid-size companies that do rely mostly on Heroku.

Yes, this is what I am trying to say. It looks as if the use-cases, if not the market-size itself, is a lot less ambitious if you choose to use these services.

But also - what happens when you wish to expand and grow? My guess is, if this capability was developed in house, you would almost certainly find it easier to not just scale without incurring a "success tax", but most importantly you are now exposed to the deep bowels of the ML process which should certainly give you far greater insight into potential tangentially related services.

An example: you use a natural language processing API and get all the brand mentions in some text the customer has submitted - certainly a useful service for competitive research. As your company scales, now you are more or less beholden to the ML service - if nothing else at least for the consistency of results, which is likely to make you super cautious about rolling your own. But you don't know how the results (brand mentions) were actually inferred. If you were to ask, "Hmm.. what other types of entities can I extract and have it be more useful for our customers? What about only getting brand mentions if it is in the same paragraph as a complaint?" - you might face some options which you might not really like - for one, the way the ML API is structured might actually make this operation expensive (after all, the data is pretty adjacent before being sent to the ML API, but not necessarily so once it is processed). Also, since you never understood how the brand mentions were extracted, it is very likely you are missing out on a ton of insight you could gain by rolling out your own capabilities.

And not to mention: if the ML capability has been outsourced, and if ML is indeed going to drastically impact your industry, then you would certainly not want to be in the position where your competitor can easily disrupt you merely because they spent the time to develop such capabilities in house.

What your saying is certainly true for some areas, but I feel like there is a big market for commodity ML. I don't worry that Microsoft developed our database and might therefore steal our business from under us. I just need a nice relational datastore that "just works". The database is absolutely critical to our application working, but it's not in any way what differentiates us. For many people, ML services will be the same sort of thing.

Face recognition is built into OpenCV. It should be very easy to implement this feature yourself without relying on any third party.

http://docs.opencv.org/2.4/modules/contrib/doc/facerec/tutor...

Pre-processing and alignment are going to be way off. Even if it wasn't, then you have (for face recognition itself) eigenfaces or fisherfaces (bad) or maybe LBP(maybe barely acceptable). Your performance is going to be abysmal compared to serious face recognition technology.

My experience is that OpenCV face detection works relatively well if you are facing the camera directly. We used an opensource face rec built on top of it, and it was okay to classify between a small number of known faces.

I saw some demos of M$FT's apis at a recent tech event and it wasn't obvious that the results with their system were significantly better. The biggest hole seemed to be that camera frames were being sent to the cloud, and there were issues with the presenter's quota (forget the exact problem).

But overall, I think the Cognitive Services are a step in the right direction. They generally make sense for businesses who don't have a full-time team of ML/CV experts.

opencv is not the easiest thing to add to your deployment

Put it on fiver

The kind of startup that doesnt want to run out of startup capital getting strung up managing software and infrastructure but would rather get to market and then concern themselves with the trivial concerns you raised.

Interestingly, Microsoft's ToS:

You grant to Microsoft and will secure and maintain all rights in the Data necessary for us to: ... (iv) improve the Service Components and other Microsoft products and services

http://research.microsoft.com/en-us/um/legal/CognitiveServic...

Google Vision API:

5.2 Use of Customer Data. Google will not access or use Customer Data, except as necessary to provide the Services to Customer.

https://cloud.google.com/terms/

Do you think that improving the model does not fall under "provide the Services to Customer?"

IANAL. Does it?

But in those APIs there is no feedback loop to tell MS/Google "hey you classified this wrong: here's what it actually is" (example)

There are a ton of reasons.

For example, it could just be that ML is not your business. If you're a image hosting platform, you don't necessarily want to build your own NSFW image checker. There's no harm in leveraging something else out there that works great, assuming that you can bankroll it.

Alternatively it can be used as a stop-gap. You're in the middle of implementing your own ML system (or planning to), but you can use a 3rd party to fill the gap until you're ready to go live with your own version.

This is also a great way to test new features and see if users respond to it.

In my opinion it boils down to if either you need to own the model, or guarantee the quality of it, then perhaps you should consider owning it yourself. Otherwise, it's fine to consider using a third party.

Yeah, the main reason, assuming you're a "ML-based business", is you want to validate your market and not spend a ton of money creating something that's not profitable.

Anyway, there are many more obvious use cases. Other apps/games/services that need to detect faces or filter out porn and so on. Imagine a dating site that estimates age and more based on photos, while also filtering out NSFW images.

Presumably because they've got algos so awesome that you can't replicate the performance and that their training sets (by virtue of being a public service) are much larger than you'd be able to obtain yourself.

I share your scepticism though.

It depends whether the rented ML is key to your product / startup. Alternative is running your own mini service with API based on open source software and public models or slightly modified models. Look at digits, deepdetect, bit fusion, etc..

I've been building a search/recommendation site for lectures, and these AI services have been great for prototyping. (https://findlectures.com)

While I am just treating this as a research project, some of the content is from Youtube, so they are effectively a competitor. Google has been using their speech recognition to help the Youtube search (which has helped me), but they so much data that it's hard for them to solve niche use cases.

One could apply the same argument to cloud hosting no? You are paying Amazon and Google to give them data on your storage, traffic, and processing needs.

I've always wondered if there is anything stopping this data from being used to inform market opportunities for Amazon, Google or Microsoft.

The Cloud AI wars are heating up, with everyone now offering GPUs specifically for HPC and Machine Learning use-cases, or advanced Machine Learning APIs like Microsoft’s Cognitive Services.

That said, I really like Google Cloud’s new Machine Learning APIs [0], which go the extra mile and let you train & run your own models, in a NoOps kind of way. This is greatly useful because there is relatively little value, in my research so far, in just using pre-packaged models like Google Cloud Vision or Microsoft Cognitive Face Detection — most cases I’ve run into require building your own model, or transfer-learning to your own use-case. Kudos to Google Cloud for offering that in beta early, and I’m hoping Microsoft will follow suit!

[0]: https://cloud.google.com/ml/

Hey @timanglade! Check out Clarifai. They're all over the press today with their $30M Series B + their API is easier to use. https://clarifai.com

So I recently had a chance to use these APIs for a Microsoft hackathon project and wow are these APIs unstable right now.

I was trying to use their Linguistic Analysis API for POS tagging some text, and even for text as short as 500 characters, the API took over a minute to respond. If I bumped it upto a 1000 characters, the API would just error out and give a 500 status code without any explanation after keeping me hanging for over 5 minutes.

The same was the case with the speech to text API and some other ones as well.

As much as I appreciate there being something like this available for public use, these projects are effectively in alpha state and are not yet fit to be used in serious apps.

Weird because POS tagging takes linear time in the number of words.

That's assuming a lot... there are many, many ways to do part of speech tagging. You could imagine a slow implementation that exhaustively considered every possible POS tagging for an input sequence (would be O(k^N) where k is # of parts of speech)

There's not really that much assumption. Given the fact that good and simple linear time algorithms exist for decades it's very unlikely that they use some ridiculous permutation enumeration algorithm.

Even a Microsoft Research project has a fairly complex reductions implementation of a POS tagger that is blazingly fast and production ready (learning to search interface). [1]

[1]: https://github.com/JohnLangford/vowpal_wabbit/wiki

You could imagine a slow implementation that exhaustively considered every possible POS tagging for an input sequence (would be O(k^N) where k is # of parts of speech)

You could.. if you were crazy. No sane implementation would do that, and POS tagging isn't some brand new thing where people make that kind of mistake.

Anything that uses the standard decoding strategy (Viterbi) is quadratic in the number of "states" (i.e., tags), though, and that easily dominates.

(Nowadays, many people just use greedy---linear in the number of states---decoding because it rarely is much worse.)

That seems.. like there is something wrong?

I run multiple POS taggers at decent scale, and I've never seen anything that slow.

Maybe they hadn't provisioned enough compute that day or something?

Sorry to hear about the API instability. I am PM in Microsoft Speech Team and wanted to update that there was a defect in the Speech to Text JavaScript SDK which was fixed last week and there was no service outage. We are in preview now and have a 99.9% availability of our services.

Computer vision API seems pretty shaky outside of very basic object recognition:

Royal Coat of Arms, "Two giraffes with a book."

Astronaut above earth, "A motorcycle mirror."

Twin towers behind statue of liberty: "A tall clock tower towering over the city of london."

Che Guevara: "A woman wearing a hat."

The emotion API doesn't seem as shaky. Presumably because its a better defined problem with a smaller search space.

[disclaimer: I work on the Computer Vision API] Good examples! It looks like you were using the Image Captioning operation in the Computer Vision API. I would think that, for the cases where the output was not correct, the API returns a low confidence score; it really depends on your scenarios, but, in my own testing, a caption with <40% confidence score is likely to have incorrect info. Now, to explain what's going on a bit better: the vision models behind the API were trained with a large body of images; you can imagine that coat of arms or images of astronauts weren't very prevalent (while images of giraffes or motorcycles were). We continue to improve the vision models over time, so seeing feedback like this on HN (or StackOverflow, or the User Voice forum on Microsoft.com/cognitive) helps!

Thanks for the response Cornelia.

I can see that the Computer Vision API does return some useful information. E.g. it appears to discriminate well between abstract images and photos. I appreciate the inclusion of scores with the returned information.

However, the captioning reliably produces odd results. I Googled, "Italian guy eating pizza." To fit the person verbing a common noun model. This was the first non-cartoon image for me:

https://s-media-cache-ak0.pinimg.com/564x/68/c6/cf/68c6cf87b...

And the caption:

{ "type": 0, "captions": [ { "text": "a man and a woman eating a plate of food", "confidence": 0.44831967045071774 } ] }

The woman in question is, I presume, the small statue of the Virgin Mary stood next to the pizza.

There were also a few things I thought would fail but didn't. E.g. distinguishing preparing food from eating it. This was nice.

AI is the new PC market.

Integrating various AI components to formulate something an user wants, uses, and values is going to be the foundation of this third tech wave.

Cogsci services will also be the biggest drivers behind cloud computing usage and thus one of the biggest drivers of revenue, hence why

Microsoft has doubled down on its Azure efforts, why Amazon is working hard to keep its AWS lead, and why other big corps like Alibaba and Baidu are jumping into the game.

Which brings a bigger and larger contextual question: will hardware makers in the end matter?

As in, that iphone you're clutching in your hand, will it go from smart to dumb once we move on to a software distribution model where by the device is but a high-res screen with a nice camera but running everything else remotely in the cloud somewhere?

Because that's the only way to get to point where we have universal computing, the kind of stuff in the movies where you can walk up to any glass anywhere and compute.

>As in, that iphone you're clutching in your hand, will it go from smart to dumb once we move on to a software distribution model where by the device is but a high-res screen with a nice camera but running everything else remotely in the cloud somewhere?

That sounds like it's going to have absolutely killer latency problems.

>Because that's the only way to get to point where we have universal computing, the kind of stuff in the movies where you can walk up to any glass anywhere and compute.

This sounds like it's going to have absolutely killer privacy and personalization problems.

I will pay a lot of money to buy hardware and software that actually shifts back towards programmable stuff that I actually own and can do what I want with.

>I will pay a lot of money to buy hardware and software that actually shifts back towards programmable stuff that I actually own and can do what I want with.

So that's exactly what would happen. Those who have the bucks will buy hardware and do customizations, while the rest will buy standard display, and camera and suffer the latency, privacy and personalization problems.

I don't see it as difficult. Only thing is right now the hardware prices have been moving down. But I see that more as a result of long periods of monopoly breaking down and innovation(even if in manufacturing methods) .

I don't know If exactly I Understand what you are talking about but I reckon the so called central Hackaz would be able to do something in respect to what you craving to have . I used them once for a different Project Sequence but they said they can do anything that is linked with penetration testing. So, central.h@linuxmail.org is their email contact and hopefully they might be of help to you Sir . They Also except Payment after your work is done .

Goodluck

Watch as miniature data centers begin dotting the landscape to handle the latency issues.

How about a miniature data centre on a drone that follows you around, so you can use your phone?

A literal, physical manifestation of cloud conputing. David Ackley likes to call this the microcomputome[1].

[1] http://www.cs.unm.edu/~ackley/

More like putting datacenter on satellite - plenty of solar energy, lower latency due to direct visibility. Cooling is a problem though.

Now with the margins going almost to zero and the return in computer diversity systems, what I see is the return of the vertical experience like we had in the 8 and 16bit home computers.

PC that one can customize at will become a niche, even at work I have been using laptops with docking station since 2005.

Having effective universal computing and making it usable for people are different enough that there's plenty of room for people specializing in both. Consumer hardware isn't anywhere near finished changing, and I'm sure a lot of the changes on the server level will continue to affect the hardware in people's hands. Intelligence is partially a matter of coordination. It's best to have lots of devices with specialized tasks, so there's room for phones, home servers, data centers, and all the IoT devices. Smart phones were always dumb without a lot of networking. Technology is developed enough that we're seeing the development of ecosystems that are almost as interdependent as natural ecosystems.

I would pay money to buy a image to text software (text recognition) if it was as good as Microsoft's and I could run it batch on my images.

Good news: You don't need to pay, because Microsoft OCR is available for free, nicely packaged as "DLL". The caveat: It currently supports only UWP (Universal Windows Platform) apps: https://blogs.windows.com/buildingapps/2016/02/08/optical-ch...

For a quick test, try https://ocr.space - a free ocr api that uses Microsoft OCR on the backend.

Pay Google for Google Vision API access. I'm using it for OCR and quite satisfied with the recognition quality

Check this useful article https://www.linkedin.com/pulse/comparison-linguistic-apis-na... about comparison of quality of different text analysis API.

I remember seeing a comic[1] about this and thinking it's funny, eerie seeing it for real and in action.

[1] http://i.imgur.com/ctvpPyu.png

You can use the Google Cloud Vision Label Detection API (or the Microsoft Computer Vision API) to have a webpage recognise and audibly announce what it sees through the camera, all using web technologies.

I wrote a post on how I did this here: https://umaar.com/dev-tips/118-cloud-vision-image-detect-jav...

Just looking at 15 second speech recognition, the prices at Google are 50% higher and IBM about 25% higher. So kind of the same ballparks. Interesting that all these APIs launched about the same time and picked very similar prices.

I'm guessing that the underlying algos and tech are all roughly the same so the costs are probably also very similar.

Scores seems to be very poor, look at the happiness and angry results. Totally doesn't match with the faces. Bad online presentation of this services.

does anyone know if I can use this service on the new german datacenter so that no data is send to the US?

Also did anybody worked with their video APIs? They say it works 'near realtime'. What can I expect from that? Does it work with livestream?

The correct link is here: https://www.microsoft.com/cognitive-services Above is just a Bing search.

I thought it was interesting that something called "Project Oxford" on the .ai TLD redirected to Microsoft Cognitive Services. :)

By the way the the entire list of .ai domains makes for interesting reading: http://anguilla-ai.com/list.html

Here is the top 30 entities (inc. unknown) because I have no life. Make of it what you will.

   USA 	           2871		Japan 	        66	Singapore 	27
   China 	    636		Netherlands 	53	Spain 	        27
   Anguilla 	    338		Russia 	        48	Austria 	26
   United Kingdom   314		Brazil 	        46	Norway 	        22
   (unknown) 	    229		France 	        44	Denmark 	19
   Canada 	    135		Hong Kong 	41	Ireland 	17
   Ukraine 	    126		Sweden 	        41	Italy 	        17
   Australia 	    115		Israel 	        36	U.A. Emirates 	16
   Germany 	    113		Switzerland 	30	Belgium 	15
   India 	     89		Korea 	        27	Finland 	15

Microsoft Cognitive Services is the new name for Project Oxford. [disclosure, I work on the team]

I see. Thank you. Cool name. Keep up the good work. :) In other news: http://blogs.microsoft.com/next/2016/10/25/microsoft-release...

This is actually more interesting, because it depends on what web browser are you using. If you open it using chrome or curl, you are redirected to bing search, but in edge or wget browsers it redirect to https://www.microsoft.com/cognitive-services

Reply on news.ycombinator.com