Show HN: DeepForge – A Modern Development Environment for Deep Learning | Hacker News Reader

Show HN: DeepForge – A Modern Development Environment for Deep Learning

deepforge.org

108 pointswilliamtrask11 comments

I love these initiatives and would love to learn about any other end-to-end deep learning architectures. It took us a long time to build up our in-house tagging infrastructure and whole model warehouse + inference servers that are connected to Kafka. I'd love something more polished though. It's a killer when you want to do ML in the real world, but have to spend forever rolling out infrastructure.

I'd be really curious to hear more about how you set up your tagging infrastructure and model warehouse.

you need applications and queryable storage for maintaining models both for production use, internal testing/validation/research, and everything in between. Maintaining a model - or collection of models - implies maintaining a ton of data that can ideally be parameterized (learning data, build info, tuning parameters, purpose, priority, etc) and there are a lot of sophisticated ways to handle this all efficiently mostly dictated by your problem space (algorithm, data volume, response time, research methodology, etc). This is all outside (or inside) of whatever application runs your actual business model and the infrastructure necessary to query your "live" model(s) in a distributed/replicated environment if necessary (as is for most big data projects).

Have you considered AzureML? http://studio.azureml.net It's really slick, and you can program it in Python and/or R, you aren't tied to MS stack.

They don't have recurrent neural networks or most Deep Learning stuff. The closest to all these, IMO, is H2o Flow

I am building Deep Video Analytics which aims to become Database+Processing framework for visual data. With DVA you can load, store, decode, annotate, apply inbuilt and custom Deep learning algorithms. Further you can examine the data through full text and visual search. It's comes with a Django+Postgres+Celery/RabbitMQ setup. And scales across multiple machines on AWS using EFS + S3.

https://www.deepvideoanalytics.com

OT but I want a service where I can upload a set of images, manually tag them with a few categories and then let some kind of AI tag the rest of them based on the tags I put on the dataset. I'm not looking for a general image recognition engine. The tags are pretty abstract things like "enjoyable".

I'm not kidding, but I proposed doing something like this as an open source project for the Segment open source fellowship. I didn't qualify but I am keen on doing this. A few priorities have taken over lately, looking for a job primarily but I will get there. My goal is slightly different than what you have mentioned here. I want to start with a few tagged images and use clustering to get more images, tag them and increase the training set volume.

https://www.clarifai.com/ can probably do this. I used it at a Hackathon to quickly toss together a food recogniser with custom classes.

http://www.scaleapi.com/ - It's backed by MT, but will get the job done if you only care about getting labeled data and not the 'wow' factor of an AI doing all the intermediate work.

Have you tried using IBM watson? https://www.ibm.com/watson/developercloud/visual-recognition...

The API allows you to upload and store a set of images tagged with a given classification under your own collection and later upload new images and the API will try to classify that based on your trained collection. But it's mostly based on object detection I guess, not sure if it works with emotions.. (Haven't played around and am new to AI).

But it's just an API only. If you are looking for a end-end service, let me know. We are trying to build a platform that makes it easy to bridge several APIs together (Like pull images from S3 and apply ML algorithms on it and store it to say dropbox etc)

Reply on news.ycombinator.com