agibsonccc

I've been looking in to this for the java world. What's your use case? Deployment in to existing applications?

Disclaimer: I am involved with the eclipse foundation (not as part of the core staff)

Hi, I just want to expand on this a bit...

The "eclipse foundation" is actually not just sponsored by IBM. Nor is the IDE. As for what support it provides, that's actually spot on and is under a similar idea to what apache and linux provide. There are of course differences: but for most people the vague idea that open source foundations exist and provide some similar functions as a way to host projects is enough for this discussion

A lot of eclipse's revenue actually comes from working groups with Jakarta EE being the biggest one. There are also IOT, Automotive and many other working groups + projects under the eclipse umbrella.

If something were to be more "neutral" what would you hope to see exactly? Something performant is typically going to be framework/hardware specific.

Hi Fareesh, I'd love to hear more about your use case. Email's in my profile.

Hi, could you describe your use case a bit? Just an alarm trigger for deer in the backyard?

Could you elaborate on some of the problems you had overall?

Few questions:

1. Did you build this for your own use cases? Interesting side project?

2. How do you feel about the need for base64 being a requirement on the endpoints? Isn't GRPC the wrong medium for this? Also, what do you see as the main limitations right now? The models?

How did you set it up on the software side though? How flexible/customizable was it?

Your concerns are right on point. I agree that spark is a great sql/etl tool. My thinking was on the "math execution" part. Ray is able to doa bit more there. I do feel like there is a bit of hype riding going on here as well.

One interesting thing that could happen is the hardware gets better, and then these distributed schedulers might not be able to keep up with all the different options on the market.

There is also the tension of the hardware vendors wanting to give away things that only run on their chips vs the software makers who want things to run on every chip. It seems like there will be a lot of competition among the various infra players in the next few years now that nvidia is starting to have real competition now (even if it's not big yet)

It's still pretty early days for ray yet. That being said, spark never really got the hang of doing machine learning properly. It "works" but not for newer workloads which ray is trying to support.

It's good someone is building a company around it. I could see them building services on top of it and build a SAAS like databricks did with spark.

I'll be curious to see how ray matures.

Graal itself still has limitations with many libraries yet, the biggest one being having any library that heavily uses reflection. I'd look at some of the work red hat is doing with some of its java libraries and quarkus: https://developers.redhat.com/blog/2019/03/07/quarkus-next-g...

"Analytics" generally means "cython only". The numpy dep ends up being a problem.

Generally if you want python/java interaction, we maintain a tool called javacpp that handles this, we even bundle cpython: https://github.com/bytedeco/javacpp-presets

GraalVM itself also depends on javacpp for a portion of its features (LLVM wrapper): https://github.com/oracle/graal/blob/315f5dcf69c2e73fd13a5f8...

I'd be happy to answer questions about the overlap of the 2. I can say I happily execute python scripts from our embedded python and even point the python execution at an anaconda distribution.

Tech side here if you have any other questions!

Thanks Nikk!

We control the whole framework down to the bare metal. We also have our own built in memory allocators.

There's a bit more to jvm than just training models. Java based application servers are still widely deployed for example.

There's a whole data engineering niche we target here as well (for the spark pipelines) where python isn't a good fit for that team.

Then there's the fact we're still easier to run keras on spark than any other framework thanks to our model import.

That doesn't account for what we're doing with inference. Many vendors in this space are just running kubernetes bundling other tools they don't control. We actually engage various large companies in custom chip development running other DL frameworks due to this low level control.

Depending on what you're looking to do, we're still the standard for pre compiled binaries compiled as jar files: https://repo1.maven.org/maven2/org/nd4j/nd4j-native/1.0.0-be...

You won't find any other framework running pre cooked avx binaries and IBM power at the same time.

Happy to talk about more depending on what your focus is.

Thanks! Of note is we also contribute to keras and run javacpp: https://github.com/bytedeco/javacpp which allows us to run a lot of things you only get in python directly in c. We need to work on getting some of these things more well known. Hopefully we can focus more on better community growth this year.

I'd argue in general that outside of python, there's not really much of a focus from google itself. They are largely leaving other language bindings to other people (see what's currently going on with tensorflow 2.0).

Their focus is more on the c bindings and allowing other people to build what they want on top of that.

Other language bindings aren't generally going to be used for anything more than inference. First class actual data science work isn't going to happen in other languages anytime soon (at least outside of julia and R which are at least trying to compete in this niche).

Have you guys looked at the tidelift model in comparison? Would love to hear thoughts there.

I was there from 2008 to 2013. This gives a fairly good break down: https://www.mtu.edu/alumni/favorites/snowfall/ I saw a lot of snow storms there over the years. Neat fact: The houses in the UP actually have doors that lead nowhere on higher floors in case your first floor is buried :).

I went to university in the Keewenaw (Michigan Tech). It's also where I started learning how to run a company. I spent 5 years of my life there.

Happy to answer questions about anything tech related there!

Hey folks:

Sorry I just saw this post. We're actually intending on implementing allreduce as well. Right now, the initial focus was on implementing something more robust. There's a lot of elements of our distributed training that isn't talked about in the post(this was meant to be more of a high level overview).

A few additional things:

1. First and foremost fault tolerance and making spark run well was a bigger priority for us. Spark and its ilk don't do well with gpu clusters. Our initial focus was more taking what we had and making it work well and running everywhere with no code changes.

A common workflow that "just works" is being able to run model import on spark and run things as is. My colleague max is behind elephas (which we've since adopted for our python interface for dl4j on spark).

2. What we've seen many people don't have is MPI. We have this hard constraint of running things in strange on prem environments. So instead, we focus more on things like multi cast udp and compression/quantization to speed things up the networking as much as we can.

3. When we go to implement all reduce we'll be focusing on reusing as much of this as we can. We'll also try to figure out how to reuse our existing parts tha twork well like our cyclic buffer re use called workspaces: https://deeplearning4j.org/docs/latest/deeplearning4j-config...

Any other feedback folks have would be appreciated.

Thanks! Usually dl4j is targeted at data engineers, not python folks. We actually have the end to end system working.

Hi Mark, I want to correct one thing. My company is behind dl4j. We are also the second largest contributor to keras. We have near perfect model import (with keras contributors maintaining it). Dl4j also has TF import and a native graph runner: https://github.com/deeplearning4j/dl4j-examples/tree/master/...

Java itself tends to be fairly verbose I know, but not only are there wrappers but for inference if you want to use python for basics and then us for inference. We enable several ways to do that. Folks tend to oversensationlize these "us vs them" framework "fights".

Rather than do that, why not just use what's good for each use case? Python for training, java for deployment?

Beyond that, you could also consider building your own wrapper maybe? In clojure, there's jutsu.ai for example: https://github.com/hswick/jutsu.ai

Congrats Plangrid. I was your neighbor back in 2014. I remember when you guys needed to expand your office and you were looking to start using some of the space we were subletting all the way back when.

Ironically, 2 years after that I got in to YC and now run our company from Japan. Time flies!

Sure! So our model server usually replaces a bunch of docker containers with different standards with zero ways of inputting data.

An interesting example was a customer that bought a startup. The startup had a docker container that leaked memory (they were using an advanced TF library, google quit maintaining it). The parent company had tomcat infrastructure. They told the startup to get it to work in tomcat. They approached us.

Dl4j itself is built on top of a more fundamental building block that allows us to do some cool stuff in c: https://github.com/bytedeco/javacpp-presets

We maintain the above as well. We have our own tensorflow, caffe, opencv,onnx etc bindings. This gives you the same kind of pointer/zero copy interop you would in python but in java.

Another area we play in is RPA. You have windows consultants who don't know machine learning but deal with business process data all day. They want standardized APIs and to run on prem on windows: https://www.uipath.com/webinars/rpa-innovation-week/custom-b...

They want the ability to automatically maintain models after they are implemented with an integration that auto corrects the model. We do that using our platform's apis. On line retraining is another thing we do.

We also do automatically rollback based on feedback if you specify a test set. If we find the test is less accurate for a specified metric over time, we roll back the model. This prevents a problem in machine learning called concept drift which means the domain changes over time.

Lastly, you have customers generating their own models and you need to track/audit them all. Sometimes to do online retraining like the RPA use case. Some customers scale to millions of models. They need this kind of tracking.

Disclaimer: I sell a product in this space.

Our model server just works with every format. We created our own DL framework and maintain that. Eg: We're the 2nd largest contributor to keras alongside having our own DL framework.

We've found it's easier to just have 1 runtime with PMML, TF file import, keras file import, onnx interop coupled with endpoints for reading and writing numpy, apache arrow's tensor format, and our own libraries binary format. We also have converters for r and python pickle to pmml as well. Happy to elaborate if interested.

For ingest we typically allow storage of transformations, accepting raw images, arrays etc. We have our own HDF5 bindings as well but mainly for reading keras files. Could you tell me a bit about what you might want for ETL from HDf5?

Congrats on your round! FWIW, (me being in a space adjacent to yours) I see these kinds of comments all the time about the "my 1 off pipeline seems good enough".

Most people don't need "industrial strength" till you hit a certain scale. They tend to optimize for ease of use/simplicity. It's one thing if you don't have to change their workflow, it's another if you have to not only have people change their workflow but also teach something new.

Convenience matters more. There's a pain threshold of "new tool" vs "this costs me x amount of time".

How are you guys overcoming this? Even though we're also in the infra space, I've never seen you guys out in the wild. Where would I bump in to you and what is the scale I would want to add the complexity of k8s + pachyderm + whatever other deps you guys have over just using an S3 bucket?

Another question: Why hasn't AWS just packaged this up and offered it as an extension of their k8s service? How are you guys going to overcome that?

Usually I see "hybrid cloud" or "on prem" as the response. If it is on prem, are you guys relying on the presence of k8s at customers? Do you guys use something like gravitational?

1) Yes I think incentive structures are a part of it. For us as authors,I know I typically think about the website very differently than the book. Having an editor peer review also helps keep me focused. What you find with a lot of open source coders, they tend not to always be good writers (me included). My book is 530 pages and took 2.5 years to crank out in between new releases. My company funded the book's development more as a loss leader. Authors don't make crap from the royalites. You're better selling it as part of a larger package such as training or support.

2. You have to know the right people. Go to the conferences and ask them. It took a long time. The profit shares aren't much. I more did it for the exposure at their conferences. We found it to be worth it as part of a larger package and networking with others in the field more than anything else. The notoriety has really helped as well. It's a lot of work, but definitely worth it if you can get it. Just be ready to put in a metric ton of work.

I'd argue it's actually the economics. End users both want everything for free expecting it to work perfectly as well as assurances that the project isn't going away while also somehow still wanting guarantees around release schedules. Those same people also endlessly complain to OSS maintainers about something they are getting for free and somehow expect the red carpet when it comes to feature requests.

Attribution isn't the issue, it's balancing the need for building a community vs the financial incentives to actually support the people building the thing you're using.

Disclaimer: Your concern is exactly what pays my salary. Take what I say with a grain of salt.

I typically see this attitude at the developer level, but the line of business never sees it like that. Devs don't have incentive to care about these things. They also just want to deal with the vendor.

What you sell is more "insurance" and a guaranteed timeline on bug fixes/releases.

As someone who supports OSS, why should you be able to demand I ship something on a certain schedule if you're not paying me? If it's a business transaction, then there's a contract and aligned incentives on both sides.

Taking this a step further, this is usually not enough which is why we then see open core business models like elastic search, gitlab, (and also my company skymind) selling a combination of support + licensing.