varontron

many corporations enable internal moves, and it can be easier to get a new job inside because you're onsite, and thus more easily recruitable, you meet people, you learn of opportunities before they're public, etc.; though obviously, not a guarantee.

Google has its fingers in many many pies. Software, hardware, transportation, OS, media, mobile, biology, genetics, algos, etc.

LinkedIn is a rolodex.

There is distinction between the scenario you describe, and the goal of YADA, which is that with YADA we strive to make the first (or existing) system useful, and to keep the bar low, by favoring the indispensable skills like SQL; or, to simplify the first, second, third, or whatever system by reducing the number of components, redundant configurations, and points of failure. Further, YADA facilitates solving a problem that many of the optimal systems create, those related to workflow, collaboration across teams, efficiency, repurposing, etc.

Thanks for taking the time to review and comment!

Thanks for taking the time to comment. You're basically rephrasing the hammer cliche, which is accurate. First though, as your first sentence indicates, you agree we still need hammers. Right now, YADA is a great tool for prototyping, adhoc reporting, data analytics, ETL, and smaller SPJAs. One testament in our environment has been to repurpose the work we do for one endpoint in another, e.g. webapp and spotfire, or vendor app and ad hoc reports. We are working with a high-volume machine-learning group, so eventually we expect to be able to scale.

Thanks for taking the time to review and comment.

YADA doesn't abstract any underlying system, it abstracts access to the underlying system. You can think of it as a pluggable web service that you slap on, say, a legacy warehouse, except it can also reach into any other system, enabling access to data using the same standard syntax.

It's like a BI or ETL tool in that respect, except those tools wall off the work you've done and limit repurposing. BI tools generate reports and that's pretty much it. You couldn't, for example, point Spotfire to Cognos, you'd connect them both to the db instead, distributing the credentials, and possibly construct the same query in both places using their interfaces. With YADA, you could instead write the SQL once, and get the data from anywhere.

I do agree with your sentiment however, that as innovators we always think there is a better way, or more commonly, that their _must_ be a better way, because we don't understand the problem fully, or grasp the complexity of the current solution. Often we discover we're not the first to think of a great idea. I'm sure there are similar tools to YADA, but I also think YADA has potential to offer a wide array of users some options and combinations of features that are distinct in the marketplace.

Thanks for taking the time to review and comment.

ETL is a viable use case, one of the first in fact, dating back to 2011, in which we marshalled 100s of millions of RNASeq gene expression values and metadata into a dw.

Regarding other ETL tools, one of the goals of YADA is to make the process easier. The tools you describe are well known and robust, but as you point out, complicated and costly. YADA was designed to be much simpler. Further, it can also be used for a variety of other things, like SPJAs, data analytics, etc, securely, and without additional overhead.

It started as a acronym (albeit a tired one) for yet another data abstraction, and it caught on.

This is an excellent point–more accessible advanced documentation. YADA has tremendous flexibility, and if you actually read the readme, then you know, a lot of use cases. There's always a trade-off between appealing to both experts and novices, but I'm certain we can balance things better.

There isn't currently an Adaptor authoring guide, so that would obviously be a good thing. There are some code samples for usage in user guide.

In any event, thanks for giving it a look and taking time to comment.