Introducing Sense – A Next-Generation Platform for Data Science | Hacker News Reader

Introducing Sense – A Next-Generation Platform for Data Science

blog.sense.io

81 pointstristanz25 comments

Why does every data science infrastructure provider want to be full stack? I really don't want to use an online IDE, I really don't want to be locked into one particular way of distributing workloads, I really don't want to move my code from GitHub. All of the individual components look great, but the all-or-nothing proposition is really off-putting.

When approaching a beachhead market, it's best to try to deliver a whole product rather than to spread one's self thin specializing for disparate consumers. Clearly users will appreciate some features of their offering more than others; shooting for full-stack at launch both gets the company more feedback and allows them the most control over the initial relationship.

I totally agree that the full-stack offering can be rather off-putting, but the business side of DS still doesn't understand how to reliably make money from this market. Cloudera (and friends) have made money from a full-stack offering, so it's easy try to imitate this success.

IMO the online IDE is very useful for expositions and small projects, but indeed a limiting playground for larger projects. I'd love to see SAAS/PAAS do better at co-locating their offerings so that, e.g., I could run part of the stack on my own hardware (which is where the data lives). Perhaps Docker would help facilitate this business model.

We definitely don't want to lock you in, just to round off the rough edges of the "on your own" experience. You can create a project from GitHub and push/pull to GitHub from Sense, and our distributed computing setup was chosen to be low-level enough to support many of the existing distributed computing tools like doRedis, but still way easier than provisioning and plumbing servers.

We know that there's room for improvement on some workflows, in particular Git interop, but we have our sights on a platform that plays well with as many others as possible. Please ping us with any other thoughts you may have!

So, what do you want? Just RStudio?

Or do you want more utilities that work together and run locally? (i.e. repeatable workflows in project bundles of scripts + data + saved notebook logs etc).

We were just looking at it. It's a nice interface -- it's logical and someone gave it a thought. They also have some of the features we (at SageMathCloud) were talking about with regards to socializing: e.g. following a user and in reverse having followers, etc. Also nice is that they have project templates (for python, julia and R, basically), where you get some explanatory *.md files to read. (I.e. not an empty project). You can also start a project from a git/github source and so on.

It seems to be heavily broken right now, or heavily unstable. Hmpf... I'm trying to start the ipython notebook (you have to do this on demand), then you can start editing the files ... but it is stuck. Now it's completely broken.

The terminal works fine now, and a project is running in some kind of isolated container in beefy linux ubuntu 14.04, running under a common "sense" user (which is the only user). They are also running some node.js client and their "butterfly" server.

Ok, well, I tried to test it further or add a collaborator, but the website just says "502 application not responding".

It didn't seem like they have real-time editing or LaTeX.

few more details. ... now it runs fine again. two screenshots attached. ... one is the project view, the other one the "workbench".

They run a plain "R" terminal, but with a neat HTML output. It's a vertically split editor, where the .r file is on the left, and you can select lines of text which are then executed via "ctrl+return". That's exactly how those R people work (instead of having cells like ipython or sagews)

The same for python files: one or all lines are fed into the "ipython terminal", not the notebook (!), and then translated to some html output on the right. plots work fine, etc.

So, on the right of the screenshots, you can see the copies of the input lines.

By default, they add a .gitconfig file with the username and email address. That's cool, and I guess this could be done for SMC as well.

There is no auto-save for files, just lost the content.

-- William Stein and Harald Schilly

Sorry about that! We just got a huge spike in traffic from HN and VentureBeat so are trying to handle the demand.

Thanks -- it started working, and I've updated the comment above.

Your email campaign is also running, that pulled me in

I took a look - registered for the free personal version to try it out. I like the ideas and a nice working demo that sucked me in. I fiddled around with modifying the R code and running it again.

It was a little confusing in places. Eventually I figured out that I need to save the modifications, before running it again. I think I ended up starting up a couple console sessions that I didn't really need. Eventually figured out they were open and that just navigating away from a session didn't shut that session down. I went back in and manually stopped them.

Things that resonate with me. Running on a server in the cloud - including instances that are substantially more powerful than my laptop Dead easy deployment (being a sysadmin or devops type admin is low on my priority list of things to learn) Interesting demo for me to play with Online collaboration - even though I mostly work solo Encouragement to practice version control with research code Scheduled jobs and automating pipelines

I will certainly keep an eye on how it develops and create a project of my own on there in order to better understand the features, what works and what is missing

Thanks for the hard work to get to this point

I suspect that I will continue to play around with it and post some of my own code

Thanks for the feedback. We're always thinking about how to improve the workflow and will take your comments into account when we next revise. Please ping us on intercom with any other thoughts!

Hey all.

We just opened up Sense to the public. Happy to answer any questions you may have about Sense and building a modern data science platform.

Tristan

Who do you think is your biggest competitor right now? Do you think that you'll be fighting with Microsoft's Azure Machine Learning offering for the same audience ? How do the two platform compare?

SAS is the huge incumbent, but open source tools like R and Spark and cloud-centric platforms are going to win, no question. We're already well into this transition and tools like Sense should accelerate it.

Nice. I like it. I just replied to you too. I couldn't see any error messages when I ran a modified example with errors.

Just signed up to test things out. Really nice interface and great job with having the onboarding process although I did get slightly lost when it came to editing the R file. Will continue to test it out a little bit later.

Excited to see if this takes off - working on a startup where an integration could be beneficial to both sides. Just curious who you feel your target customer is?

Looks interesting. Seems like they're attempting to solve similar problems as Project Jupyter[1] & ScienceBox[2].

[1] http://jupyter.org/ [2] https://yhathq.com/products/sciencebox

Here's an incomplete list of some related web-based coding/data analysis/Python services:

- https://wakari.io/ -- closed source, IPython, freemium/enterprise

- https://cloud.sagemath.com -- open source; more academia oriented; completely free (right now) [disclaimer: I founded this]

- https://c9.io/ -- open source, freemium

- http://runnable.com/ -- closed source, no business model

- https://www.pythonanywhere.com/ -- closed source, freemium

- https://www.terminal.com/ -- closed source; freemium

- https://www.nitrous.io/ -- closed source; freemium

- https://codio.com/ -- closed source; freemium (free for public)

- https://codenvy.com/ -- closed source; freemium/enterprise

- https://koding.com/ -- closed source; freemium

- http://www.wolframcloud.com/ -- closed source; expensive

The interface looks nice. How do you get data into the platform? The field I work in (genomics) usually uses a bunch of flat text/binary files instead of databases. How would this platform interface with that kind of data?

You can upload small to medium files directly into your project, which lives on a distributed filesystem. For larger data, we typically recommend pulling data directly from S3 or production databases.

Freaked out for a second there, because I thought that this was announcing that Hello.is pivoted before I got my Sense Sleep Monitor.

Good Saas, but obviously not graph stylish friendly

This is fantastic!

nice. good work!

sense.io gives... a blank page

Nice work.

Reply on news.ycombinator.com