Show HN: Retool: Excel-like, with higher order primitives | Hacker News Reader

Show HN: Retool: Excel-like, with higher order primitives

retool.in

190 pointsdvdhsu73 comments

Hey HN! I'm one of the creators of Retool, and we're excited (and nervous) to be posting this on HN.

Retool is a visual programming language, like Excel. But instead of cells being the base unit, widgets are. Widgets are prebuilt React views, and you can edit their properties. They can pull data from SQL queries, and other widgets. The page we made is an example of what you can do.

If you're curious how it works, here are the docs: https://docs.retool.in.

Any feedback or questions welcome! :)

Congratulate on the launch! I always think of something like this could solved most business process problems.

Would love to see a client-side model abstract layer(in javascript) to dealing with API stuff.

Are you from the https://www.notion.so team? UI looks eerily familiar.

Ha – I do use Notion a lot, and really love their design. Much of our design is definitely inspired by them – the widget picker, for example.

you may want to uh...branch out a bit. the widget picker in particular is basically identical

.. I guess good artists steal?

Why change something that works, just because it isn't uniquely designed?

Yeah... when i saw the widget picker, my first thought was, 'did they take their css?'

The other thing is, it doesn't really fit. Your usage of beige on white isn't that poppin. Switch it up!

Hmm. The problem we have (and presumably one Notion has, as well) is to somehow show off all our widgets, without it being overwhelming. We previously had them hidden in a submenu under "add widget", but we found that the few people we tried it on were confused and weren't sure what widgets were available.

I remember when I first saw Notion: it was beautiful! And the widget picker was exactly what we needed -- it acts as an index for all the widgets, with a short description for each. So I decided to steal the idea. None of the code here was copied (or even referenced). I think that when there are good design patterns (e.g. tabs), stealing an idea is okay.

In this case, I think that I might've been a bit too lazy, as you noted. I think we should've thought a bit more about the design. Thanks for the feedback! We'll get it changed in our next version. (My excuse is that we only switched to this widget picker overnight, and really really wanted to do a show hn today)

Demonstrating full functionality within an app is a lost art. Go take a look at MS Office that was out around with Windows 3.1. Microsoft Access, for example, had a tutorial included that walked you through everything - from setting up your initial tables all the way to raw queries, aggregate reports and the like. Lotus Improv 2.0 even had a "buddy" (sort of like the much maligned "Clippy" for Office 97) named John or something which would walk you through their software, demonstrating the structure and functionality of it over say VisiCal, Excel, or its own product 1-2-3.[0]

Specifically for your product, you might want to change a few things.

* 1: It's fairly busy when I click the demo. I'm overwhelmed by your "do you need help" real-time chat widget, which overlays your sidebar blocking out some functionality, and container3[2] which has 3 options.

* 2: I shouldn't even be seeing "container 3", as an end-user, or any integer primary keys.

* 3: I'm presuming "container3" is what your users want to interface with first. In which one do "I" pick? The second box of the two is particularly out of place, as the end-user hasn't had time to explore your app, so how could they even have questions or comments?

* 4: The side-bar gets overlapped by your interactive pop-up for help on my laptop for that chat feature, making a busy layout even more busy. Again, I haven't had time to explore your app, so how could I have questions? Do some mouse-movement analysis or something, and use that to determine if they're exploring the app productively or blindly clicking around in confusion. If the latter, then perhaps pop up the chat modal.

* 5: The column names aren't "humanized". 'first_name', 'last_name'. Even worse, "store_id" is a numerical value, rather than the actual store name.

* 6: On the initial load, there is no customer pre-populated. So not only does it have no content in those containers, but there are some display components which are clearly not supposed to be there[1]. (The more conventional way would be to have a details modal pop-up when you do select a user.)

* 7: Overall, the interface is unconventional and awkward for a standard user coming from any 'data-driven' app they'd use at their day job. I.e., you have both an overview table and then a details table below, which drills-down into two different detail levels (an overview of the customer, then their specific rental details). As a store manager, I probably want to see either the customers summary information (net revenue per month) or a list of specific titles they rented. Not both at the same time. As it stands you have 3 different levels of data being presented at once.

* Edit: Speak of the devil. I've never used Notion before, but I signed up for it and it has a "Clippy" right there. I guess they probably grew up with the same software I did haha. Their product isn't particularly powerful or groundbreaking (compared to say, what Asana was at the time) but I'm guessing that I'm not their target audience (seems like "yet-another-Trello/Evernote hybrid-with-an-Electron-app").

I'm guessing their target demographic is "John working sales at Dunder-Mufflin paper" who needs a central repository of his customers and purchasing habits ('Sally's Flowers is about to run out of stock', I should give her a call). Or John's manager, Tina, who wants to see aggregate statistics on how her team is doing with that new re-targeting email campaign software, she used 20% of her budget on. Are her 6 sales guys producing enough of a sales increase that the monthly fees are generating sufficient ROI?

I probably wouldn't pay for their software, but again I'm an emacs power user who's written thousands of lines of elisp on top of org-mode to centralize all of my information/aggregate analysis. Regardless, their interfacing tutorial animated helper is a prime case study of 'how to introduce someone to your product' without overwhelming them. But if they're your direct market competitor, it'd be quite questionable to "borrow" that flow. You're on tenuous ground as it is with your layout + widgets.

I don't use web apps out of principle if I can't at least pay to purchase a license to host it locally (Atlassian style), as I've been burned by tons of acquire-hire-kills. As such, I'm not sure what the ecosystem is like and if "Ivan" type animated helpers are commonplace. If they aren't, HN users take note -- this is a prime example of how to introduce your new users to your software.

---

[0] Both are pretty much 'abandonware' at this point, so I have no problem telling you to go pirate it.

[1] https://i.imgur.com/tGyEVSi.png

[2] https://i.imgur.com/FvCLBQP.png

Prepare to be acquired

How can I download the docs for offline reading?

Sure! Email me; I’ll send you a PDF, ePub, HTML, or something. Thanks for your interest!

It looks good! Where can I download it?

Tried dragging a header in. Tried deleting it by pressing "del" after selecting it, didn't work. Tried dragging it off-screen to delete it, didn't work. Then I noticed the welcome-message chatbox on the bottom right was hiding a big red button ;)

I can imagine keyboard support wasn't the highest priority in getting all of this stuff to work though, so not complaining, but I think it would make this a lot more appealing to power-users.

it is really good tool. we have tons of spreadsheets at work. instead of building automatic solution, we are looking for quick way to recreate them. we built a tool to attach function to extend data pulled from db. It would be good to pipe data between different components which means each component has I/O ports. very inspiring works

I wish things like this would start of as an open-source and/or stand-alone app. It looks great, and I want to play with it. But Excel is at it's core used for very sensitive data. Financials, budgets, customer lists, biz plans, etc.

How and why should I trust a brand-new startup? Doubly so when they don't even list security in their docs.

tl;dr; I love the idea, looks like a great implementation, please let me demo (and then ideally purchase) an on-prem version.

Thanks! I agree -- we've literally been writing the docs all throughout the night, and don't expect this to be used by any customers yet :)

Security is definitely a concern when we have access to your data. In the coming week, we'll definitely open-source the backend and allow it to be deployed via Docker. We're also thinking about doing a complete on-prem solution -- email me if you're interested! (email in HN profile)

Thanks for the feedback!

+ write privacy / security policies to guarantee we never see nor touch your data; coming soon

You can't just 'promise' you'll never touch someone's data. You have to make it impossible in the design of the technology

For companies, it's often enough if you're legally bound to not touch data, or else cough up a sufficiently high compensation.

That will be the case with our on-prem version. The backend is open sourced, so you can audit the code yourself. You still use our frontend, but you can be sure we won’t be stealing your data, since you can just open DevTools and see whether we’re making dodgy network requests to our server.

Finally! I was wondering when someone would finally implement something like this. It makes so much more sense to have a proper database to contain the data and a notebook like interface instead of shitty cells to display it. For me spreadsheets are the biggest UI fail in computer history.

> For me spreadsheets are the biggest UI fail in computer history.

I can't wrap my head around this perspective. To me, they are one of the biggest UI successes in computing history.

Spreadsheets provide an extremely approachable interface for managing data. A 2D grid is easy for people to understand. Entering data is easy: you click a cell, then type. And that is literally all you need to know in order to get some use out of spreadsheet software -- you're just using it as a dumb tabular data store, but it turns out that there are lots and lots of common scenarios where a dumb tabular data store is all you need.

That's where lots of people stop learning about spreadsheets, of course. But that's fine, because the software is still providing utility for those people. And for the fraction of those people who are inquisitive enough to want the software to do more, they can start extending the basic metaphors they learned coming in the door with additional power -- sorting, filtering, functions -- without having to learn new interfaces at each step along the way. Then some subset of those people will discover that there's actually a scripting language under the hood, which gives them even more power.

If you wanted to design an interface that provided the widest possible funnel for picking up new users, while simultaneously letting those users power up to new levels of capability as seamlessly as possible, you would have to work really, really hard to come up with a better solution than the spreadsheet.

Imho, the problem is Excel has so many common-law bugs and boneheaded workflows they're stuck with for historical reasons that the state of the art can't improve. The whole "add a row on the bottom and find that none of the styles apply or formulas have extended that far" is moronic. Not to mention the boneheaded behavior when you paste anything that looks like it could be data (even if it is a date, but in a slightly diffferent format) and it just gets perma-mangled.

Excel is close to good, but it can never be actually good because it's got decades of workflow to protect.

> add a row on the bottom and find that none of the styles apply or formulas have extended that far

Tables (or lists in previous versions) solve that issue nicely.

Really? I'd heard that "Format as table" made that happen, but I tried it recently and the formulas and constraints didn't continue as I added rows to the bottom. Is there some secondary thing also called "table" that I should be using?

Create Table is a different command than Format as Table. The former creates a data table, the latter format cells to provide the appearance of a table.

Oh. Well that's perfectly clear.

Format as Table turn the area into a Table also.

"When you have a data range that is not formatted as a table, Excel will automatically convert it to a table when you select a table style."

https://support.office.com/en-us/article/Format-an-Excel-tab...

As others said - you need to create a table instead. Terrible naming but awesome functionality. It's the best hidden secret of excel. It also has the side effect of naming the whole range so you can easily lookup() in just that scope. (Or in general reference values without an offset on the page and with proper column names)

https://support.office.com/en-us/article/Use-calculated-colu...

Have you seen the Joel Spolsky video You Suck at Excel [0]? He demonstrates how using tables can avoid many of the common pitfalls.

[0] https://www.youtube.com/watch?v=0nbkaYsR94c

Data tables are a huge win where they apply... they can result in significant performance improvements in addition to all the other benefits. (A former co-worker of mine once wrote a simulation using Excel, and switching to tables give the system several orders of magnitude better performance.)

I work in finance and Excel is de rigueur. I work at a company where IT skills are exceptionally low. Excel allows people easy access to data even when they don't know the web 2.0 gui style like a millennial. Having been forced to become 'advanced' at Excel, it is an extremely capable tool. It only falls down when you get hundreds of thousands of records or you need concurrency. At this point I find that if you have the skills to conceptualize the data, then you could also learn SQL...

"It only falls down when you get hundreds of thousands of records"

If Excel is falling down for you with only that many rows, it's possible that you're using one or more 'volatile' Excel functions (ones that are recalculated whenever any cell in the sheet changes, e.g. OFFSET() or INDIRECT()).

I agree that certain functions can cause problems. Actually the biggest culprit in my kind of work is VLOOKUP(), which I know could be replaced with INDEX()/MATCH() but less conveniently.

I find that filling down with a function like VLOOKUP(), whilst you have multi filters on, is the real killer. And yes you can rethink your workflow so this is not necessary, but if you have to do that you have immediately reduced the utility of Excel, which is easy manipulation. If you have to 'paste special' your subset into a new sheet first, then it means you can't easily test different filters or scenarios. If I have to do something more complex to get my answers, then I would begin my manipulations in sql instead.

I still find that once Excel gets anywhere near its row limit (circa 1M), there is a big increase in memory usage, which I have found can cause a big slow down on ordinary desktops that normal office workers have (i3/i5 4GB ram). Curiously (and without extensive testing) I feel this is much worse in later versions of Excel (after 2007 and the BIG change), and my Excel colleagues agree.

It sounds like your workflow is quite different from mine. Specifically:

1) As you say, you can use INDEX/MATCH instead of VLOOKUP. I only choose the latter if I'm 100% sure I'm not going to save the sheet. I don't know whether VLOOKUP has performance problems, but at some point someone's going to insert a column and break the references.

2) I've never had a filter (I presume you mean filtering on the content of columns by clicking on the column headers) which affects the value of a formula. I tend to have one-way data flow: inputs are clearly marked (ideally, but not always, on their own sheet(s)), and you use filtering only on output sheets.

I'm not sure about RAM usage. I use Excel much less these days for things with that many rows.

I wouldn't go as far as to say spreadsheets are the biggest fail in computer history. In fact, I would say that they were pretty revolutionary for its time. But now that it's not 2001 anymore, I definitely get frustrated by some of the things we still use Excel for in 2017. I work in engineering, so my needs are a little different. I would much prefer Matlab for managing sets of data, except for its shortcomings with handling non-numerical data.

"I definitely get frustrated by some of the things we still use Excel for in 2017."

That's a testament to how bad _other_ solutions are. For most people, the UX for Excel is far superior to that of SQL, matplotlib, ggplot2, sed/awk, CRM tools, ...

> I definitely get frustrated by some of the things we still use Excel for in 2017.

Excel suffers a lot due to the fact that its core computational model has been static for decades. While Microsoft has been good at adding commands that manipulate the grid, it's not been nearly as good at enhancing what the grid itself is capable of computing. Just to illustrate, I'm continually annoyed that 'sort' and 'histogram' are menu commands and not concepts that can be expressed with an array formula. (Although it could theoretically be done with a UDF written in VBA.)

But that specific example aside, my frustration is really that the Excel expression language is the beginnings of a good dynamic functional programming language, but is missing most of the good stuff from that paradigm. (Non-scalar values, local bindings, higher order functions, custom functions aside from UDFs, etc.)

What's maybe most frustrating about this is that Microsoft Research (Simon Peyton-Jones) has done several talks suggesting how it might be done. Microsoft is definitely aware of the possibilities... they've just chosen to direct their priorities elsewhere.

https://www.slideshare.net/kfrdbs/peyton-jones

https://www.microsoft.com/en-us/research/publication/improvi...

Neat. One issue with the demo is that the initial impression makes the tool seem very sluggish, when in reality the sluggishness apparently comes from fetching data dynamically over the network. While demoing that capability is nice, I would default to using local data.

One thing that I noticed while paying attention to the sluggishness was that customer information view showed the previous data until new data arrived when selecting another customer. I would consider that fairly bad thing, especially as there are no indicators that the data is not actually valid.

I think a natural extension to this system would be to add a form builder or something that adds the ability to modify/submit data. Combined with something like AWS Lambda (and other AWS services) you could build quite powerful "applications" with relative ease.

Thanks for the feedback!

- local data -- good point -- will switch that up so the demo goes faster in future

- no loading indicators -- thanks! I think I've used the tool so much I've now gotten used to it, but this is definitely something we'll add in our next release

- form building -- interesting. We were thinking about doing API queries to modify data, but that would require backend code. Were also thinking about just having a database local to retool -- so you can take notes on your customers, say.

Nice. This could use a 5 min video to show the flow + value-prop...

Hello! Have you had a look at Klipfolio.com ?

We're using it but it's not flexible enough, and the code editing sucks. I think there's definitely a market out there for this thing, so I hope you succeed at what you do, and do the best you can.

Multiple sources is very much needed, and definitely a way to edit all the code in a code editor (i.e. plaintext only, no drag and drop). We have 40 branches and maintain a separate dashboard for all 40 branches. Having to edit manually all 40 of them sucks. I want to copy the code for 1 dashboard and do find+replace - I don't want to use the drag and drop and so on.

Some comments, wish you all the best

Thanks for the comments!

I think editing by code is definitely better when you have 40 separate branches. What if your datasource (e.g. postgres instance) was a variable, and you could change this for each branch? Perhaps forking would also be useful, if there were specific charts you needed for each branch?

I think this is something that we're interested in building -- do you mind sending me an email at hn@davidh.su? Would love to chat more. Thanks!

Hi there. We pay hundreds of thousands nowadays to build data collection tools in salesforce (ie forecast that needs to be inserted by salespeople). Your solution is better and it would be cheaper and more flexible. I have always been thinking that embedding a tool like yours is probably the best way to go in sales force (rather than coding things out).

We will embed Tableau for the analytics, so why not embedding a tool for data input?

Integration with different data sources will be key though.

Hi. Thanks for the message! That’s a prime use case we were thinking about. What’s your email? I’d love to chat more – mine is in my profile.

I agree that data integrations are key. I think that postgres is a start, but for a tool like this to be widely useful, we have to be able to consume and write to various APIs, in a way nontechnical people will be able to understand.

What's the performance like for large datasets, is there a timeout on query time, or a caching scheme behind the scenes?

Also, this looks similar in functionality to ChartIO and Mode, do you have a summary of how this is differentiated?

Edit: I wanted to add: great work!

Could you provide a link to Mode? I'm not familiar with it and the name is epically un-googleable.

Fast. We use connect directly to your SQL database (mongo coming soon) so that's the perf bottleneck. Currently no query timeout -- if you host it yourself, the server is also async, so timeouts won't be an issue. Caching behind the scenes is something we'll build over the next month.

I think that Chartio and Mode are very good answering questions, and exploring data. If your problem is "what is our CAGR since 2015", Chartio is good for that. Mode, I think, is more similar to us, and certainly better for data analysis. I think, however, that we are more customizable than Mode -- imagine writing a search box in Mode, for example, in order to drill in deep on a certain customer's orders.

I think we're more about building apps that would be rote coding otherwise (e.g. building a CRUD app to manage orders), or empowering non-engineers to build and maintain these apps themselves. Instead of hiring more engineers, what if you made everybody in your company able to quickly build simple software? I think Excel has been very empowering, and we want to do the same.

This is almost ground-breaking. Can you imagine the power of this thing coupled with AI widgets? With Zapier-esque connections to APIs.

Great UI and concept. Can't wait to use this if I can find time.

So this is a web version of Lotus Notes, right? :).

Nice work. I really like the drag & drop stuff which snaps to a grid. Awesome!

That's amazing, amazing. I love it and can't discover how to use it.

So this is basically just a way to show data from your database in custom widgets, right? It doesn't allow you to update the database in any way, correct?

"Build tools in minutes, not days." is not clear. What is a "tool"? To build a "tool" using a "widget" is a perfect nonspecific thing to do. Also, what is Launchaco?

Still, I like it a lot, seems to be a much better solution than all those KPI dashboards and also to people using https://www.blockspring.com/ to get database data into spreadsheets.

As of now, no, but we're probably going to release buttons that trigger API calls in the upcoming week. So you'll be able to reset passwords / clear redis caches / anything else you want, since you'll be able to hit arbitrary API endpoints.

EDIT: good feedback on the messaging -- thanks! launchaco is a static landing page generator I just used (after posting this) to generate a landing page -- working through kinks right now! (I noticed a lot of people were landing in the app, clicking around, and bouncing, so maybe a landing page would be a better way of explaining things)

Thanks! Got data you want to explore in your database but can't be bothered to spin up some React boilerplate? Use us instead!

Our goal to let ordinary people build the tools they want, much like Excel has. To that end, I think we're missing these features:

- input and store data, instead of connecting to a postgres db - build wrappers around APIs so you can pull in / write data to/from anywhere - make it as non-technical as possible, but building a GUI query editor, and possibly changing the widget templating syntax

Nice, I can almost copy and paste my most recent comment again:

Warning, plays a sound to make you notice a fake chat message in the corner. That's scammy.

Intercom is a pretty well known support/engagement/communication tool. How is this scammy?

Not sure I'd call it scammy, but I've been increasingly annoyed with the number of things I need to swat away or make an active effort to ignore in order to spend an uninterrupted 15 seconds skimming over a page to see if it's worth more of my time.

Fair point :)

I can't argue much, I use Ghostery and also this host list which means I usually don't see those things: https://github.com/StevenBlack/hosts

A bit off-topic, but you might want to check this out if you plan on continuing to use Ghostery: https://en.wikipedia.org/wiki/Ghostery#Criticism

As mentioned I supplement with a host blocker.

I feel Ghostery has been very transparent about its use of data and has been clear about what I opt into and out of with any given update.

Good looking out though!

It fakes a message meant for me when there is none. It does so with an invasive sound.

I'd say that presenting a fake chat interface with a fake message is a bit dishonest, but then again it's so common nowadays that I barely even notice it (and by the time my brain processes what happened, my hand is half-way through moving the mouse pointer over the "Close" button).

Hey -- sorry you were bugged by Intercom. I used it at a previous startup and it was pretty useful for communicating with users in real time. I'll see if I can make it not play the sound on page open!

Also, it is covering the most important commands of the interface with its round logo.

I agree. I know it's Intercom and that everybody loves it, but I find it very very spammy. I'm not even mentioning that they slow the page a lot, but the sound, and the insistence on opening chat windows with idiotic default content even if I'm visiting the page for the 100th time, that is horrible.

Intercom definitely needs to die ASAP. But for now, apparently you can block it by blacklisting js.intercomcdn.com in your adblocker.

It may just me be, but from the title I was expecting Retool to be something for Excel, not like Excel. Perhaps a better title would be "Show HN: Retool: spreadsheets with higher-order primitives"?

OK, we affixed "like" to "Excel" above.

Edit: we also changed the URL from https://demo.retool.in/demo by submitter's request.

I don't know is there any benefit in using Retool, when compared to https://www.nubuilder.net

completely strange this "app", I barely could try the demo, it supposed to work ?

Reply on news.ycombinator.com