Ask PG: Is this what you meant by "web-based Excel/database hybrid"? | Hacker News Reader

Ask PG: Is this what you meant by "web-based Excel/database hybrid"?

sheetjs.com

75 pointssheetjs82 comments

As far as I can tell from a quick look, this isn't what I had in mind. I'd expect to see a grid I could start typing into, not a page of text.

I was thinking of something for ordinary end users. So anything that tells users "This particular parser assumes that the first row of every table is a header row" is ipso facto not what I had in mind, because ordinary end users don't know what a parser is. Or a header row for that matter.

While you are here: you've funded a few startups in the space, including skysheet (http://www.skysheet.com/) and fivetran (http://fivetran.com/). Are they targeting this "hybrid" or are you thinking of something completely different?

Neither of them are. Grid (http://www.binarythumb.com) is the closest thing to it.

We (Fivetran) connect to databases but don't support editing, our product is for analysis. Our users data comes from production SQL databases and cloud services, so editing wouldn't really make sense.

I had never seen your "web-based Excel/database hybrid" suggestion before, but I'm curious if you believe it should be grid driven at all?

Our approach is more page (or in terms of Access: form) driven.

This is how an individual entry might look like: http://ycindex.silk.co/page/Dropbox

Here's how it can look in a grid/table view: http://ycindex.silk.co/explore/table/collection/startup/colu...

Or in groups: http://ycindex.silk.co/explore/groupby/collection/startup/gr...

It's hard to predict the future, but it seems grids are very basic and powerful.

http://datazenit.com/ is a web based database tool which has data grid, similar to the one found in Excel. You can check screenshots on the homepage.

It's pretty awsome.

During the PC boom in the 90s many of the hot products were productivity apps that allowed non-programmers to do "computing" (for lack of a better term).

- Access/dBase/Filemaker

- Word/WordPerfect

- Excel/Lotus

- Photoshop

- Harvard Graphics (if anyone remembers that one)

- Pagemaker

- Corel Draw/Illustrator

These were the types of apps that many young programmers, myself included, wanted to work on.

After the internet happened effort moved away from productivity apps, and towards communication apps. That's where all the hyper growth was, so that's understandable.

In the meantime, not much has really changed with productivity apps. The big apps from 90s are still the big apps of today. But as browsers have improved significantly in the past couple years, I think there is going to be a resurgence in web based productivity apps. Webflow is a good example. They are basically building a traditional style productivity app in the browser.

I think the problem with web versions of excel-like apps is the lack of trust when it comes to hosting your private information on the sites that would run them.

Do you really want to have your expense/profit reports, customer lists, etc. hosted on some app?

I don't even use Google spreadsheet because of this sole reason and I don't plan to switch. I would like to switch but yeah, I just don't trust private financial data in the hands of any service provider.

This is a good point. Beyond the privacy concerns, there's the problem of not having any control over when the software disappears. One of the better general purpose web-based productivity apps was DabbleDB. I got a friend hooked on it and he invested a bunch of time basing many of his companies internal processes on it. They were subsequently acquired by Twitter and shut down the service.

We've tried to build this concept when we've built mysimplegrid. http://www.mysimplegrid.com It's very conceptual. Is it closer to what you're thinking of?

I don't know what pg meant, but I know what I'm still looking for: Microsoft Access, available in a cross-platform online way.

And no, things like Openoffice^W Libreoffice Base or Kexi are about 1,000,000 miles away from even being useable, much less good.

Doesn't Salesforce have 'sort of' similar software in the Cloud?

> And no, things like Openoffice^W Libreoffice Base or Kexi are about 1,000,000 miles away from even being useable, much less good.

... which is why we started from the problem of document compatibility (something I really wish GDocs and Numbers for iOS focused on). In the process of building and discovering test cases (repo: https://github.com/SheetJS/test_files) we found and reported bugs in OO, Apache POI, roo, xlrd and other parsers

https://github.com/SheetJS/js-xls and https://github.com/SheetJS/js-xlsx are the parsers underlying this.

Despite the fact I've long moved on from MS Access, I know exactly what you mean. There needs to be an application that anyone can open up and start doing database stuff with, that is cross platform but accessible to both end users who want to explore through clicks (query builders) but also those who want the power to script it (SQL and VBA).

Access really delivered on that count. If it wasn't for the two factors of relatively poor performance (terrible performance when run "across" a network) and lack of cross-platform behaviour, it would be my go-to tool.

I think there really is demand for a cross platform database gui that sits on top of a serverless database file. If this file can be uploaded to a website that also enables excel-like data manipulation and then downloaded/shared back as a file for manipulation by this gui tool then even better.

If this tool is powerful enough to realise that it shouldn't try to execute across a network and somehow flip over to a "remote execution" mode where it targets a server rather than a file, then even better.

That's the dream, nothing has ever come close.

Opening an MDB across a network share. Opening large databases for long running simulations was par for the course at a company where I previously worked. One of the first things I did was make sure files were on a local drive before simulating which turned a 6 hour run into a minute long run.

I've never used MS Access before. What is the difference between it and PHPMyAdmin or HeidiSQL?

What specific things made MS Access so easy to pickup and what made it powerful?

For anyone else that is looking for what was described above, I'd be pretty happy to work on something new for a change.

If you can please provide me some details (my email is in my profile), that would keep me motivated. My only gap in knowledge is MS Access and what it did well and what it didn't do well.

the form builder. Takes about 30 mins to get a form front end built that ordinary users can use by double clicking. You can customize quite like crazy with VB script.

My experience was 8 years ago, its just so easy to hack something on a whim to solve some office task that's annoying (for internal use only, the security was a joke).

The graphical database designer tool was perhaps useful for non-techies too. And the forms could be extensively customized with VB script if it didn't do what you wanted out the tin (change this color when X happens etc).

Excel, and Access specifically were unique in that they helped groups validate their need for software. By the time they had given in and tried to build something themselves, be it in excel, or Access, it allowed the logical extension of 'here's the data we need' to 'here's how we need the data to connect and interact'.

I think anyone who inherited an access system saw the progression from taking an excel like interpretation, to putting forms on front of it, to trying to connect the forms, to ultimately print some form of report. It was CRUD within one app, that didn't require learning 37 different technologies to make one screen work.

At most, you might have to pick up VBA if you wanted to do something fancier.

As much as Access makes me cringe, seeing the apps that were reasonably well architected and talked to MSSQL backends didn't look too out of place.

I've not used either of those tools, but here's some stuff that access does that I often find lacking in other tools:

1) You can copy a grid from excel, do a "paste as data" and access will automagically create a table, with the right data types and even pick up column names if headers are present.

2) You can view a table as a grid, similar to excel, and just start typing away to create data. You can do this before you even create the table to just start creating a table.

3) The drag / drop query builder is extremely easy, especially for things like cross tab queries. You can click "view as SQL" to then turn it into SQL if you want to tweak stuff. (I actually learned SQL this way)

4) foreign keys are made through a drag-drop interface "relationships"

5) The form wizard makes making forms really easy.

6) You can right click a row and delete the row. You can right click a column and delete the column. Any error messages are typically understandable and not cryptic.

7) All your data is in a file. A file which you know where it is, and you know you can click that file to open and see your tables and forms.

8) You can save queries which act almost like views / CTEs and in most queries you get a grid which you can just start editing data in if you so wish.

If anyone is unfamiliar with access I'd strongly recommending getting hold of a copy to play around with, it's how to do database right for end-users.

I never found the GUI query tools to be particulary intuitive once you moved beyond single-table queries. GUI or no, you still need to understand basic relational database concepts to be able to set up multi-table joins in queries that actually produce the results you want.

In access an expert can set up relationships which will cause them to be automatically joined when a less expert user adds both tables to a query.

Some knowledge is usually required to make sense of the results though but I'd say a non-expert still has more chance with access than any other tool.

I learned about relational databases just from playing with Access at one of my first (boring, nontechnical) jobs. Just the fact that the GUI gives you a valid query for multi-table joins, even if it's not the one you want, is extraordinarily useful if you don't know SQL and want to poke around.

The other responses are on point, but I'd add that Access made it super-easy to share database apps, since they were just a file (or files). No servers, etc. Just email it over, or drop it on the share. For the typical Office user, using such a file didn't involve installing any software or setting up any servers. Distribution was super-easy with Access.

PHPMyAdmin is too low-level for what I'd want. For myself, it's fine, but I can't just hand it over to users for querying, updating, adding records, etc. With Access or Filemaker it's easy to set up something that I can hand over to nontechnical users. When I want something web based, I always seem to end up having to code an interface for it myself, which feels like a waste of time.

The GUI - it looks like and works like souped up excel. If this is something you're working on, buy a copy of Access and play with it for a few hours.

PHPMyAdmin and HeidiSQL are administrative tools for managing your database. MS Access was an application development environment and database combined. Access was full of GUI's and "wizards" (the interfaces where you answer some questions, click next, answer some more questions, repeat until finished).

The result was a tool that power users (users who are very good with a computer, but not a full fledged programmer) could use to build some pretty amazing stuff.

It has been mentioned elsewhere in this thread, but one of the amazing things about Access was the way it could figure out what you wanted to do through a wizard. Provided your tables and relationships were set up correctly, you could do some powerful stuff. For example, if you have two tables with a 1:many relationship, you can display a form with a sub-form where the data is linked. Navigating between records in the parent form would automatically load data in the sub-form.

It's a product that you really should use a little if you're really interested in a project. Despite its warts, Access was a really incredible and ambitious product.

SQLite backend and <Frontend in Any Language>. What's the big deal?

Access is the frontend

Ah yes, Code: It's Trivial[1]

Jeff Atwood quoting Benjamin Pollack

" Developers think cloning a site like StackOverflow is easy for the same reason that open-source software remains such a horrible pain in the ass to use. When you put a developer in front of StackOverflow, they don't really see StackOverflow. What they actually see is this:

  create table QUESTION (ID identity primary key,
  TITLE varchar(255),
  BODY text,
  UPVOTES integer not null default 0,
  DOWNVOTES integer not null default 0,
  USER integer references USER(ID));
  create table RESPONSE (ID identity primary key,
  BODY text,
  UPVOTES integer not null default 0,
  DOWNVOTES integer not null default 0,
  QUESTION integer references QUESTION(ID))

[1] http://blog.codinghorror.com/code-its-trivial/ Appropriately enough the blog is complaining about the attitude being displayed on Hacker News

The big deal is that in an office you tend to have access to the following only:

* Microsoft Office (slightly out of date version) * Internet Explorer 8 or less * Adobe Reader * Crappy SharePoint installation

If someone can find a way to replace Access that doesn't involve going through corporate IT's 50-year long approval/installation process, it'll be a massive hit.

Or find a job at a more agile company.

That's beside the point. Yes, if you don't want to deal with Access then you can go and find a job at a smaller, more agile company.

However, there are always going to be a lot of people who do work in vast, stuck-in-their-ways corporations - banks, energy companies and other MNCs.

An Access replacement would be aimed at these people, not a specific individual who could just 'find a job at a more agile company'. As a result, it could be very successful.

That's called Monopoly. It has nothing to do with Access. OpenSource Alternatives exist since Star Office days so it can't be said some alternative of Access don't exist

Uh huh. And the important part of Access is the front-end. And no one from the open source or hacker community has even come close, not even in the same ballpark, not in 20 years now. Twenty years!

So clearly there is, in fact, a big deal. If you think you can write Access in a weekend, feel free, post it here when you're done. Good luck!

Do the Forms from Google Docs come anywhere close? (Serious question having not really used either product much.)

No. Not even close.

The product I'm working on, Silk [1], is trying to do basically reinvent Access/FileMaker for the web age.

I'm not sure if it would do what you'd want since we're not trying to replicate many of the power features of Access, but would love to hear your feedback.

[1] http://www.silk.co/

Neat, thanks for sharing, I'll check it out :)

Phprunner.com is another one that is a little like this genre.

Why don't you use QuickBase or Zoho creator? ( http://quickbase.intuit.com/ or https://www.zoho.com/creator/ )

http://apenwarr.ca/log/?m=201203

That's excellent. And I think it shows that there is a large and, I was going to write untapped but that's obviously not true, tapped-by-exactly-one-company market to replicate Access in an online, cross-platform way.

Something with Wordpress business model would be great. You build a tool to build tools (or a tool to build ERPs). Provide a few samples, ready from the shelf; a way to end users build simple enough systems and a way to software architects build complex and niche ERPs and sell through your marketplace.

The architects would be the WP designers, creating the ERP "themes". Sounds neat to me.

Acces? What about foxpro!

Is a big shame that MS get rid of Visual FoxPro. I still dream of build something like that.

What make VFP far-far-far better than Acces? Powerfull data/dbase language (instead of basic basic), You can code triggers/stored procedures/views, etc with the fox language itself (same for reports, forms, menus, etc= Everything is Fox and fox only).

Good enough to build full complex apps. Decent wizards.

Probable the only good thing that acces have instead of fox is better looking GUIS and better form builder..

Every time somebody starts one of these Google buys it and shuts it down.

My first "engineering" job was building a tool for accessing a huge list (42,000) of interweaving and overlapping requirements for safety at an oil company.

Using Access was amazing. It made it trivial to build a functional, if ugly, UI that was perfect for inside a corporate environment.

The idea of a "modern" Excel/Access is intriguing, but it's not without it's pitfalls. Basically, who uses Excel/Access a lot? Big Corporate America. Can they put their data up anywhere and access it over the web? In some cases, yes, in most cases I've seen, no. So any solution, to really make a dent, has to be usable as a self-hosted app. No quant at a trading desk is going to willy-nilly upload their proprietary models to a startup running on EC2.

We encountered this push back with our software so we had to write an Excel add-in (code-signed, of course) for our corporate customers so they could use our software locally on their data sets without having to leave the comfy confines of their corporate firewall.

With all that said, I'm sure there is some segment of the population who don't mind their data up in the cloud if the UI on top comes close to Access. You just gotta find that segment.

One advantage of an in-browser tool is that the hosting can be anywhere, from the cloud to your local computer.

I set up a simple nodejs server (using the module `j` http://npm.im/j) that exposes all of the XLSB files in a directory on a computer and generates Numbers-compatible CSV output so I can read the data on iPhone when I need to. The data is hosted locally, and unlike with saving CSVs the output is always up to date

What's your product called?

https://www.repustate.com/sentiment-analysis-excel-plugin/

whoa that is seriously awesome tool! how do you build such excel plugins? have you also thought about listing it on the microsoft office store?

Unlikely. This still requires you to create all your data offline. Likely what pg is referring to (because I know I and many others wish they had it) is an easy to use browser based MS Access replacement that actually works, and also can quickly report out to a web-based MS Excel replacement so that you can build formulas they way you're used to, not learn SQL. Something like DabbleDB.

I'll be the Pollyanna here and say: maybe! Probably not this iteration, but maybe!

Features I'd suggest: * Some explanation, in a big font, of what we are seeing on the homepage of sheetjs.com ("SheetJS lets you get more information out of your data, instantly") * A big button that takes me to a sample sheet, with some sample queries I can run. Basically, a live tutorial

Good luck, this looks cool!

Fivetran is probably the closest to it. They're also YC (W13) which makes sense. https://www.fivetran.com/

Tiling window manager users: What do you do when you see "drop a file here"? I usually just move on but I wanted to try this. I am staring at awesome right now trying to think-of/find a graphical file manager that does not have a thousand dependencies.

If few dependencies are your thing, you could try rox? http://rox.sourceforge.net/desktop/ROX-Filer

My package manager tells me it depends on sh, libsm and gtk2. That's not much.

I actually used to use roxterm3. When I mentioned dependencies I should have said "does not depend on kde-base|gnome-core|xfce.

I split my screen between `nautilus` and `firefox` and drop the file from one to the other. What aspect particular to tiling window managers gives you difficulty?

Awesome/Xmonad/i3 do not come with a GUI file manager (for obvious reasons). I don't have gnome/kde/xfce installed and point and click file management is not going to be the straw that broke the camels back. The only thing that comes close is rkward, but it relies on kde-base which is the most obese of the three big environments.

I think he probably meant something closer to DabbleDB, which shut down a couple of years ago :( Still a great idea.

Dabble DB was amazing. I only discovered it a week before it was aqui-shutdown.

Edit: I'm trying to access the demo video on archive.org and was able to get into the help area for anyone interested in the details of how it worked:

http://web.archive.org/web/20090430200143/http://dabbledb.co...

Here is a video on Youtube. I don't know if it is the one you were looking for.

https://www.youtube.com/watch?v=MCVj5RZOqwY

This is slightly off the mark as you can't edit data with this tool, but this past year I worked on building a tool for manipulating data from an rdbms in the browser using only clicks. It was an effort to get beyond traditional enterprise reporting frameworks, but more importantly it was my "learn Clojure by building something kinda useful" project. I have it hosted on Heroku here: http://efreportsdemo.herokuapp.com/

Since I mostly use this project for resume fodder, it only uses one dyno and could probably stand to be a bit more secure (my postgres db should probably be a "follower"). Tangentially, it really wouldn't be very difficult to get this to support multiple datasources - some even csv file based.

I'm a huge proponent of IndexedDB, so I groan every time I see a new app developed using WebSQL. From the WebSQL spec:

> Beware. This specification is no longer in active maintenance and the Web Applications Working Group does not intend to maintain it further.

WebSQL isn't going anywhere, and it's the only good client-side database on iOS, but without IE support and a spec-based future I'm not sure why folks continue to put good development time into this API.

There is a version using SQLite (compiled to JS using emscripten): http://sheetjs.com/sexql/sqljs.html

However, the source is about 2MB, so I thought it was preferable to use WebSQL on iPhone (and fall back to SQL.js when necessary)

You mean something like www.silk.co? Full disclosure - I work there. It turns CSVs into a web-based CSV/DB hybrid. Row = page. Column header = key. Column content = value. Collection = Worksheet. You can link worksheets and pivot the views. Also has basic visualizations - pie, line, bar, table, grid maps.

surely not? this is like google docs with no design or features.

And note that Google Sheets actually has a query() function that allows you to run SQL like queries on your datasets. Few people seem to know this, but for me it's one of the features where I prefer it to Excel.

I didn't, that's really useful!

Sharing the documentation links for the benefit of others, since it's not easily found:

Documentation for the query() function: https://support.google.com/drive/answer/3093343

Query Language Reference: https://developers.google.com/chart/interactive/docs/queryla...

You can run queries over data in Excel from inside Excel - using the Get External Data option then selecting the spreadsheet you are working on as the source.

However, while investigating this I found that Excel gets quite upset if you import the data from your query into the same location as the source...... :-)

I was always wondering if FileMaker could find some sort of role like this (if it's even still alive)?

It's still very much alive. IIRC, a lot of universities / K-12 schools and non profits use FileMaker in some manner. I've supported a high number of users in the past and non-technical people surprisingly adapted well managing to be productive with it. The iPhone + iPad apps were decently usable upon launch and hopefully they've gotten better over the last couple of years. I prefer FM over Access and even Excel as far as small tasks like data matching / cleanup / etc. are concerned where a development effort isn't warranted.

Why can't I use a csv file?

The thing that is most valuable for exploring data for us is pivot tables. Both Excel and LibreOffice are single threaded and take forever once you have a few hundred thousand rows.

you should add yourself to the list at www.datapad.io and then get in touch on twitter using @luckymethod

I want a vim/excel/database (and sed/excel/database) hybrid more than I want a web based one. I've no trouble accepting that I'm weird, though...

I think what you want is something like Microsoft Access on the web, only less crappy. I think its been tried many times and never done quite right.

I think this is a really cool tool! I'd love to be able to select columns/rows in the results so I can copy/paste them elsewhere.

So I can use this to work with an Excel file that I already have on my computer. Why would I not just open that file in Excel?

SmartSheet is pretty much an excel/database hybrid and is pretty popular.

doubtful, the site is pathetically slow.

I'm glad the site is still up.

The last demo was hosted on Github: https://news.ycombinator.com/item?id=6722197 https://github.com/SheetJS/sheetjs.github.io

... and we managed to kill github pages: https://status.github.com/messages/2013-11-13

So I'm very happy that the hosting hasn't died yet

what was the original statement made by PG?

so if anyone can help me understand what PG meant and what it is that people would like to see with the "modern" Access/Excel version, I might be able to create a quick prototype and get your feedbacks. Please email me and I'd love to send you an alpha version.

Basically, I'm missing what MS Access was like as I've never encountered it in my career path as a developer. If I could learn what made MS Access so lovable (or unlikable) as a pillar to shape the prototype from what I understand will need cross-platform, local, optional cloud upload (I totally get that you wouldn't want to upload your company's sensitive spreadsheet), and some of the basic functionality of a database management tool that are commonly used.

http://ycombinator.com/ideas.html

22. A web-based Excel/database hybrid. People often use Excel as a lightweight database. I suspect there's an opportunity to create the program such users wish existed, and that there are new things you could do if it were web-based. Like make it easier to get data into it, through forms or scraping.

Don't make it feel like a database. That frightens people. The question to ask is: how much can I let people do without defining structure? You want the database equivalent of a language that makes its easy to keep data in linked lists. (Which means you probably want to write it in one.)

Reply on news.ycombinator.com