Table-to-JSON | Hacker News Reader

Table-to-JSON

lightswitch05.github.io

64 pointstalhof832 comments

I'm having a hard time understanding why everyone's jazzed about this. When is this a useful thing to do? I've literally never once encountered a problem where this would be useful.

I'm not trying to sound harsh, I just don't understand when I'd ever need to do something like this.

This library is like a building block to e.g. building your own interactive table functionality. Let's say you have an HTML table, and you want to allow users to dynamically search or sort the table. Step 1 could be to translate the table to JSON, step 2 would be do all your JS magic on the JSON data set, and step 3 would be write it back to the table. This approach is cleaner than trying to do a bunch of logic on the DOM text. So, this plugin is step 1 done for you.

I suppose that's my issue. Dynamically searching or sorting wouldn't require converting to JSON, I'd rather work with the existing table object:

tableName.rows[rowNum].cells[cellNum]

Instead of loading all of that into a variable (a second time, it's already an object on your page), trying to keep it in sync, binding events in both directions to keep it in sync, the load of doing twice the work, the lag of redrawing the entire table every time you want to do something, etc..

I don't see why this is cleaner than working on the DOM. I've done this a number of times and only found headaches when trying to build something larger than a silly toy or a list of 30 items on a table. IIRC DataTables did something like this internally last time I looked at it, and that thing konks out and either slows down your page or crashes the browser depending on how many cells you load into it.

I think the issue is that, if you are doing something like sorting, you actually are going to have to redraw the entire table.

Also, for things like plain-text searching, it's faster to do it on a collection of objects detached from the DOM in JavaScript than to scan the DOM, and especially if you're doing non-text comparisons (like comparing a number with the parseFloat representation of the values in one of the table columns for all rows).

And finally, if you are going to read and then redraw in the DOM, it's best to do it as grouped atomic operations due to the inefficiency making the browser reflow (https://developers.google.com/speed/articles/reflow).

For what it's worth, this is how Dynatable works internally, though I haven't had any issues with it being slow, though it's possible I just haven't used it with large enough data sets yet (I've only used it on tables of probably 1000 rows or less so far).

I had to do a version of this for myself a few weeks back in conjunction with datatables. We actually also use form elements in the table as well, which I had to handle as special cases. We'd then send the JSON back to the server, which would generate an Excel file.

Why not just send back to the server instead? First, sending the JSON back turned out to be faster than rerunning the query. Second, datatables has a full text search option which made things simpler for our case.

(Datatables has an export plugin, but it uses flash and only generates thinly-veiled CSV.)

So there's a use case for something like this.

Have you thought about sending the HTML of that data back instead of computing the JSON encoding? Your server side could actually parse the DOM and possibly preserve things like fonts/styles when loading into excel (I know how giddy managers get when their auto-generated reports have styles).

In our case, we didn't want to send it all back, and we had our own style to enact on the server side. Sending back the HTML is perfectly fine (albeit a little larger) but as this is JSON, you could easily send back the style to the server if you so wanted.

Hey, great question! Well, I don't know why other people might need this plugin, but I needed to gather data from a dynamic user-editable table. Its not all that complicated getting data from an HTML table, but my specific table is dynamic in sense of how many column and rows are available. Also the heading names. I considered keeping track of the contents as the user edited it, but in the end it was just easiest to get the end result rather than keeping track of all the edits. So, thats where table-to-JSON came from. Had no clue so many people would be interested in such a simple plugin.

Nice ! Could be used in a progressive enhancement strategy : render your html then extract the json out of it to feed your MVWhatever (Angularjs, Emberjs, Backbonejs, nameitjs) and let javascript take on for the cool client side stuff.

It's a nice start to something that looks pretty useful. There are some things to consider though:

What if you have headings as the first column in each row? What if you have headings in the first column, and the first row?

Also, what is the use case for data overrides? In my opinion, it completely defeats the purpose when your intention is to take the contents of the table.

The code does not try to be clever:

    var getHeadings = function(table) {
      var firstRow = table.find("tr:first").first();
      return notNull(opts.headings) ? opts.headings : rowValues(firstRow);
    };

The code is also going to fall down if you have things like wide/high cells. But HTML tables are so flexible and so routinely abused, I guess the problem is too hard to be solved robustly with any generality.

You could do what they're suggesting without being too clever. For instance, our dynatable plugin does it this way:

    table.find('thead tr').children('th,td')

But it could easily be something like:

    var headings = table.find('thead th');
    if (!headings.length) headings = table.find('th');
    if (!headings.length) headings = table.find('tr:first td');

The HTML specification's algorithm for determining the applicable headers for a table cell might be useful:

http://www.whatwg.org/specs/web-apps/current-work/multipage/...

This was developed with reference to a corpus of real web tables.

I was actually really excited to read that, thinking I was going to implement it straight away in our plugin. But it seems to require scanning up the table from the current cell, for each cell, to get its header, and it also requires knowing the rendered height of the cell, which I don't fully understand. I'm sure if I actually sat down and wrote it out, it'd make more sense.

At any rate, it seems like trying to implement their algorithm verbatim in javascript would be pretty slow for large tables. Maybe it's better to just use it for inspiration.

Also, their algorithm seems to come at if from the perspective of, "I have a cell, what is its header?" It would be more efficient in reverse, if we could instead answering the question, "I have a header, what are its cells?". What dynatable does now is sort of a hybrid approach (we find the header cells, make a dictionary, then loop through the rows, assigning each cell value to its attribute in the dictionary). The issue is, according to that article, any table cell could potentially be a header cell, not just those in the header row or in th elements.

Your more than welcome to submit a pull request for any new features you'd like me to add/support. As for my code. I wrote it in a day for a specific page. After seeing people with similar needs on stack-overflow I decided to clean it up a bit and make it a plugin, never really spent a whole lot of time on it. I think my filterable plugin deserves more attention this this one does (http://lightswitch05.github.io/filterable)

I've written a plugin that actually does this. I haven't had time to write the documentation yet, but our Dynatable plugin [1] has been in use for almost 2 years now.

It's an alternative to the Datatables plugin. All its logic works on the set of data in JSON, so it first translates an html table into JSON, then vice versa.

The cool thing about this process is, you can write your own translate step to easily work with e.g. JSON from an Ajax request. And you can write your own render step to work with e.g. a styled list instead of a table.

By default, the translate step translates an html table to a JSON object, naming the attributes by the table column headings (underscore format by default, but can be hyphenated, snake-case, or lower-case too).

[1] http://os.alfajango.com/dynatable

EDIT: If anyone is interested, the part of the plugin that essentially does the "table to JSON" part is the getFromTable function here: https://github.com/JangoSteve/jquery-dynatable/blob/master/j...

Other useful things this function does is, determines whether the data is a string, number, boolean, or date (or other user-defined type) to do proper column sorting logic, and much more.

EDIT 2: Also, I can't speak for the author of the table-to-json plugin, but for dynatable, it's useful to have data-override values for the cells for cases like when you have a date in "Jan 1, 2013" format, which is meaningless when doing filter or sorting operations on the JSON; i.e. you could use the data-override to populate the proper date format so JS knows how to process it.

I'm curious -- what do you think the tradeoffs are for Dynatable versus Datatables?

> There are some things to consider though

To add to your list, there is of course that old chestnut of "it doesn't support the full tables spec" (which in all fairness is quite complex).

The non-webdev part of me is rather depressed that we've resorted to storing N typically homogeneous records in a way which results in the serialization of field names N times.

Wouldn't it be nicer to make the default format an object of arrays rather than an array of objects?

In before "but gzip!"

TSV or CSV?

But to do easy and ~beautiful~ work on the objects (and to work on them the way most JS libraries expect data) you'd have to zipper the arrays together anyway.

FYI, I'm not the creator. I just saw it on G+ and thought it's worth sharing.

I am the creator, thanks for posting it on here! I had no idea so many people would have any interest it it, for me, it had a vary narrow use-case

Awesome. Something else that would be nice is to be able to go from JSON to HTML table. There are a few libraries out there that do this but none seem to be updated or recent. Any suggestions would be greatly appreciated.

Nice work!

If you're using node.js/Express, you could do this using Jade: http://stackoverflow.com/questions/5204905/dynamic-html-page...

It might be overkill if you don't need interactive features, but my ATable[1] library takes a 2-dimensional array representation of the data, and turns it into an HTML table. I will add support for an array of JSON objects as the data source soon.

[1] - http://jarwol.com/aTable/

Ha, I've been working on something like this in a general case.

https://github.com/kevinburke/hulk

Cool. This would be cool for web scraping, but probably limited in number of scenarios you would use.

Would have been nice for some Google Finance data I'm scraping in python.

How hard do you think this would be to rewrite in node w/o jquery? As was noted I could see a lot of scrapping potential for this.

Can we go the other way? JSON to Table? That would make this a really useful library for saving table states.

I've had multiple people ask me about that. There are already tools out there that do it, so I didn't really have any motivation to make it

Would be great as a chrome extension

Reply on news.ycombinator.com