[1] https://jsonviewer.codeplex.com/
It is really important to chop up your JSON files into smaller sub-files. This will not only make it easier to backup and read manually but will usually give you a speed boost (can read to and write to more then 1 part of the "db" at a time).
For an arbitrary example, a binary format could have an index table of objects at the start of the file, and then you could perform partial reads to access only the subset of objects you care about. That's something you could do in a text format too, but if the file is edited in a text editor you can't guarantee that the user remembered or bothered to update the index when they added a new object. The parser would effectively not be able to trust the index, and have to parse the entire file. (I suppose you could use CRCs or something to enforce this, but then you'd end up with a very brittle format that people get frustrated when trying to edit.)
Really, the true advantage of a binary format is you generally assume that nobody messed with the data behind your back, so you can have duplicate data (like an index) if you want without worrying that it's out of sync. This pretty much goes hand in hand with the fact that you can't just open it in a text editor and fiddle with stuff.
TLDR: Human-writability and high-performance are arguably mutually exclusive features.
I would rephrase that a bit and say the true advantage is flexibility, as you're not subject to the constraints of textual data.
The integrity of the data is a separate matter, and should be carefully verified rather than trusted implicitly. A huge amount of security vulnerabilities, and program crashes in general, come from errantly assuming that user-supplied data is correct.
It was a silly observation but it is also true at some level. JSON might be easy to read, but reading a 100M json still needs a special editor.
Another funny observation Joe made at some point when a response to him calling JSON out, because "after all, you can see JSON" was that, your eyes cannot see JSON, they see photons bouncing from the screen. You still use an editor or some other translation program to display it and read it. So at some point might as well use binary (thrift, protobufs, sqlite, ...).
or
curl ... | jq . | less
It's super handy. Drag and drop json files into a new tab.
Link: https://chrome.google.com/webstore/detail/json-formatter/bcj...
If not, I think I would just split data into multiple smaller files. Something like the python code below should work, assuming the file can fit in memory. If not, I assume you can find some json lib that can work in streaming mode and then do the same thing.
import json, os
json_input = '''{
"foo": 1,
"bar": 2,
"baz": 3,
"bug": 4,
"thing": 5
}'''
entries_per_group = 2
if not os.path.exists('sub_files'):
os.mkdir('sub_files')
main_d = json.loads(json_input)
iter = main_d.iteritems()
for group_count in range(10 ** 6):
sub_d = {}
try:
for _ in range(entries_per_group):
k, v = iter.next()
sub_d[k] = v
except StopIteration:
break
finally:
json_output = json.dumps(sub_d, indent=2)
with open(os.path.join('sub_files', '{}.json'.format(group_count)), 'w') as f:
f.write(json_output)Here is how it works - keep the network tab ready. When you see the JSON data request, click on the request and hit the "Preview" tab. It gives you data in a collapse / expand format.
Advantages: 1. There is one less plugin that scans all your browsing activity, 2. Slightly extra battery life when just browsing and not developing stuff.
Disadvantage: You need to keep the network tab ready, otherwise you will have to reload the entire page with the network tab open.
EDIT:
My apologies if it wasn't clear. I was talking about the "Developer Tools" option in Chrome, in which there is a "Network" tab. It is available in "Chrome Menu" > "More Tools" > "Developer Tools". Alternatively you can hit Command + Option + I in mac, or some equivalent in Linux / Windows to get there.
I've never really had to handle really large JSON files, as I'm not a big fan of those but for smaller files I tend to be lazy and paste it in jsonlint.com. It's usually just for reference or debugging purposes (like finding the name of a property or some strange value).
cat unformatted.json | python -mjson.tool | vim -
python -mjson.tool < unformatted.json | vim -
< unformatted.json python -mjson.tool | vim -;)
[1] https://github.com/atom/atom/issues/8864 [2] https://github.com/atom/atom/issues/979
Instead, use a tool like Schema Guru (https://github.com/snowplow/schema-guru ; disclaimer: we wrote this at Snowplow) to programmatically extract the JSON Schema (http://json-schema.org/) which represents all JSON instances in the file.
Imagine you have a json structure like this:
{"records" : [ {"name":"joe", ...lots of other fields in each record}, {"name":"fred", ...lots of other fields}, ... thousands of records ]}
Now a lot of people in this thread have mentioned tools for collapsing the json, but the trouble I have with this is that you can't browse the record names without opening each one and looking at all the fields. It would be nice if there was a tool that took a few key identifiers (name, id) and "bubbled them up" so that in a collapsed view you would see something like:
{"records":[ {...click to expand...}, // name=joe {...click to expand...}, // name=fred ]}
I have the same problem when working with XML as well (though in XML sometimes the ID's are attributes of the parent which mitigates the problem, but other times the ID would be nested in a child element of the structure more like you would have in json). I even found it to be enough of a problem that I wrote my own XML editor to solve this issue.
Of course one issue with this is that there's so clear standard as to which fields of an object represent "ID" information which would be important/useful to "bubble up" to the next level when collapsing. It would have to be something user-configurable (though having some sensible defaults like looking for "name" and "id" keys would work in a lot of cases). In the XML world, there's probably something to do with Schemas that would help with this problem, and fancy editors which understand your data using a schema, though some of the editors I looked at which went into that level of detail seemed like way overkill for what i wanted to do.
So essentially my question here is whether this concept of "collapse child, but keep important identifying information of the collapsed child visible" exists in any json tools? is this a thing that has a name/buzzword associated with it that I don't know about? or is it purely an issue that's my own personal quirk which nobody else cares about?
A pivot table will do exactly what you want and more. If you hate excel, You could load it into a database and do a groupby query. I suspect most JSON in that data structure were auto generated from CSV files or database anyways.
Another option is to just convert that JSON to another JSON keyed by name with Lodash then inspect it.
var anotherJSON=_.keyBy(oldJSON, 'name');
For the most common cases, I just copy and paste the string into the chrome console if I need to inspect some random JSON file really quick.
If it is too big to copy and paste, pipe it into a html file as a global variable and view the object in the console.
You can also just open a node repl and load the JSON into memory. node lets you auto complete property names by pressing tab, this is great for inspecting some unknown object.
JSONedit should work well with files up to few MBs, with larger files loading time may be not acceptable. Side note: for editors that generate structured view file size may be not precise metric - number of elements might work better.
You have to be a little careful about not accidentally printing out the whole thing to stdout, but after its loaded into memory you can check keys at a given level, drill into the substructure, etc.
> I want to be able to easily collapse and expand elements
That feature is often called "code folding", many programming oriented text editors do it.
That's a pretty large bug. Considering what JSON is supposed to be used for, passing data between two programs/processes, files larger than 1 MB should have been accounted for.
Does it freeze then crash, freeze until the OS takes care of it, or freeze temporarily then resume?