Modern CSV Version 2 Beta is now available
moderncsv.com
moderncsv.com
- Multiple cell/row/column editing - Easy navigation between files - Keyboard shortcut customization - Command palette - Read-only mode for super large files
With version 2, I'm adding some basic data analysis tools, some new themes, M1 compatibility for Mac users, and a whole bunch of editing commands, most of which are user-requested (well, technically all since I use my own product). The current beta version will work until June 25. It includes all Premium features without the need for a license. I'll be happy to hear any feedback you have about it!
string -> interpreted as a number -> displayed in scientific notation -> saved to disk with the last 3 or 4 digits zeroed
Pessimist: the glass is 1/2 empty
Excel: the glass is 1900-02-01
---
We've had bugs raised against our software because the clients use meeting titles like 1-2-1 for supervisory reviews which when extracted for reporting and opened in Excel (they love to dump data into Excel no matter what report functions you include directly in the application) get interpreted as a date even when output as a quoted string in the CSV file. Try explaining to a client that we have formatted it correctly and Excel is reading it wrong…
(We could of course output Excel files directly, but some of them can't download office documents from web apps because of security policy at their end.)
Given how badly common tools mangle unambiguously correct CSV data, how many variations there are which make “unambiguously correct CSV data” a somewhat small proportion of what is out there, and how many tools not only expect but require mis-formatted data and/or output it, it is scary how much the format is relied upon in major industries.
In a nutshell, CSV isn't a format. It's a family of formats, and it's not even a well-specified family of formats.
At least in semi-technical circles, I've had some success in using this to push back against CSV suggestions and get them to use better things. I'm sure that in non-technical circles I'd have zero success with this, though. It sure ain't a magic talisman you can use.
JSON isn't exactly a rigidly specified format, but it's got a lot less flex in it and I've not had as much trouble with it. Biggest problem I have is just getting people using dynamic scripting languages to please output either a string or a number, but don't just output "whatever the scripting language happened to decide based on what code paths I happened to run" when you don't even realize your code ends up casting it back and forth without you knowing and what comes out is effectively random from my point of view.
There is RFC4180. Though by 2005 when that came about there were already so many different cases around that it became just one of a great many possible variants.
I try not to push back too hard about CSV, for fear of “well, there is this XML format that is supported”! (bad enough in itself, but sometimes the “XML format” is even more poorly specified than the client's CSV edge cases which we are expected to guess).
JSON is nice as long, as you say, that strings are real strings and numbers are real numbers.
Oh, and dates/times are in an RFC3339 (or ISO8601) numeric (no localised month names, etc.) format either in UTC or with the timezone always specified, as strings (though at a pinch I'll accept a posix time_t for datetime if based on UTC). Not specifying how to handle dates/times/both is the major problem with JSON in my experience.
Just gave this program a try and it lets me save changes without removing any of the double quotes, yay!
The crazy part is that it wouldn't be all that hard to handle it properly. Excel could examine each column and apply a uniform transform on each column instead of applying transforms on a cell by cell basis. They could even put in real effort and let the user choose the format for each column as part of the import process. You know, like being able to specify "text" for columns like SSNs or Credit cards that you aren't going to do math on anyway.
Version 2 looks great, it's a hard balance to add features and not become bloated so will be interesting to see what it's like going forward.
(I love the product, not affiliated, just been using it every day since November 2020 as a happily paid user and recommend it to anyone who has the "pleasure" of working with CSVs)
I suppose I'm a trailblazer, after the video game space and Jetbrains, of course.
Such a magical time that was.
Not impressed.
Edit: On second thought, it's probably grayscale intensity hex values of a 1920x1080 image. I can reproduce that myself. Feel free to send me your file if you want, but it might not be necessary.
9,9 ... until 1920 columns.
...
until 1080 rows.
For now, I may put a band-aid on it with a popup asking if you want to disable the feature when there are a lot of columns. I have some ideas on how to make it more efficient. I'll see what I can do before the full release.
You might want to remove the part of your web site that says "View Large Files Quickly" because I was very excited when I read this and then very disappointed when it wasn't fast at all.
Notepad++ can open the file instantly and allows me to move around very quickly. You'd probably need something close that level of performance before you can claim that your product is fast for large files.
Gotta eat your own dog food, yes?
You have to change the setting in the Settings file (Edit Settings command) under the "User Value" column. Changing it under the "Default Value" column is a common mistake that's really my fault, so I intend to rectify it soon.
If it still doesn't perform like that for you, let me know.
In[1]:= comma = {"Albania", "Algeria", "Andorra", "Angola",
"Argentina", "Armenia", "Austria", "Azerbaijan", "Belarus",
"Belgium", "Bolivia", "BosniaHerzegovina", "Brazil", "Bulgaria",
"Cameroon", "Canada", "Chile", "Colombia", "CostaRica", "Croatia",
"Cuba", "Cyprus", "CzechRepublic", "Denmark", "EastTimor",
"Ecuador", "Estonia", "FaroeIslands", "Finland", "France",
"Germany", "Georgia", "Greece", "Greenland", "Hungary", "Iceland",
"Indonesia", "Italy", "Kazakhstan", "Kosovo", "Kyrgyzstan",
"Latvia", "Lebanon", "Lithuania", "Luxembourg", "Macau",
"Mauritania", "Moldova", "Mongolia", "Montenegro", "Morocco",
"Mozambique", "Namibia", "Netherlands", "Macedonia", "Norway",
"Paraguay", "Peru", "Poland", "Portugal", "Romania", "Russia",
"Serbia", "Slovakia", "Slovenia", "Somalia", "SouthAfrica",
"Spain", "Suriname", "Sweden", "Switzerland", "Tunisia", "Turkey",
"Turkmenistan", "Ukraine", "Uruguay", "Uzbekistan", "Venezuela",
"Vietnam", "Zimbabwe"};
In[2]:= Plus @@ (Entity["Country", #]["Population"] & /@ comma)
Out[2]= Quantity[1990645326, "People"]
In[3]:= period = {"Australia", "Bangladesh", "Botswana", "Cambodia",
"Canada", "China", "DominicanRepublic", "Egypt", "ElSalvador",
"Estonia", "Ethiopia", "Ghana", "Guatemala", "Guyana", "Honduras",
"HongKong", "India", "Ireland", "Israel", "Jamaica", "Japan",
"Jordan", "Kenya", "NorthKorea", "SouthKorea", "Libya",
"Liechtenstein", "Luxembourg", "Macau", "Malaysia", "Maldives",
"Malta", "Mexico", "Myanmar", "Namibia", "Nepal", "NewZealand",
"Nicaragua", "Nigeria", "Pakistan", "Panama", "Peru",
"Philippines", "Qatar", "SaudiArabia", "Singapore", "Somalia",
"SriLanka", "Switzerland", "Syria", "Taiwan", "Tanzania",
"Thailand", "Uganda", "UnitedArabEmirates", "UnitedKingdom",
"UnitedStates"};
In[4]:= Plus @@ (Entity["Country", #]["Population"] & /@ period)
Out[4]= Quantity[5210743028, "People"]
BTW, I'm suprised this is written in Qt. Looks very modern Windows, in a good way of course :^)
Excel formulas are worst abominations of semi-programming. My coworker took advanced excel courses and I sometimes just can’t help her with these monstrosities because everything around them sucks if it’s something more than a simple max-3-term expression. If it was just python or lua, I’d cast few spells and she’d understand and use them much more easily.
> GNUPlot interaction.
> Scripting support with LUA. Also with triggers and c dynamic linked modules.
> Implement external functions in the language you prefer and use them in SC-IM.
Does anyone here have experience using it? Is it stable and reliable, and does it handle large files well?
Another option is to import into an org-mode table and then re-export as csv when needed.
In my humble opinion the light theme would be better as a default because I am used to Excel. I suppose most Excel users would say the same.
I'm posting this since the developer is here and replying to the comments.
Is the delimiter configurable?
1. Calculate columns using expressions that reference other columns
2. If/then logic to build rules around calculations
3. Aggregates
4. Lookup values from other columns or columns from other CSV files that have been loaded
5. Joins
6. More string manipulation functions (e.g. extract substring, trim left, trim right, regex operations)
Make full use of whatever awesome multifile read/edit the software presumably has already and then maybe go a little beyond. The "configuration" csv might have some hashbang equivalent defining a line offset for the header that isn't mapped into input/output files (perhaps a concept of "negative line numbers" for project properties?), contains references to those peer files and maybe activates project options like "display peer file contents in cells that are otherwise empty" if you want your working surface to remain "2d".
The argument to have this add-on (I'm not sure that it should be part of the product or a separate extension) would be that it would be a shame to learn all the UI ergonomics of the plain reader/editor and then not be able to leverage them for some calculations as well.
I would actually be wary of trying to make this too much like a spreadsheet, even in appearance. You'll be pulled into that strange attractor, but you can't hope to compete in that space. Whatever oxygen isn't consumed by Excel itself is long gone between LibreOffice and online spreadsheets.
Another possibility would be integrating with SQLite via CSV export and giving people an SQLite console: https://www.sqlitetutorial.net/sqlite-import-csv/
I recently was given a pile of hand-edited CSV files (a one off data transfer between two systems). The original export was missing one column, so being able to merge that in would have been very useful.
I'm sure with Pandas if you know the magic incantation it's easy to do (join two files on the ID column, and add column X from file 2 to file 1) but I wrote my own script for these files.
In others I've resorted to opening the CSV in Excel, doing an XLOOKUP and then fixing up the mess Excel created in Modern CSV afterwards.
(To the sibling commenter saying you can do it with SQL: yes and I'd love to. But how do you pipeline that with multiple CSVs that may have slightly different column names? If done manually it seems quicker to do it my way)
I just tested the email form myself and saw the same thing, but my address was added, so yours probably was too. I'll be sure to fix that.
I develop a similar desktop application called Text_Comparer [0], days ago I update it to v1.7.2
Includes features that cannot be found on ModernCSV or Rons CSV editor, read the included help.md
[0] https://www.pipiscrew.com/threads/text_comparer.42/
if you cannot access the page, try a proxy.