Hacker News new | past | comments | ask | show | jobs | submit login

Looks nice. Do you plan on adding more data transformations functions? I’d love to be able to use this to do some serious data munging.



I do. I have phone numbers and more extensive number transformations on my to-do list. Those probably won't make it in there for version 2.0, but maybe 2.1 or 2.2.


Awesome, here are some ideas for you to consider:

1. Calculate columns using expressions that reference other columns

2. If/then logic to build rules around calculations

3. Aggregates

4. Lookup values from other columns or columns from other CSV files that have been loaded

5. Joins

6. More string manipulation functions (e.g. extract substring, trim left, trim right, regex operations)


An opposing viewpoint: please don't add any of these suggested features. I prefer the programs that manipulate CSV to only express the actual contents of the file. Formulas are the antithesis of that. The beauty, and utility, of CSV is its simplicity. All of the requested functionality can be easily performed in SQL, Excel, Pandas, R, Krangl, etc. Please keep formulas out of a CSV editor.


I agree with that purity argument, but I wonder if it really has to preclude all spreadsheet calculator ambitions: just take the "multiple files project" approach and make it good. One csv for raw data, one csv for configuration/formulas, one csv for intermediates/results.

Make full use of whatever awesome multifile read/edit the software presumably has already and then maybe go a little beyond. The "configuration" csv might have some hashbang equivalent defining a line offset for the header that isn't mapped into input/output files (perhaps a concept of "negative line numbers" for project properties?), contains references to those peer files and maybe activates project options like "display peer file contents in cells that are otherwise empty" if you want your working surface to remain "2d".

The argument to have this add-on (I'm not sure that it should be part of the product or a separate extension) would be that it would be a shame to learn all the UI ergonomics of the plain reader/editor and then not be able to leverage them for some calculations as well.


I am not talking about per cell formulas like in Excel. I am talking about a basic expression language that operates on the entire column, more like a database. Modern CSV already has some basic data transformation facilities, I'm only suggesting a few more in the same vein that make it a more potent data editing tool.


Possibly the most efficient and effective method here would be to integrate Lua into the internal data structure used for cells, and let replwoacause do whatever they like with a scripting language. I can definitely see how that would be useful to people.

I would actually be wary of trying to make this too much like a spreadsheet, even in appearance. You'll be pulled into that strange attractor, but you can't hope to compete in that space. Whatever oxygen isn't consumed by Excel itself is long gone between LibreOffice and online spreadsheets.

Another possibility would be integrating with SQLite via CSV export and giving people an SQLite console: https://www.sqlitetutorial.net/sqlite-import-csv/


Happy paying Modern CSV user here and I say Yes please to item 4.

I recently was given a pile of hand-edited CSV files (a one off data transfer between two systems). The original export was missing one column, so being able to merge that in would have been very useful.

I'm sure with Pandas if you know the magic incantation it's easy to do (join two files on the ID column, and add column X from file 2 to file 1) but I wrote my own script for these files.

In others I've resorted to opening the CSV in Excel, doing an XLOOKUP and then fixing up the mess Excel created in Modern CSV afterwards.

(To the sibling commenter saying you can do it with SQL: yes and I'd love to. But how do you pipeline that with multiple CSVs that may have slightly different column names? If done manually it seems quicker to do it my way)


I think you can do this in PowerQuery in excel and export as a clean CSV for the future fyi


Thanks, I'll look for that next time. The problem with Excel and these CSV files was it mangled the date column, which I only found out about after a bunch had been fully edited. I used modern CSV to rescue this and so have been suspicous of going back to Excel to do any further manipulations.


The good news is PowerQuery has data types! Although it is not as flexible in editing particular records if that is a requirement (just bulk edits, joins and merges)


Isn't the whole point to have a lightweight system for viewing and basic manipulations of csv files, so that you could safely import them into more powerful systems that let you do complex transformations, visualizations,... ?


At this point why wouldn't I just go straight into one of the more powerful systems then? They all have sufficient viewing capabilities as well. The idea behind adding more utility functions to transform data is to make Modern CSV a nimble but capable CSV tool without the need for a more robust solution (like Power BI, Tableau, etc.)


Because csv import into Excel can be a bit of a nightmare if the data is formatted in a way not covered by the import wizards. I've written Python scripts before to format data so that Excel would parse it correctly (which tbf was mostly the fault of the party producing said files), I wouldn't have minded having a light-weight visual tool for that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: