I had a coworker who had the idea of having a system that automatically compiling xlsx files to Apache Spark, so that you could have the easy interface of Excel while having the processing power of Spark to crunch bigger data sets.
He actually quit the company to build it; I should find out what came of that.
Well, there are spreadsheet backends using Apache Arrow for storage. You could use something like pola.rs or Arrow DataFusion + Arrow Ballista for distributed processing of the dataset.
He actually quit the company to build it; I should find out what came of that.