The "Suddenly any financial analyst in your company, given the right permissions" comment is telling. The companies that would be most likely to use a service like Blockspring would be the types of companies that would never be willing to give and trust data access and processing to a random third party. The Privacy policy does not help alleviate these concerns (tl;dr: we keep everything). It is also unclear if the business offering (https://www.blockspring.com/business ) offers self-hosting.
I recently discovered the joys of ExcelDNA, which lets you do spreadsheet extensions using C#. One of the first things I did was spin up a plugin that allows users to request data elements from CSV files hosted on a server, essentially a giant remote VLOOKUP. The product uses TCP/IP so it can run anywhere - client self-hosted, or remote over the internet. Throughput was pretty high - 70k elements per second over a network. This, and I'm a novice programmer.
So I look at Blockspring, and my first thought is that I don't quite get it, although if they ran on proprietary, internal data sources, it would make sense. (Do people really download Amazon listings into Excel? I suppose). Or, if they could get a company like Factset or Bloomberg to let them do direct data pulls from their databases, then you'd get masses of high quality data to analyze.
In terms of the core "into excel" functionality - maybe I am understating my novice capabilities, but ever since I discovered ExcelDNA it seems like you can do almost anything with Excel and C#.
> if they could get a company like Factset or Bloomberg to let them do direct data pulls from their databases
this exists. bloomberg has a plug-in that lets you pull much of its data directly into excel using normal-looking formulas. the catch is you have to pay $20,000/year.
These are not normal parameters, they are "Secrets", a special kind of parameters that are saved internally in Blockspring and automatically applied to functions that use them, after the account owner has set them.
User Content. We collect your personal information contained in any User Content you create, share, store, or submit to or through the Service, which may include photo or other image files, written comments, information, data, text, scripts, graphics, code, and other interactive features generated, provided, or otherwise made accessible to you and other Users by Blockspring via the Service in accordance with your personal settings.
This is pretty awesome, and it's exactly the sort of product that makes me wonder if the "in the future everyone will need to code" predictions are going to pan out. With better tools that can 'intelligently' help a non-coder[1] use APIs and algorithms, the coding bit of coding becomes somewhat redundant. We'll still need the problem solving and analytics skills, and some of the organisation but actually sitting and typing in syntax could well vanish. It's feasible that a suitably easy to use programming language would make coding so trivial it actually stops being a 'skill', in the same way that wordprocessors and printers have made copy typists redundant.
[1] As a caveat to that I would also argue that a competent Excel user actually is a coder, just with a higher level macro language than most other developers, but still.
I wonder if why spreadsheets become something like programming-lite is that it allows for rapid prototyping and iterating.
This because the user can manipulate the content of variables directly, and watch the result change as they do. This then allows them to test smaller parts of the logic before aiming the result at another piece of logic and thus building a larger structure piece by piece.
The problem with using spreadsheets for this is that as you move stuff around the logic parts break.
So what if we had something like a variable grid and a logic sheet? So that any variable read or written was up in a grid like that of a spreadsheet, for easy reference and manipulation, while the logic was of somewhere else. Thus moving things around to make room for more logic didn't influence the variables etc.
And to take a step back, i wonder if this ease of prototyping was one of the things that made micro-computers so potent. This because they booted directly into a Basic interpreter (prototyping), while at the same time there was no persistent storage you could screw up with crap code (just flick the power and get back to it).
I think flow-based programming was supposed to be something like that. As spreadsheets show us, we haven't really figured out how to scale this approach to more complex applications.
Spreadsheets show us that 2d grids and a lack of other programming concepts make it very hard - it's a distinct subset of flow programming:
A unit of code an take an arbitrary number of inputs from a 2d grid and give you 1 output. Real Flow programming lets you have multiple inputs and outputs and aren't constrained to 2d. I think the biggest constraint is representing the dimensionality and interaction ... but we have classes and deep package structure in Java, so it's probably more of a effect than cause, but still a blocker to wide adoption.
The code in spreadsheets is always a 2nd-class or 3rd-class citizen. It makes it ridiculously hard to both test and review and there are essentially no good checks on what output you're getting from the input.
I've worked in orgs where C-level execs without a technical background were writing crazy complicated spreadsheets and nobody had any idea how they worked or if they were correct. When you bury the code, you bury the necessary processes around the code.
Why is everyone hating on Excel, and spreadsheets in general? They are such easy tools for creating, displaying and auditing calculations (like financial models), manipulating rows of similar data, and generating regular charts and reports.
Sure, their ease of use makes them popular with, and accessible to, people who don't think like we do. This doesn't mean they are bad tools for everyone. Python's duck typing may horrify Java programmers, but it doesn't mean that all Python code is bad or has type errors.
> Why is everyone hating on Excel, and spreadsheets in general?
Speaking for myself, because spreadsheets are intrinsically "information silos." Traditionally, this has been due to the data being locked up in the spreadsheet's proprietary files. This offering only exacerbates the issue by incorporating data from external sources which lack data management guarantees available in other solutions.
> They are such easy tools for creating, displaying and auditing calculations (like financial models), manipulating rows of similar data, and generating regular charts and reports.
When their use is restricted to this, spreadsheets do provide value. The problem is that many times the spreadsheets start off as a data integration scratch pad (as it were) and then become the "system of reference."
Used as a tool which enables one-off types of value it's hard to argue against them (so I won't :-)).
> Python's duck typing may horrify Java programmers, but it doesn't mean that all Python code is bad or has type errors.
This is a false dichotomy[1], or a straw man[2] if you prefer, as both Python and Java are general purpose programming languages.
Excel is not a general purpose programming language. It is not a programming language. It is a program which allows people to use a very specific DSL to benefit from it in novel ways. And when they do, any chance of autonomous integration is lost.
I think this concept has far reaching effects on optimizing white collar operations if you're able create social features based on the data.
I'd want to know who else in the company is using this data? Who has used it on the past? Have they done work that is similar or even a duplication of the work I'm doing?
These information management issues are currently hidden, but result in lost productivity. Just this past month my friend at Google found out another person had already done his analysis and he could learn from the previous work. Just knowing someone had previously used the same dataset could have saved him 7 weeks of work.
> Just knowing someone had previously used the same dataset could have saved him 7 weeks of work.
Information about data is not black-and-white in the corporate world. Even in the same company, internal data is usually on a need-to-know basis, with necessary NDAs. A service that tells you who else is using the data would defeat the purpose.
As an extreme example, what would happen if a service allowed data-sociality with HIPPA-protected data? Not good things.
> internal data is usually on a need-to-know basis, with necessary NDAs
I'd argue that we're in the "unknown unknowns" territory, rather than NDAs being the blocker.
In a company of any significant size, Joe over in Finance is hacking away at a spreadsheet utterly and blissfully unaware that Susan in Corporate would gladly chop her left arm off for the data he's producing.
I don't think open access lists for HIPAA data would be a bad thing. Why do you? In fact, it sounds like a great idea. Shine some light on who is accessing what data.
The fact that a certain specialist has accessed a patient's records would reveal something about their health. Even the frequency of general access conveys information about your health.
A nice hack for someone who wants to use this tool (granted that it is free for non-corporate customers):
If you are a data-scientist and would like to extract public-data from 1 of the APIs they provide, you can use this tool, pull that data (take it in raw form) and then export as both a spreadsheet and as CSV.
You can then use the CSV with your data-analysis toolset (Python probably has support), and voila! You've saved yourself the headache of wrangling APIs.
Apart from that, I'm going to go with the opinion of the other comments made that you probably don't want to be putting your private Excel info on another acquisition-driven startup.
I doubt anyone with hardcore spreadsheet data would trust a third party plugins. My employer rolls most software (incl Excel plugins) in-house for that exact reason - can't let outsiders near the data.
This is such an interesting angle on making technical tools more accessible. I can't say I have a use for including functions in spreadsheets — but when I imagine where this could go it's exciting.
For anyone who wants to view the actual blockspring UI/UX along with what APIs are available.
I personally prefer seeing the product rather than reading marketing materials. In many cases (not specific to blockspring) they're completely different in terms of design, messaging and stated features.
As an early user of blockspring I was excited to login and add another one of my APIs (bitcoin related) that would be perfect for having in spreadsheets. Unfortunately I couldn't find any obvious way to add an API. Naturally I hacked it by going to one of my existing APIs (thanks google, I couldn't find a URL from the UI otherwise) and cloning it. Here's the UI of the create API page[1] for anyone who's interested.
Edit: You can create new APIs on blockspring by going directly to a route[2] which still works. I saw Alexa had an API but it needed authentication credentials, and since I already had found Alexa's undocumented API[3] I added it to blockspring[4] just for fun.
How does this differ from building plugins? Excel already has a programming interface and I'm pretty sure you can script Sheets with python. Is it just easier?
There was Resolver One, which apparently got acquired by PythonAnywhere, which allowed Python scripted spreadsheets. It was AWESOME when I tried it, but tad too slow and no takers in the office because spreadsheets get passed around and need to be verified/rerun by people who do not know any programming. Even VBA at times.
"They just need an easier on-ramp to their internal database to do things like comparing projected revenue to actual revenue in order to close out this month’s financials."
This is a small gap. On one side, you have people copy-pasting CSV files into excel and doing it one-off.
On the other side, you have specialized SaaS products for business workflows. (Wouldn't you rather just use some accounting software to run your companies financials, rather than muck around in Excel?)
In the middle, you have people writing either code or Excel Macros to automate some of this. Looks like BlockSpring is meant to automate that group away. Just having spent 3 minutes with blockspring, there's friction. (Log into this service, fiddle with the parameters, clean up data, remove mess, etc)
I can see the IFTTT / Zapier comparisons but this is a lot more work and far less reward. Those tools give immediate results.
At the end of my integration with Blockspring, I'm left with a table of data that I STILL have to analyze in Excel.
Goldman Sachs already has a lot of financial and internal data available in spreadsheet plugins via Slang, and it's generally very useful for financial models (at least when I was there). This just seems like a way to generalize that for the masses and for other types of data.
Basic skills in medicine and accounting are useful to have and sort of prevalent. I'm sure you put on a band-aid before. Basic skills in automating certain computer tasks seem to be valuable to many people -- almost all office workers, I'd wager.
I don't see how that devalues the profession. There's a huge difference between what people can achieve using stuff like this and what I do every day.
Maybe basic skills in stuff like this will even translate to a better appreciation of what professional developers do. A guy can dream...
When it comes to programming, 'putting on a band-aid' is analagous to 'entering user/pass credentials'. Yes, everyone can put on a band-aid, just like everyone can log on to something. Neither take skill or understanding, and both are 'just things you do'.
On a tangent, a great many people have positively woeful understanding of biology. I think biology should be a mandatory subject in high school. After all, the one thing that absolutely every human has and uses throughout their entire life is their body; that can't be said of pretty much any other high school subject.
"Everyone should learn to write, but not everyone should be a professional writer."
But, it's not that we want "everyone to learn to code"; what we really want is for everyone to learn how to use computers to build powerful simulations so that they can test and explore interesting and complicated ideas.
Certain issues, like climate change, become much more tractable to discuss once you start using Excel rather than Word.
I don't think "anything" is a good term. Its good use cases are only of read-only data. Reading (and preprocessing) data from your services/databases inside a spreadsheet is quite awesome.
No no no. People at my company already use spreadsheets where they don't belong (in the place of databases, note taking apps, regular expressions, and one time someone tried to do page layout with it). The last thing we need is for these bozos to program with it. I see the need for an intermediate entryway into programming (Hypercard), just don't tack it on to an already bloated program like excel.
I think you need to sit down next to those "bozos" for a day or two and see their actual workflow. I've seen and worked closely with people using Excel instead of databases; hell, myself I used it as a map editor for a 2D game (for page layout I prefer PowerPoint).
The reason Excel is so popular is not because non-coders are dumb. It's because "tools" provided by IT department are usually piece of crap that makes work more difficult for people using them. Take using Excel instead of databases - why spreadsheet is better? Because it has easier data entry, and it's free-form. When you need to modify your workflow to include some additional notes, you don't have to write to the manager of the manager of the internal procedures to maybe consider updating the schema somewhen before next decade. You just add a new column and you're done.
I'm sure when someone creates a program that actually solves the problems those people encounter every day, they'll gladly switch to using it. But it's easier to say that users are stupid instead of actually caring about what their needs are.
I don't think the users of spreadsheets are dumb. I think they don't have an appreciation for organization of data to allow for efficient machine-readability. Your use of a spreadsheet as a game editor shows that you're one of the exceptions that proves the rule.
The free-form nature of spreadsheets, in my experience, almost always makes for human readable repositories of data that are nearlyuseless from an automated processing perspective. I cringe when I'm asked to export data from a database into a spreadsheet for users to edit, because I know I'm going to get back an unparsable mess that I'm going to have to invest time in to make usable again.
The spreadsheet is, effectively, shifting the user's inability to understand why data needs to be organized well off to a programmer (or some other grunt) who ends up fixing all the data up. (Or, alternatively, nobody fixes it up and the business ends up making bad decisions based on garbage data that came from a spreadsheet that's a disorganized mess.)
> I don't think the users of spreadsheets are dumb. I think they don't have an appreciation for organization of data to allow for efficient machine-readability.
Well said...it's that concept of machine-readability that is so core to efficient data investigation and interoperability with other enterprises (such as web apps)...theoretically, such a concept is learnable without a decent jump into the computer science/programming waters...but honestly, I haven't run into many lay persons who have figured it out well enough for themselves.
I would say that Excel is more a writing environment rather than a reading environment, the problem is not with Excel itself, it's just there is always a lack of validation, consistency and proper backups / history with Excel. So people just jump on it because it's simple to use and just make sense but the problem is that they don't see the broader picture and how the things they are actually creating are going to be used in the future (and if they can).
And considering that even running some CRUD app framework like Rails or Django is verboten in many corporate environments, Excel (or Access, but even that seems to be getting rarer) is simply the most powerful tool available to most people at work.
Isn't it exactly the opportunity at hand? No matter how much you don't want them to, people will still use spreadsheets, and the reasons why don't even matter. Instead of getting in another lost battle (Hypercard), just give people what they want, and combine their love of spreadsheets with their need for data connectivity.
Which, from an interoperability perspective, is an improvement, no?
I feel like this is one of those things we as geeks tend to miss. We look at that document and say "Ew! Look at that crappy HTML." But we miss the point that, no matter how crappy the HTML is, the fact that it's HTML at all makes it better in many ways than the same doc was when it was locked in .DOC or .DOCX.
That was exactly my first thought as well. The LAST thing any company needs is more excel sheets—unprotected, easily lost, bloated, no-backup excel sheets.
Leaving alone the observation that, in many companies, the rank-and-file don't even know how to insert a new row in a spreadsheet, let alone muck with functions..
It's the ones that know the functions that you have to worry about. Especially finance people. Excel is the software they are most comfortable in, and they never want to leave it. My boss just made our company events calendar in excel. Why not do this in a calendar program? Because he can do it faster in excel.
If he can do it faster in excel, isn't that a good enough reason?
If the problem is that the spreadsheet later becomes an unworkable mess, isn't that an argument you can use to convince him to use other tools? And if he remains unconvinced... maybe excel was right for him after all?
That's a very good reason. I perfectly understand not willing to learn yet another fancy tool that looks nice, but is unfinished, buggy and will probably be gone next year. Some people are too busy doing actual business to waste time on that.
The biggest benefit of Excel is its flexibility - you can do what you need, how you need, and you can adjust and readjust your sheets on the fly to match the changing and messy needs of doing business. That database you're trying to force those workers to use? It's probably designed wrong the moment you save the schema, because the needs changed.
So Excel is unsecure and people keep losing their spreadsheets by moving them between each other on flash drives? Solve that. Maybe with a Dropbox-like solution for syncing documents (no shiny webapp bullshit, no logging in to domain NAT Samba Kerberos Windows Briefcase Sharepoint nonsense, just "a folder that syncs" and enforces some security).
I have the feeling that a lot of problems between IT and non-tech workers stems from the former treating latter like morons instead of trying to help them to do their jobs efficiently.
>I have the feeling that a lot of problems between IT and non-tech workers stems from the former treating latter like morons instead of trying to help them to do their jobs efficiently.
No. I know this sounds bad, but they really are morons.
OK, maybe that's too much. They're not morons. But neither are babies (could you learn a language as quickly as they do?). Like babies, they just don't know stuff. And they never have an incentive to learn it. Programmer is always there to save the day.
When the ability to code is seen as essential to be in a C-level position, I predict that many frustrations with IT will just magically disappear.
>I predict that many frustrations with IT would magically disappear if IT stopped calling everyone who doesn't know how to program morons or babies.
That's already happened. Have projects magically aligned to budgets and schedules? Have we been able to fire all the programmers and replace them with business analysts yet? Have Excel sheets become maintainable?
Certainly it's a good thing if IT doesn't hold itself as superior; but that does not mean that it's not a bad thing that most business people don't have what is becoming a more and more essential skill.
1. People have to read this calendar.
2. People have to edit this calendar.
3. Other people in the office don't know Excel as well as him.
4. Other stuff is designed to work with our calendar software, but not excel.
A hammer can drive a nail into wood. With a little help, it can also remove nails, smash glass, and pry things open. But if you're using a hammer to stretch pizza dough, get help.
Those are pretty good arguments you can use to convince your boss to start using your calendar software! So why hasn't he switched? I see some possibilities:
- No-one has pointed out these shortcomings to your boss.
- You have, but he is stubborn and refuses to see the downsides even though they are obvious to everyone else.
- You have, but you're overstating the downsides and the current setup actually works well enough as it is.
Even though he may be able to start working faster, there's no guarantee that the time-to-completion is faster. Also, in a calendar program it's going to be orders of magnitude faster to change stuff like the design in hindsight. And you're likely able to achieve designs that are physically impossible to do in Excel.
I think a better idea would be an easier way for those people who built a spreadsheet (that was supposed to be temporary but then became permanent, and then became critical) to transition to a database (that lives on your own machine) would be a better idea. More data lives on spreadsheets than databases in a typical office environment, and part of the reason why is data entry speed, but another part of the reason why is that it's hard to get people to transition to the database.
This could be a good approach. Let people figure out optimal workflow with their spreadsheets, and when those spreadsheets stabilize, then help them move to a database. Aside from data entry speed, part of the reason is flexibility, especially when you're just figuring out what's the best way to collect and process data for a particular task.
As a developer in a company that does custom integrations, I wholeheartedly agree. Let them pay us to convert their unstructured files, instead of letting some automated importer do the job.
I'm only half kidding; the margin on our Pentaho Kettle transformations is quite good.
I would love to see a modern Hypercard. Use http to share plain text info; use "Supercard" if you want to build a complicated visual presentation with bells and whistles (rather than cobbling together from html, css, js, etc.). There has got to be some demand for this.
I've yet to see a product that turns non software engineers into software engineers. These types of products only make salespeople commissions and then sit unused. Crystal Reports is a prime example. I've seen it sold many times to people at companies who buy into the notion that software engineers can be replaced by drag and drop tools. The people who buy into the sales pitch lack the desire or discipline to learn it. Software engineering is difficult work and requires patience, which is often lacking in people who think a half-assed design document written in MS Word is anywhere equivalent to writing bug free scalable code that needs to run 24/7.
Case in point, one of the example scripts in the blog post (https://open.blockspring.com/pkpp1233/get-amazon-new-price-b... ) requires you to input a Amazon Product Access Key and a Amazon Product Secret Key as parameters.