Hacker News new | past | comments | ask | show | jobs | submit login
Power Fx: open-source now available (microsoft.com)
186 points by swuecho on Nov 18, 2021 | hide | past | favorite | 83 comments



My non tech users are still catching up with Power Query.

I sort of gave up forwarding news related to low code from MS, as every week there's something changing or being deprecated.

Moreover, every time I tried to use anything related to the Power Platform (so I could teach to others) I ended up in a world of pain. Is anyone actually using Power Apps, etc... Successfully?


I too quit playing catch up with MS no code stuff - they change so much it's definitely not "fast iteration", it's "throw everything and see what sticks".


> I too quit playing catch up with MS no code stuff - they change so much

That might be different this time, though. Afaik it's the first time Microsoft releases a user-friendly reactive language as open source; all their previous tools in this field have been strictly proprietary.

Being open source, it can be maintained by its community of users without becoming abandoned, even if Microsoft loses interest in it.


Open source != Open governance. Consider Signal as an example. It's open-source but there's no developer "community".


That's probably for the better.

It has the benefits of the open source (you can inspect, and modify the code, exists forever, etc), without the downsides (design by commitee - sorry, community -, endless bikeshedding, code churn for the sake of it).


This will probably have developer community around it. Given that any one that builds on the platform will be invested in maintaining it and the users will be somewhat technical and the businesses that adopt it will probably put some staff time towards it.


SOURCE OPEN. sigh


What does this mean?


source open is different from "open source". look but don't touch


This is released under MIT license, though. That's open source.


Sigh another language to learn. Just released but I will see recruiters asking for five years experience in it for job openings.

It is like they took the macro language of Excel into its own app.


May I suggest your non-tech users to try EasyMorph[1] (I'm the founder)? It's easier than PowerQuery (it's completely no-code) and at the same time more powerful. 80% of PowerQuery capabilities require coding. Our tool is 100% visual (and has a free edition).

[1] https://easymorph.com


Respectfully, “easier” and “more powerful” are meaning-free descriptors. How is your product easier and more powerful? If you’re going to post an ad, at least do me the courtesy of telling me something I don’t know.


I agree, brief descriptions like sound vague and empty. So let me support my claims: EasyMorph is easier for non-technical people because unlike with PowerQuery all of its functionality is available without coding. Also, EasyMorph is more powerful than PowerQuery, because it has a bigger library of available data transformations, for instance, interval merge (a join where a key value must fall in a range specified by two fields), fuzzy matching, binning, de-aggregation (breaking down a total), or replacing substrings using a lookup table.

In terms of programming, EasyMorph is basically a high-level functional programming language (but entirely visual) with immutability, pipelining, automatic parallelization and concurrency. It allows arranging loops similar to FOR..NEXT, FOR..EACH, DO..WHILE/UNTIL loops in programming languages. It has subroutines, conditional IF/THEN/ELSE branching, exceptions, mutexes, and a built-in key-value storage.

Besides that, it combines data preparation with no-code automation because it has many so called external actions for exporting to/updating databases, sending low-level HTTP requests to web APIs, performing file operations (download, rename, delete, clone, zip, unzip, etc.), running external applications, and even executing SSH commands on remote machines. The full list of capabilities is rather extensive to list in one post and can be found here: https://easymorph.com/all-integrations.html.

Finally, EasyMorph is just faster due to its in-memory columnar engine with several tiers of data compression and aggressive optimizations that leverage CPU cache. We regularly see customers that switch from PowerQuery because EasyMorph is 1-2 orders of magnitude faster.

At the same time, EasyMorph is 5 times less expensive than Alteryx and has a very generous (as our users say) time-unlimited, data volume-unlimited free edition.

We're entirely bootstrapped, profitable, and almost double our revenue every year for the last several years.


Thank you! This comment was very informative.

A couple more questions: do you have a roadmap for supporting non-windows systems? And also do you have plans for open sourcing any of your code?


I am really curious: How does this compare to Knime (quite popular in EU). I have onboarded a couple finance departments and I agree with you that the flow approach is easier for the teams.


I run some websites that consult the public on new construction developments over a few weeks. We use Power Automate to route comments and bookings from a form into email, notifications, MS Teams, calendars and spreadsheets. It avoids needing a formal platform that users have to learn and needs to be customised. Plugging into existing communication channels (like email and chat) makes people more likely to use the service. And that is much easier with PowerAutomate than using Microsoft APIs. It is easy to add one more channels (like SMS) without lots of work. You can also tie things into a particular MS Teams site which makes authentication/security/privacy easier to deal with. I can send notifications and data to the MS Team and let someone else worry about who should have access. And at the end the Team can be deleted and the data scrubbed alongside it.


I use it at work together with Azure Functions. Power Apps is great for small functionalities you need for Azure resources. My latest example is a logic app that triggers an Azure DevOps release. When I trigger this logic app via HTTP I also pass some parameters that the release definition expects.

Another example I made is to get Microsoft Forms answers that users submitted straight into a SQL Server DB. You literally cannot programmatically get the answers in any other way, by design. Its either logic app component or export to XLSX.


Sigh. I'm trying but it's impossible to decipher anything regarding Microsoft's low-code platform.

Does anyone know if it's possible to run Microsoft's low-code platform on premises? Otherwise I don't really see the point in having an open-source language.

Don't get me wrong. I do see the point in having an easily accessible business programming language. I just don't see the point in that language being open-source if it's tied to a proprietary service which I can't host myself.


It’s interesting reading the comments here as I assume everyone here has more coding experience than myself. We have all the Power Apps at my agency and I have yet to find a use case for them. They seem really interesting but until there are good use cases I don’t see this catching on for most users. I’m trying to find some way for this to fit into my work and my employees but even the Microsoft training that I’ve attended doesn’t give much info. I’m sorry but automating notifications isn’t interesting and feels like new ways to micromanage. I feel like for this to thrive is to have good use cases put forth for regular users, then you will get buy in.


The most relevant use case I can think for reactive programming is data transformation between formats.

Say you have two departments that need to share data, but their data stores are structured in different ways. You can build an automated process that reads data from the first data pool and converts it into the format needed at the second pool, where the other department can use it.

This is a quite common use case in machine learning for example, where data from numerous different sources need to be stored in a centralized pool ('data lake') where the ML process can analyze it to build predictions.


Ok that’s pretty interesting. Some of my employees and coworkers have been complaining about taking things from SharePoint lists and making them in a more “screen friendly” format and removing the manual work. We are pretty locked down into what software we use so maybe we could adapt this.


Microsoft already offers several tools for that.

It feels like they're just throwing out new programming languages for no reason sometimes.


A bit disappointing that it's only the C# interpreter that has been released and not also the JS version used in Power Apps. Still neat to play with though.


It doesn't have semicolons at the end of the lines, it looks more like VBScript or VBA to me.


Gives the same feeling I had when I was a teenager and saw Prolog. I instantly had the idea: "Cool! You can even manage a hotel with that and with a few simple queries find which rooms are available and who is where!". Of course, nobody manages hotels with Prolog.


Great, I installed the Visual Studio for Mac (that I didn't know existed) and it immediately froze my Macbook, something that hasn't happened to me in recent history.

So I guess they really do support the full Microsoft experience on Mac.


I think if low code should take of this is the tool for it, the biggest "low code" tool is excel whit hundreds more excel users than programmers, if not I think probably we over estimate how much people care about develop apps


I would say an overwhelming majority of corporate users don't care about creating their own apps or work flows or whatever else. They care about the PowerPoint or excel sheet they are actively working on for their next meeting. Where this stuff is helpful is in the business side analyst jobs, the folks who produce the data for the aforementioned persons PowerPoint. That's where most advanced excel is happening and a subset of those folks will go further and use some of these new tools. It's the folks who have built access databases to do some of their work. And future state it will probably end up like all those orphaned access databases that are all over corporate America running some small but important reporting for some business unit who could never get hours with IT to actually get a real production solution.


Given how fiddly Excel is, which leads to extremely frustrated formula-writers and depressed advanced users, I wouldn't call Excel low-code. A rite of passage of learning programming pre-Internet, Google, and Stack Overflow (and Clang/llvm) is spending hours pouring over code, just to find out it was a missing semicolon or errant " symbol. Excel programmers go through similar shit with some regularity still (owing to the fact that it's not considered "real" programming and proper tools are elusive). (You don't have to go through that be a programmer, but there's comradery from having gone through similar rites of passage.)


Good point. Excel is no more low-code than SQL.


PL question: Power FX follows the spreadsheet model where a change in one variable propagates and updates all variables that depend on it. I am curious why this model isn't more popular. Is it more computationally intensive?


Complexity, humans are not good at complexity beyond certain levels. One good example of such model is in React + Redux story. Based on single variable your view might have 10/20 changes and to this day I see people debating what is the right way to do it (React hooks vs Redux is latest episode).


In my experience/opinion as a programmer, property-bound state tracking (where you say C = A + B, and changing A or B changes C) is easier to understand than push-based updating of C when you change either A or B. Property binding (FRP?) causes the exact set of values dependent on an input to change when the input changes, with no chance to forget (though you can still forget to register a change callback on C). But it's not a complete solution for when you want to take different actions based on what changed (eg. pushing a button to send an email).


When you want to take different actions, you can use the Stream metaphor (a.k.a. Observables) instead of variable bindings.

Streams maintain the same functional reactive execution paradigm of spreadsheets and property bindings, but you can compose several of them into a new stream with complex conditional logic. You can use the streams as event propagators, and understand the program state as a series of successive updates every time a new event appears on the stream.

https://www.didierboelens.com/2018/08/reactive-programming-s...


That's essentially what the "copy formula down the entire column, updating relative references as you go" pattern of Excel abuse does, right? Might be quite interesting to have a UI that takes all the tedious and distracting column-filling from your shoulders but keeps the tabular representation for debug/understanding.


Yes, that's how it is done in a spreadsheet.

A stream allows you to generalice the computation, not depending on the number of data items that were available when you defined the function; i.e. not different from a function with a loop.

Applications like AirTable and other modern outliners are exploring interfaces to model that kind of generalized data tables and relations with visual metaphors.


Have you ever looked at a complex Excel spreadsheet written by someone else and tried to work out that the hell is going on?

It is really hard to work which sheets/rows/columns/cells are dependent of which other which sheets/rows/columns/cells. And that is before we even get to stuff like:

=FLOOR("5/"&DAY(MINUTE(YYYY/38)/2+56)&"/"&YYYY,7)-34

(and much worse)


I realize it's "low code", I code, but I cannot see how to play with it locally other than the console/REPL example towards the bottom. Will this be available in Excel? Access? Or do I need 'Dataverse'? I will try the console/REPL example.

I work at an engineering firm, and we are stuck with Excel forever. I've tried bringing in Pluto notebooks and Julia (compare with Matlab, etc.), SMath Studio (vs. MathCAD), RStudio, Python, WxMaxima, but to no avail. If Power FX is integrated with Excel and Access, I could see it being useful for us.


Reading the comments here I wonder if there can be an opensource project doing the spec implementation, or define a spec for implementation. I can see people frustrated with Power Query or constant changes everywhere.


This is neat but how is it low-code?


Apparently, a programming language itself can't be low-code. I suppose PowerFX is designated as "low-code" because it's intended for use in low-code apps, basically, becoming the "code" part of "low-code".


In case you want to play around with it, here's a Blazor version.

https://fx.hidekisaito.com/


> Natural language support with GPT-3 and no code builders that create Power Fx formulas can lower the bar even further.

GPT-3 is not mentioned again in the announcement. What is exactly is it being used for?


It’s being used to generate announcements of open source no-code solutions


I am surprised I have never heard of this...this looks awesome!

For anyone who has used this, how does the performance of Power FX compare to a comparably-difficult formula calculation in Excel?


When will they open source something meaningful, like Windows, Office or Edge? Prob don't want you to see the telemetry and government spying source code.


I'd say .NET is pretty meaningful.


You can see the source code of Windows whenever you like. Pay for a Shared Source Initiative license and Microsoft will show you. Or just look at any of the old leaked versions. In neither case is there anything untoward going on in the codebase. Mostly what you’ll find is just integrated proprietary upstream-vendor source code that Microsoft licensed themselves for use in Windows, but does not have an unlimited right to redistribute in source-code form.


When it is in their commercial interest to do so. Which in the case of Windows and Office is probably never.



Are function names going to be translated as you did with Excel, so that there are a billion macros written in completely different languages?


So is this like embeddable engine? Or standalone server (like Elasticsearch)? Or UI SDK? How about the widgets (e.g for auto-complete)?


Most of the functions I see in Excel are just old BASIC functions. Have they reinvented BASIC?


The core of their audience for this could very well be people who learned COBOL and BASIC for work 40 years ago and want to go back to programming, minus the "pick python will you? Or go, or js, or PHP, or anything reliable and useful and open" part.


Or Cobol?


I am the cofounder of Budibase, and I agree with the many comments here around Microsoft throwing tools at the low-code space and seeing what sticks. The most popular source of users we get are PowerApps users who are fed up with the constant change and moving states.

If you are looking for an open source alternative to Powerapps - check out Budibase [1][2]

[1] https://github.com/Budibase/budibase

[2] https://budibase.com

I am happy to remove this post, as it is slightly promotional, but I do feel it is very relevant.


This looks amazing. This app would make half of my current job more or less redundant.


haha that is certainly not the purpose - it's to speed up the dev process so you can focus on other things.


Could you provide a comparison of Budibase vs. Power Apps? What are the pros and cons?


At the highest level: Budibase - open source + self-host via docker, K8s, DO Budibase - use handlebars and JavaScript for data binding Budibase - desktop, tablet, mobile apps Budibase - public and private apps Budibase - built in automation interface

Powerapps - very reliant on microsoft ecosystem (sharepoint, excel) Powerapps - uses Power fx Powerapps - mostly made for tablet and mobile apps Powerapps - private apps + portal Powerapps - i think powerapps makes use of another Microsoft tool for automation

I feel the components and outputted apps are better within Budibase. PowerApps is more mature than Budibase so it has an ecosystem of consultants


Cool, thanks!


Who's the user for this? It looks too "dumb" for a "real" programmer, and way too "smart" for everyone else. There's no way a real user will be able to navigate that "formula bar".

Source: have observed dozens of regular users through one-way mirror _while at MS_ ~18 years ago. The product later became Dynamics. Users would struggle to find the extremely prominent "OK" button sometimes, or press whichever button was highlighted by default even when they didn't mean to. Anything more complicated than that, it' completely hopeless.

Their own user testing probably shows the same thing, and that's probably why it's open source.


It's for advanced Excel users, they basically deal with the complexity of programming, just in their strange way, and I think it's a good way to reduce "Excel abuse".


Kind of like “what would you do for the excel abuse case in an environment not tied down by decades of legacy compat?"

That was my first impression (seeing this for the first time today). In an "all built on top of js" world you might just make a snapshot of the entire stack (except for the "browser substitute") part of the document. Make "open new document" essentially the equivalent of github fork from an ever evolving "the empty document" template. You might see quite an impressive reward from moving fast and breaking things instead of sticking to the paradigm of eternal 20th century compatibility.


My wife is a goddess of "Excel abuse". She (a financial analyst) unearths and uses the kinds of features in Excel I didn't know existed. But she can't code and isn't interested in learning how to do so. It's a different mindset entirely. I think MS is blindsided by having so many programmers around that 99% of the people out there aren't anything like engineers in terms of their mindset, let alone programmers. To those folks all this "accessing of properties" might as well be some inscrutable alien language.


It will not replace Frink as my desktop calculator/programming environment for "low-code" stuff especially since no other PL/utility handles unit conversions so effortlessly. I do like the command line/REPL example though. Python units libs, and other PL unit libs take too much typing. I've even made graphic apps with calcs and units for certain jobs. If this Power FX had a well-integrated units library and use, I would consider it.


It is a good question. If you want something more convenient for complex manipulating data than Excel, but don't want to use a 'proper' programming language like Python, then you would probably be better off using a visual data transformation tool - such as our Easy Data Transform.


Certainly not someone who can’t or won’t navigate a UI but maybe a motivated regular user. Usually people who are capable scripters but not professional developers.


yeah, the type of person that would use this would know Python (or JS), in which case they'd want to use that.

I noticed the same thing (technical spreadsheet users) and built Wax[0] to make it easier to build apps on top of Google Sheets.

0 - https://www.wax.run/


Darn it, every time MS does something like it I feel an urge to get myself a Windows laptop. Gotta control myself.


Just run it in a VM, no random reboots to hold you hostage.


What are these "random reboots" people keep mentioning?

My Windows boxes usually run about 12 to 14 days between (manual!) reboots, and those reboots are normally done to switch between operating systems.

It's been literally years since Microsoft removed automatic reboots and even updates can be be postponed for several weeks and scheduled at the user's convenience.

There are plenty of bad things about Windows, but tired old stories about "random reboots" or bluescreens just aren't part of it anymore.


Windows update will under specific circumstances automatically reboot the system. Any unsaved work will be lost. This, to my knowledge, has not been fixed yet.


I'd be very interested in a source for that as I've not encountered this and haven't read about it in technical publications either.


When there's an update that requires restart waiting to be installed, don't restart Windows for few weeks and leave computer idle outside active hours. Windows will install the update outside active hours and reboot automatically.

It happened to me before. I know of three occurrences of this happening to other people in this year alone.

Quick google search:

https://www.extremetech.com/extreme/316460-how-to-stop-force...

https://superuser.com/questions/957267/how-to-disable-automa...

https://community.spiceworks.com/topic/960453-windows-update...

https://answers.microsoft.com/en-us/windows/forum/all/disabl...

https://techgenix.com/turn-off-windows-10-updates/


Interesting. Still doesn't match the "random reboot" claim.

First of all, rebooting outside of active hours and while the machine is idle is the expected and intended behaviour (esp. in Home edition).

Secondly, pushing back updates only works for several weeks - again working as advertised and by no means random.

Inconvenient? Maybe, though I honestly fail to see a scenario in which continuous operation for weeks is absolutely required on a desktop system, but YMMV.

But it's neither "random" nor unexpected in any way. If a properly scheduled reboot of an online machine once a month is unacceptable, then Windows just isn't the right OS for the use case.


Yes, they are not completely random, but users may perceive them as such. Technically, nothing on computer is truly random.

Non-technical users don't configure active hours usually. Many are not even aware of this feature.

I'm not sure I would call a feature that can (and does) cause data loss, when user doesn't follow minor update schedule, only inconvenient.

The three cases I mentioned, none were using Home edition. 1st was sister's co-worker that was running some heavy computations of the terrain they were analyzing. He left workstation running over night, expecting to get completed results the next day so he could continue working on it, but system restarted and he lost one day, because he needed to run it again during the work hours. 2nd was a friend writing master's degree. She had all her research documents opened at relevant positions while she was going slowly through the research while writing. She put computer into hibernate mode each night, so she could just resume where she left of next day. Same story, Windows rebooted, closing a few dozen documents. Fortunately, she had her work saved, but she still lost her state/organization of documents and needed to find relevant section of each document again. 3rd was an accountant who was working late and left computer for a few minutes. Again, Windows restarted closing everything. I find current design of updates and active hours very narrow minded. It may seem good in theory, but from my experience, it's bad in practice.

But hey, maybe Windows is not intended for professionals any more. Just home users and gamers.


Sigh. You are contradicting yourself a little here.

> Non-technical users don't configure active hours usually. Many are not even aware of this feature.

Ok. Fair enough, but then don't end with this:

> maybe Windows is not intended for professionals any more. Just home users and gamers.

All the anecdotes you provides lack crucial details, such as was the system prompting for restarting after updates, how were updates configured, what was the actual version (19004 21H1, older?), etc. It honestly sounds a lot like some important bits and pieces have been left out.

If you're using the Professional Edition and are worried about data loss because of update-related reboots, configure a proper update schedule and group policies - that's what the Professional edition is for.

You can't come to the conclusion that an OS is not for professionals if you don't actually use the corresponding features and configure it accordingly.

Btw., Windows has a feature that allows apps to auto-reopen and restore their state after a reboot. The fact that most software doesn't support this is not the fault of the OS either.


> All the anecdotes you provides lack crucial details

I don't know all the details and I agree that they could have prevented it, if they had known how to do it. The point of these anecdotes was to show that there seems to be a serious issue with the design of this feature to cause problems this frequently.

Issues IMO are:

- default setting to restart when user is logged in and has applications opened is kind of hostile towards the user as it can lead to data loss (updates are important, but not more important than user's data)

- there is no way to configure this in the Settings app (nor is there any information in Windows Update settings that informs the user about this)

- setting is buried inside group policies. It's hard to find the correct setting without instructions. (and there are a lot of bad instructions online regarding this)

- all things considered, this feels like a dark pattern.

> If you're using the Professional Edition and are worried about data loss because of update-related reboots, configure a proper update schedule and group policies - that's what the Professional edition is for.

Configuration has reset for me once after a feature update. It might have been due to a bug, I'm not sure. Anyway, it happened long ago, under Win 8 Pro I think. Not really relevant anymore.

> Windows has a feature that allows apps to auto-reopen and restore their state after a reboot. The fact that most software doesn't support this is not the fault of the OS either.

The fact that it proceeds with a restart even, if not all running applications support this feature, is the fault of the OS. It's a good feature and hopefully it will gain more support in the future.


I completely agree with your issues regarding the settings, lack of discoverability. The whole configuration system of Windows is a mess and needs to be replaced.

Windows updates in general are a convoluted mess that breaks things regularly in unexpected ways (e.g. an update broke Visual Studio's profiler for over a year).

I only don't agree that unexpected restarts are a common occurrence these days, I'm with you w.r.t all other complaints.


Just a quick question - do you work for or contract with Microsoft or their contractors? Because the whole Internet basically disagrees with your take here.


OH but each time you boot it up, it's slow as molasses as it force runs those updates, ask me how I know.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: