Lambda: Turn Excel formulas into custom functions
techcommunity.microsoft.com
techcommunity.microsoft.com
Users: Great! Let's start using it everywhere!
Users: Hey! Our spreadsheets have become very slow and hackers break into our systems by executing arbitrary code in our spreadsheets
Microsoft: OK! From now on you will have to save workbooks that can execute arbitrary code in a dedicated file format, which will only open after showing 15 warning messages.
....
Microsoft: _Hey, we have this new feature. It's called Lambda. You can execute any code you like and use it as functions in your spreadsheets._
new capability that will revolutionize how you build formulas in Excel
Which isn't really true. I can call macros using the =function(x) capability like forever.
* You can write it in one language (excel formula language)
* The language is simpler and known by almost all users, while Javascript and VBA are only used by a tiny proportion of users.
* The language is more secure (i.e. you can't execute arbitrary code, access files, call DLLs etc)
* Because of the above, users don't need any security permissions / get warnings when running it.
* Because they are standard excel formulas, you get OOTB support for other excel features such as dynamic array formulas and access to the full catalogue of worksheet functions (even in VBA, Application.Worksheet only had access to a few basic excel workbook functions, so if you wanted to do a Xlookup for example you are implementing it yourself with arrays and loops)
I think most people on this site know how powerful the concepts of functions and recursion can be!
To be honest, I think this is awesome and has been sorely missed.
If you just had custom functions, you can trace them back pretty quickly and end up with an understanding. Teams will also probably create ‘known’ custom functions for their use case, like converting account Chart of Account codes to finance COA codes etc.
If you want a platform to succeed, make it capable of satisfying most users needs within its sandbox; using macros is just giving up and working around it.
"Let's just create a Lambda for this"
Ok... But which one?
Here's a Stack question dated 2008: https://stackoverflow.com/questions/167343/c-sharp-lambda-ex...
2014: https://news.ycombinator.com/item?id=8116224
> spreadsheets might be an interesting programming environment if you were restricted to the native functionality with a small addition. Namely, add a new value type: "anonymous function,"...
2019: https://news.ycombinator.com/item?id=21356824
> Excel needs exactly one thing to blow open the doors on productive programming: a new "function" data type. Since it's just a data type, you put it in a cell just like any other data type. Have some way to call it, like `A1(arg1, arg2)` or something. Now you can leverage the full capabilities of Excel to manage it, name it (named ranges), etc just like other data. ...
From the same thread:
> VBA is just an escape-hatch to a 'real' programming environment; my claim is that excel sheets & formulas alone could be a 'real' programming environment in its own right, no escape hatches necessary.
I wonder if my comments inspired someone. :3
The company never seemed to gain traction, and unfortunately the open-source tool released which was based on ResolverOne had none of the power or elegance of the original.
I'd be interested to know if MS had consulted Giles Thomas from R.S. prior to this - it's certainly giving me a bit of deja vu.
Reminds me how early word processors (the person, not the software) were convinced to program word processors (the software this time) just by calling the programs "macros".
I'm not joining Microsoft's beta program right now, but I'm curious if anyone knows the data type of a =LAMBDA?
(This was Multics Emacs, a predecessor to GNU Emacs.)
Elastic Sheet-Defined Functions: Generalising Spreadsheet Functions to Variable-Size Input Arrays
https://icfp20.sigplan.org/details/icfp-2020-papers/46/Elast...
Fastest beeline from research lab to the end-user I've ever seen.
I'd be curious what an actual lambda thing would look like in Excel.
It is a bit weird and confusing because you currently can’t really use the anonymous functions without naming them, but maybe they’re going to relax that restriction eventually?
One last thing to note, is that you can call a lambda without naming it. If we hadn’t named the previous formula, and just authored it in the grid, we could call it like this:
=LAMBDA(x, x+122)(1)
Edit: typo
So, these are anonymous functions. The more interesting question is whether they're closures - that is, whether a LAMBDA nested in another LAMBDA can reference the latter's parameters, and how it interacts with LET (https://support.microsoft.com/en-us/office/let-function-3484...).
“If you create a LAMBDA function in a cell without also calling it from within the cell, Excel returns a #CALC! error.”
https://support.microsoft.com/en-us/office/lambda-function-b...
Specifically:
1) Put =LAMBDA(x, x+1) in A1
2) Try calling =A1(1) from any other cell - it will return #REF!
I'm on the Office Insiders beta track and LAMBDA does work otherwise.
That would be more along the lines of lambdas in the traditional sense of anonymous functions, closure captures, higher order functions(?), etc. Could be cool, but IDK if it would be useful? Can't think of any specific use cases right now, but perhaps once people get used to it, there would be tons of them.
Of course the real fun starts when you can do:
=LAMBDA(f, LAMBDA(x, f(x(x)))(LAMBDA(x, f(x(x)))))Well, except for the Name Manager part in their implementation, which seems to be a total disaster. I really hope this is a first version and they are going to keep improving it as they say. One should really define the functions in the cells and be able to reference them like =A1(2). The last thing a beautiful functional environment needs is globals.
Super curious what the future of Excel holds.
Name Manager though... Would be really nice if they could come up with some idioms for writing these functions in a multi-line format, with indentation, and give a slightly nicer editor. I realize that might be a bit tricky without changing the language syntax, but after the 2nd nested if-statement I find that I really struggle to follow someone's single-line Excel logic...
As is, yes, Name Manager seems very ugly.
> A good practice is to create and test your LAMBDA function in a cell to make sure it works correctly, including the definition and the passing of parameters. To avoid the #CALC! error, add a call to the LAMBDA function to immediately return the result:
> =LAMBDA function ([parameter1, parameter2, ...],calculation) (function call)
> The following example returns a value of 2.
> =LAMBDA(number, number + 1)(1)
> Assuming the LAMBDA function is in cell A1, you can reference the cell that contains the LAMBDA function in the following way:
> =A1(1)
What this seems to say is that (1) you get a #CALC! error when a cell contains a bare =LAMBDA(...) expression, and (2) if the lambda calls itself (doesn't produce a #CALC! error any more), then you can call it with a different argument by referencing the cell containing the self-invocation (the "=A1(1)" example above). This seems like a weird model, because just "=A1" would give you the result of the self-invocation.
Maybe the documentation intends to say that you can do the "=A1(1)" call iff the cell containing the lambda is not a self-invocation (but then shows the #CALC! error)?
[0] https://support.microsoft.com/en-us/office/lambda-function-b...
> If you create a LAMBDA function in a cell without also calling it from within the cell, Excel returns a #CALC! error.
For (2), I think what they're saying is that if you need to test the function with different arguments, for example, it may be more convenient to reference the cell with the lambda.
=LEFT(RIGHT(B18,LEN(B18)-FIND("-",B18)),FIND("-",RIGHT(B18,LEN(B18)-FIND("-",B18)))-1)
But if only Excel would support regular expressions like Google Sheets [0], could be done as easily as: =REGEXEXTRACT(A2, "[A-Z]{2}")
I'm sure adding regex isn't a trivial thing, but simple pattern extraction seems such an absolutely massive usecase for every everyday user that just I cannot fathom why Microsoft won't support regex. It would make Excel vastly more powerful for its purportedly non-coding users, especially since GSheets has had it for years now. Maybe someone on the product team believes regex feels too much like "code"? As a triple nested function involving subtr, strlen, and array indexing isn't? =MID(B1, FIND("-",B1)+1, 2)Actually, adding that into the list of Excel functions would be trivially easy. The hard part would be convincing all of the managers that it won't dramatically increase the amount of work they need to do in terms of tech support.
Otherwise an exciting development.
Excel is basically a REPL with cells.
This seems like it lowers the learning curve for, at the very least, adding DRY principals to more every day use cases.
I consider myself fairly comfortable in Excel. The number of times I've been burned in my own (or more likely shared) spreadsheet by things like copying down a formula that got modified in one instance but not all and related issues is staggering.
Being able to have some cells where core logic lives makes it easier for less technical people to understand what's going on, and makes formulae a lot more reusable.
"As you’ve probably noticed, we are improving the product on a regular basis. The desktop version of Excel for Windows & Mac updates monthly, and the web app much more frequently than that."
One common use case this helps takes the general form "If A2+B2 > 10,then A2+B2, else 10". "A2+B2" needs to be updated, it has to be updated twice. Alternately, you could have a "helper column" C2=A2+B2, but this adds clutter to the whole spreadsheet.
I have a number of UDF's which lend themselves well to this, such as a triangular distribution calculator. Execution through Lambda should allow the undo stack to continue working (normally ditched by executing VBA) and hopefully give a performance boost.
LET(myval, A2+B2, IF(myval>10, myval, 10))
https://support.microsoft.com/en-us/office/let-function-3484...Without IFERROR, a common pattern would be IF(ISERROR(A2+B2),0,A2+B2).
With IFERROR, you can just do IFERROR(A2+B2,0)
Excel is such a good tool in many ways, and such a bad tool in many ways. Really experienced power users can follow Excel formulas much easier than blocks of imperative code. But the more complicated it gets, the harder it can be to follow it all. But that also goes the same for software, too. I do think that there is something powerful about the "debugging" you always have turned on in Excel, in that you always know what value a formula has produced, even after it has run. And you can (usually) easily see at what point an error started in your calculations.
For the programmers out there who aren't fans of Excel, or aren't super familiar with it, if you haven't seen "You Suck at Excel with Joel Spolsky" [0] you might be pretty amazed at what you can do with Excel at an intermediate/advanced level.
100%! I like to do that in other cases as well, just to keep formulae simple enough that someone else can easily audit the whole spreadsheet.
I'd rather have 5 extra columns in a calculation, then have a huge formula in a single column. This habit is so strong that I often do the same thing with Pandas: adding extra columns to a dataframe for intermediate calculations, when it would be better to write a larger function and .apply() it all at once.
I've never used Pandas, but I imagine I'd have the exact same instincts to break calculations up as you do, since Excel was my "first programming language" that I was first exposed to in the 4th grade, haha. I certainly didn't learn much advanced stuff at that point, mostly because of the time period and being in a rural Midwest area there weren't a lot of programmers around to learn from and the internet was rather different in the mid 90's. :) But Excel planted the seed of programming in my mind, even though I didn't know what "programming" was.
I commonly use a lookup function in that context (vlookup, xlookup, or index/match).
Anyone else was a huge fan?
(disclaimer, I’m the founder)
Do you have plans in foreseeable future to bring those features? Formula formatting, debugging (at least with F9), code navigation (jump to function definition, etc) and so on.
I completely hear you on this one! I can't share more about what we are doing in the future but I will say that I definitely share your sentiment. I would love to see us add much needed tools for debugging and authoring formulas. Akin to what you get with great IDEs.
However Google sheets is nowhere the install base of excel, so this is a really big deal
(So no, Google Sheets does not support this)
The latest improvements in Excel really do seem to be re-widening the gap between Google Sheets and Excel (Dynamic array formulas, Let, Custom data types, powerquery improvements...)
I think Google Sheets has a pretty solid user base.
Highly anecdotal, but there are far more complex business processes still running in Excel that are not going to be translated over to Google sheets, and they are all offline behind a network firewall.
Those types of sheets will really benefit from this improvement.
Now if they just supported python instead of VBA for scripting!
And these are not for particularly complicated functions.
I'd really love to have this feature in Sheets. It would simplify a lot of what my sheets do, and also make it more accessible to a Sheets power user that gets scared off from code.
I think Excel can now run QEMU, come to think of it.
(without macros. With macros, Excel can run anything.)
As others have mentioned, this is essentially a user-defined function. I generally shy away from these as it will make it difficult to share spreadsheets with others as they may not even know what lambdas are. Auditing lambdas will be a nightmare, as will tracing dependencies.
Excel formulas have long been too hard to comprehend if you were not the original author... and even if you did author it, 4 weeks later you won’t remember how it worked without a half hour of review!
> If it is hard to developers, it is impossible to regular users.
What are you talking about? Regular expressions are used everywhere. On the backend it's text parsing, on the front end it's input validation. I have never written a complete application without using it. You're also the first person I've heard grumble about them.
I get that if regex is used in an overly convoluted or messy fashion, they become unreadable and unreliable. Just like assignment operators, nested division, or any other basic programming construct. But they are also remarkably powerful at solving simple pattern matching in a robust way. I recommend you go learn them instead of making baseless claims about "the number of developers who spit on the floor" when talking about them or whatever.
Many developers struggle with the "language" of regex - and no matter how many times I "learn it", it doesn't change the fact that I have to pull up references every time I'm building out an expression.
Grandparents post re: Excel had me curious actually - because I (personally) find the use of Left, Mid, Right, etc generally far more logical and readable than trying to parse a regex string.
A horrible but practical way to use them.
Being able to describe them in an Excel-like fashion and have it spit out a working Regex would be nice.
yes still SQL is everywhere
And the complexity of SQL to someone who already codes is marginal. Here in Excel we are talking about the complexity to someone with no coding experience.
The promise is that a Java (then Ruby) developer, could simply design the objects needed for the program, and the fields which need to be persistent could be automatically mapped to the database using ORM.
The reality is quite different of course, there's a reason ORM is so widely derided. But ORM is more about skipping the bookkeeping involved in setting up persistence for application code, rather than testability or opacity of SQL.
My guess in my organization (huge, 100k people) we probably have ~50 FTEs who do stupid work solely because of this feature not existing.
The "keywords" are language specific/dependent. Everytime I have to google how to do a specific thing in Excel I then have to spend 5 times longer to translate the instructions into my language.
It is a problem because in a locked down corporate environment, a worker/user cannot easily change the language.
From the description:
Functions Translator helps people use a localized version of Excel by helping translate from the US Excel function names, or research how to create a solution on the web with predominately English content.
Easily find the equivalent localized functions and formulas in any of the supported 15 languages. Functions Translator will automatically configure the language settings to US and the Localized version, and people can provide feedback on the translation of functions if it is not what they expected.
[1] https://www.microsoft.com/en-us/garage/blog/2018/03/new-gara... [2] https://www.microsoft.com/en-us/garage/profiles/functions-tr...