Great question. Personally, I would check out the J environment. It is a descendent of APL, but uses standard ASCII characters. It is an array oriented programming environment, so it is built for math and is really fast for an interpreter. Programs are very terse and succinct and measured in characters instead of lines of code. It would be very ideally suited as a kind of very advanced calculator.
J comes with a very lightweight IDE and you can download all kinds of libraries and do things like plotting.
If you are ok with RPN (reverse polish notation), I'd check out some Forth languages which make heavy use of the stack. There is a really cheap (but nice) commercial forth (forth is kind of like a family of languages which share a core ideal rather than one language) called 8th (https://8th-dev.com/) that has a free version as well. It is literally a small download with a bunch of C executables. You can run fully interactive (just like a calculator), run as a script that calls out to the interpreter, or compile your program to an executable. You can cross-compile to Windows, Linux, OSx, Android, IOS.. etc and depending on your version has libraries for ODBC, SQLite, GUI, encrypted binaries, automotive grade Linux...etc. The creator is super responsive on his forum too.
Of these two, J probably makes a little more sense as a calculator and the plotting is nice, but array programming takes awhile to click. 8th is much lighter weight, very easy to deploy, easy to learn (although forth does take some work), and better documented in my opinion.
I'd also give Julia a try, but it might not be as lightweight as you want.
Another option is to just use a physical calculator if you'd like or Excel?