6thaccount2

In the Midwest (Kansas, Oklahoma,... etc), the US has seen a massive amount of wind developed in the past few years.

To give you a feel for what I mean, in a few short years, the SPP region went from a small amount of wind to having greater than 60% renewable penetration. This means over 60% of generation serving load came from renewables at one point. ERCOT (grid operator for Texas) has also had really high numbers (upper 50).

Y not touch the sparse matrix of that system?

Yea...if you're an OR/Applied Math guru, there is some magical stuff you can do.

The problems we solve (and the timeframe we're under) are among the hardest in the OR sector. Not using CPLEX or GUROBI isn't an option.

GUROBI has some benchmarks on their site that are illuminating.

I work at a non-profit and not a University, so I can't use them for free.

My very nice, but slightly used car cost me less money and it has been driven over an hour per day for 7 years. That makes those solvers very hard for me to justify outside of production where the license costs are worth every darn penny.

Can the problem be formulated in a linear way? When doing optimization you have a function that you are trying to minimize like the cost of dispatching power plants. If there is no exponents or trig functions (and whatever else that could make it non-linear) you can solve the problem while adding constraints.

GLPK can also do mixed-integer linear programming where some variables are integers like (0,1).

If you need to optimize something that is nonlinear, that requires something else I think, but read the doc.

Thanks for the tip... I'll check it out.

It's really good to have GLPK & Open Solver (COIN OR Project) available for the public.

The commercial solvers (CPLEX, GUROBI, XPRESS) might be significantly more robust and fster, but they are unbelievably expensive even for a non-commercial license.

Open-Solver allows me to prototype on small models without getting a loan.

Why should someone concentrate on that? This area is mostly understood fairly well.

It's just the current amount of storage is tiny compared to the need. California has had 15 GW ramps before to give you the scale of what you'd need for only a grid with renewables and storage and technically you need much more than that.

Uncertainty is one of the big topics in this area. Load forecast is very accurate, but wind and solar is difficult to get right in the day out timeline.

They are different...I can understand the APL ones. I have no idea what a "monoid in the category of endofunctors" means :)

Energy markets used to use Lagrangian Relaxation and Dynamic Programming to find the least cost dispatch. Some still do, but the larger ones (all the ones in the US) use Mixed-Integer Linear Programming and Linear Programming as the engines can handle a lot more and still solve.

The problem is that you'd need to get your hands on a large dataset and none of the true ones are public. I think some of the national labs have synthetic models of various sizes to play with. Seriously cool field.

Yea...were you using Numpy or just trying to do someyin base Python?

That doesn't make sense to me. You might not save much time when going from C++ to Rust, but what about C++ to Ruby or Python?

There are things I can do with a few lines of code in Python that when I looked at how to do in lower level languages I just decided I could live with slower performance.

I used to work in retail at Circuit City. Our inventory was a mess, so corporate hired a company to fix that. At 4 AM a bus rolled up with people that spent all morning scanning items. We had to monitor them for theft and such (yes we saw several attempts).

They did an awful job. For example they scanned an item and then saw there were 10 items behind it and just multiplied the first item by 10 on the scanner (think of thumbdrives where the first one is a 512 MB drive and towards the back there may be a 256 MB drive). Afterwards, our inventory was worse than before. Corporate of course couldn't know this. I know your story is different, but they could still be getting duped.

Agreed to a degree.

I find Python in the REPL to be great, but I think Common Lisp with Slime and maybe Clojure with Cider take it to a new level. You can write your code and when it errors out it drops you into the REPL where you can update the code incrementally while it is running.

If Python gets me X% increase, Lisp and Smalltalk (and maybe Forth) get you some X% + Y%. The parent comment is right that 30% is probably a back-of-the-envelope calculation and not a rigorous one.

I'm only a novice/superficial language geek, so I can't tell you why Lisp and Smalltalk can do that though. Extreme late binding? Hopefully someone more educated can reply.

It would be that on Debian that already has the package (maybe Ubuntu does as well as they're similar). If not I suppose I'll figure out how to add it to my package manager first.

Thanks for the video! I'll give it a go in a few days when I'm done with a work conference.

I tried following the provided instructions in the install guide a few months back and it didn't work out. I can't remember any useful information (sorry), but have a new Linux computer now that might work better. It seems like a pretty cool project! Thanks for all the work you put into it!

Dyalog APL or J. J is free and Dyalog APL is free for hobby stuff, but costs money to do commercial work.

I think you've said it best. Yes there is now more free in most given setups than non-free.

I contrast this with when you paid for your compiler and all tools in most cases.

That is kind of my whole point though. I'm not working on open source software or at a tiny company, so it isn't free. I'm not sure if I could commercially use it as a solo dev either. Most community editions aren't much better than a trial license in that regard. Let me know if I misunderstand though. I'm not the best at license stuff.

I can try later. What would the command look like? I assume I have to add the package somehow first.

Language wise yes. Sorry if I wasn't clear.

C# => Free; Java => Free; Common Lisp => SBCL is Free; Python => Free; Ruby => Free; C => Free; C++ => Free; Rust => Free; Haskell => Free; JavaScript => Free; Smalltalk => Pharo is Free; Perl => Free; Forth => GForth is Free; Fortran => GFortran is Free; Kotlin => Free; Scala => Free; Clojure => Free;

Now there are some paid languages like Wolfram Mathematica, Allegro Common Lisp, Lispworks Common Lisp, Delphi Pascal, Dyalog APL, IBM Fortran and some languages with paid tools like Visual Studio for .NET and the Ultimate Jetbrains IDEA IDE for Java/Kotlin/Scala. Clojure has the paid "Cursive" IDE to go with the Jetbrains IDE. However, you can use other Free IDEs to go with most paid tools (Ex: you can use several alternative IDEs for C# and Emacs for Clojure). Also, the "paid" languages are fairly niche in comparison to the free ones at the top. How many people do you know writing Dyalog APL, IBM Fortran, Delphi, and Lisp works programs? How many people do you know writing C#, JavaScript, Java, C, and Python?

If we're just talking about software in general, then yes there is a TON of expensive and closed source programs and I don't expect that to change anytime soon.

I thought it was neat that they talk about trying to parse the "J" programming language.

Oh I agree Dyalog is super nice and that it is free for personal/hobby use, but using it at work costs ~1k/year. I'm always a bit leery working with such systems when most software is now free. If your company already uses it, that's great and if they already have a site license it is essentially zero cost to use from your point. If you're that weird guy asking to use the heiroglyphics language...it can be a tough sell to management and IT.

I'd like to play with this as it is a free APL that I could use for work without paying a license (like Dyalog APL requires). J is another free array language, but it doesn't use the APL characters that I enjoy.

I've had a little trouble in the past getting it to install (this was version 1.7) on Ubuntu. Granted I've never been an expert at installing from source, but a more in-depth installation guide or YouTube tutorial would help some.

Thanks for doing this btw! I hope to eventually get to check this out!

Now this I have trouble believing. What "basic" tasks did they struggle with?

Thanks. That sounds right...it has been many years to where the spirit of what I was trying to say still fit, but let me clear it up:

When Windows Phone7 came out, my friend had a budget WindowsPhone7 with a single core that outperformed dual-core Android at the time when it came to being a phone. The WP7 was a lot less laggy. Then when I got my first Windows Phone8 it was dual-core instead of the then Android quad-cores and it out performed those as well in anecdotal comparisons of mine.

I agree with everything you've said except for the UI being bad.

If this was desktop I would agree with you. Having a good start menu makes sense and the Unity (at least usable) and then Windows 8 (abomination) show that a UI like that doesn't always make sense outside phones and tablets.

The flip of this is true as well. For a phone, my uses are far less complex. I don't have a zillion folders, apps, programming environments, mapped drives...etc. I need internet, SMS, camera, email, office apps, and that is about it. With Windows Phone I had that on my fully customized home screen with all the icons sized just the way I wanted them. I also liked that searching for apps was just slide to the right and scroll by alphabetical order. With Android there is just so much crap going on that I don't use that gets in the way. Now I'm sure some people do use it, but I do not.

My wife had a Nokia Lumia Windows phone as well. She likes the fact that there is better support and apps on Android than what we had with Windows (especially at the end), but she really misses the Windows UI and she had Android for many years before going Windows.

I'm a Linux fanboy that hates desktop Windows with a passion.

However, both I and everyone I know that has ever used a Windows phone (there were several of us at work) absolutely loved the platform when compared to Android and iPhone. I'm running a top end Google phone and still missing my Nokia Lumia Windows Phone. That phone had a great OS and was Rock solid.

The problem was nobody wanted to give it a try. The hardware looked bad, but it honestly didn't need to be as good as Android as it wasn't as bloated. This is somewhat anecdotal, but dual core Windows Phone7 ran smoother than quad core Android at the time.

Dyalog APL in general supports a lot of old and new technology from interacting with R & Python to Jupyter Notebooks, ODBC, .NET, COM, DDE, Parallel multicore functions, graphics, multiplatform IDE, GPU support via Aaron Hsu's compiler, functional programming via DFNS & Tacit Function Trains...etc

If it was free I'd probably be using it.