The Emergent Features of JuliaLang: Part II – Traits | Hacker News Reader

The Emergent Features of JuliaLang: Part II – Traits

invenia.github.io

107 pointseperim18 comments

Interesting to see this here as I've seen Invenia post on the Julia subreddit from time to time.

Can anyone from Invenia explain how they make money or what they're trying to do? I work with electricity market optimization every day (main topic listed on their blog and site) and am curious what they're trying to bring to the table. There are already lots of vendors in this area (ABB, Siemens, GE) that use the state of the art MIP/LP solvers to run the US grid (what CAISO, PJM, SPP, MISO, NYISO, ISO-NE, and ERCOT all do). On the retail side (think DER aggregators) I know SolarCity is trying to get into that game.

Julia is a great numerical language and it has a good solver agnostic library, but rewriting stuff in Julia won't get you much of a gain in this market. You mention machine learning which has several use cases for the grid, but I don't see a product or a mention of Invenia working with any utilities, ISOs, or vendors.

Btw: I hope I don't come across as negative (I wish you the best of luck). I do see a lot of companies do things like try to sell block chain to utilities without them even knowing what kinds of problems they have.

"Can anyone from Invenia explain how they make money or what they're trying to do?"

Whenever you have to ask that question about a tech company, you can generally assume that the answers are "venture capital" and "attract more venture capital", respectively.

That's only if they are offering some sort of product. Based on their website blurb, they're probably getting paid as consultants by electric companies:

> We interact directly with the grids, helping to plan for generation, flow and use of electricity in advance of real time operations. We help the system operators to optimize the power grid to ensure reliability, efficiency, transparency, while reducing harmful emissions.

Each system operator that I know of (I'm not omniscient) has a planning department (1-30 years), as well as groups which handle near term planning (1 day to 6 months out).

It wouldn't be unheard of to work with outside consultants on special projects, but that would normally entail working with someone that has very specialized experience in power systems operations (ex: expert in voltage stability). I'm not sure if that is the case here, but maybe one of their employees will stop by and enlighten me. Again, wording is really tough in online forums. I am genuinely curious about the role they serve. Although AI is hardly new in power systems (lots of lisp and prolog applications were created in the early 90s), those were mainly expert systems as opposed to the machine learning of today. Like I said, there are definitely some applications, but usually nothing that some Python or R can't handle. I'm not sure if there are many products that would sell in this space. Maybe their employees have picked up enough domain knowledge coupled with good development skills to help with machine learning projects like you have guessed? It is certainly possible.

If what I've seen is representative, there could be a big opportunity in overhauling the short/medium term forecasting piece in a lot of those companies and adding in a versatile timeseries and analysis platform, which would then allow you to do more accurate forecastings more often (and given the number of readings you're gonna get that might need some thought). And since you're already developing something new you can probably get a nicer architecture out of the whole thing anyway, which might allow you to use all that data in more ways than just for that single purpose. Getting rid of some lock-in from certain database vendors in the long-term might also be nice.

Just speculation, I kinda agree in that I doubt yet another optimisation package is gonna be so useful.

That is pretty funny, I spent time writing market optimisation for energy companies.

The reason people use Julia in that area is JuMP code is VERY easy to write, and maintain.

Yes it's nice, but there is also multiple Python libraries (Pyomo and Pulp) as well as numerous domain specific languages for solvers such as (GAMS, AIMMS, AMPL...etc). As someone who really likes Julia, it probably isn't enough of a difference here to justify a new product. It is an excellent choice for them to make I think.

We are largely using Julia since it is more suitable than Python for pursing novel approaches, rather than using off the shelf packages. Further, the packages are more flexible and we can easily make PRs to improve them without e.g. having to write C, and we often do just that.

Thanks for taking the time to reply. It really is a nice language!

Invenia has been interested in electricity grids since 2006 and has provided many kinds of services to interested parties, for example load forecasts. We are continuing to work on many things in the area. Recently we published some of our blue sky research on grid optimization using modern machine learning ideas (https://www.climatechange.ai/CameraReady/43/CameraReadySubmi...).

Load forecasting is a reasonable thing. There are already enough vendors for that to where I wouldn't focus in that area personally, but if it is working for you, then that is good.

I read your linked paper last night and I'm not 100% sure what to make of it. If I understood it (big assumption :)) it seems like the intent is to use heuristics (Ex: particle swarm optimization) to seed a warm start for a nonlinear solver for the ACOPF problem? Is that right? Although the nonlinear ACOPF problem is academically interesting, the industry is using every last ounce of performance from the best MIP solvers (CPLEX & GUROBI) for the DCOPF unit commitment problem.

The 5-minute economic dispatch problem only takes a few seconds as it is formulated as a LP problem, however, the unit commitment process is a MIP problem though and I don't think the nonlinear solvers will have anywhere near the performance we need even with a warm start.

There is some interesting work being done on ACOPF (ARPA-E competition project that ASU is heading up).

Part I https://invenia.github.io/blog/2019/10/30/julialang-features...

Which of the 4 features did you find most interesting?

I like the convenient unit syntax, if this works well with IntervalArithmetic.jl, julia can become a competitor to Frink https://frinklang.org/

Here's an example of them working together rather effortlessly:

    using Unitful, IntervalArithmetic
    const m = u"m"
    const k = 2π/10m

    f(x) = cos(k * x)/(k^2 - 1/x^2)

    julia> f((10.0 ± 0.1)m)
    [2.5924, 2.60024] m^2

If I try to put in a dimensionally incorrect argument to f, I get an error:

    julia> f((10 ± 0.01)u"s")
    ERROR: MethodError: no method matching cos(::Quantity{Interval{Float64},𝐓*𝐋^-1,Unitful.FreeUnits{(m^-1, s),𝐓*𝐋^-1,nothing}})
    Closest candidates are:
      cos(::Float16) at math.jl:1085
      cos(::Complex{Float16}) at math.jl:1086
      cos(::Missing) at math.jl:1138
      ...

This isn't the most friendly error message in the world, but that can certainly be worked on and with a bit of patience it's pretty clear that it's saying that the problem was that it tried to compute the cosine of a unitful quantity which is nonsense.

_____________________________________

Unfortunately I had to use unitful intervals (i.e. (10.0 ± 0.1)m) instead of intervals of uniful quantities (i.e. 10.0m ± 0.1m) because IntervalArithmetic.jl demands that it takes real quantities, but Unitful quantities are subtypes of Number instead of Real.

Just as a followup, I just realized that the Measurements.jl package does know how to deal with non-real uncertainties, so one can instead do

    using Unitful, Measurements
    const m = u"m"
    const k = 2π/10m

    f(x) = cos(k * x)/(k^2 - 1/x^2)

    julia> f(10.0m ± 0.1m)
    (2.5989 ± 0.0014) - (0.0 ± 0.16)im m^2

Really useful examples; especially the data in rows versus columns example :)

Congrats lyndon we went to uni together dope

Reply on news.ycombinator.com