Hacker News Reader

Comment by doctoboggan | Hacker News Reader

Our data pipeline looks like this:

hardware/PLC --modbus--> kepware --mqtt--> mosquito broker --mqtt--> mqtt2prometheustool --http--> Victoria Metrics

The mqtt2prometheustool is something we developed in house. I am looking at removing one or more of the above steps and using telegraf instead, as it can ingest OPCUA or modbus data directly.

We use excel files just as the output of our reporting tools. For analysis it's the standard python data science stack of pands/numpy/scipy. Most people work in Jupyter notebooks, and their tools are eventually moved to services in our k8s cluster.

Temp and voltage are the main "cell level" datapoints we collect. I don't think we have any vibration sensors at site now.

View on HN

I'm working in the IIoT domain too. Your workflow is interesting towards the end. Any particular reason, why you don't write it to some db like Timescale or Influx at the end without any prometheus conversion?

Victoria Metrics is a Prometheus compatible time series database and was being used before I joined the company. I haven’t had any issues with it so I didn’t see a reason to pull it from our stack. Are you saying Timescale or Influx can natively ingest MQTT messages?

Similar to your mqtt converter you can make them ingest via Kafka or some other adapter in between. EMQX broker can directly write into TSDB but did not research into others whether they can or not.

Is there any pros to use PostgreSQL for metrics in 2025?

Not the OTEL "telemetry" but sensors sending measurements from the field "telemetry". IT tech is, as usual, stealing perfectly defined engineering words and making them something else.

TimescaleDB is perfect if you also have relational data that you need to join with field data to the point that there no pros of using anything else for this use case, say you have 100000+ sensors and you need to group them by the customer site relations while aggregating per day statistics.

As someone working in the IoT space - why pay for kepware for something that can be done in a few weeks by a developer? Telegraf or even a bit of programming will save a lot of pain

I am actively exploring replacing kepware with telegraf. It seems very promising. Kepware was purchased and deployed before I joined the company. I think it's the defacto standard in the controls world and they never really considered anything else.

Do you have experience with modbus in telegraf? If so I'd love to chat for a bit to learn what you've learned.

It's just making mqtt messages from modbus is such a simple task that you can just write the software yourself. There are many intricate stuff in modbus that maybe does not translate too well for generic translators and where you want to have full control. For example messages over multiple registers, breaking interfaces between vender versions and all that legacy stuff.

Telegraf is also nice (but only used it for mqtt topics) but the same applies here. The functionality is fairly simple. In telegraf depending on your data your .conf file gets fairly large and has to be maintained. If you have your data model already in code it's fairly easy to just write it yourself and gain the simplicity of just using the classes you have anyway.

In my current stack the data ingestion both the initial data->mqtt and mqtt-> database/cloud is just small programs that share their internal data objects. It's very easy to maintain for a small team imo.

Does this include thermals on batteries? And how much power is spent on keeping them at the optimals? What about SoC/SoD figures?

Because without that the 20 year promise is bullshit.

I can sort of name ballpark figures for the above, the thing I can't get is how this can even approach profitable w/o hype and subsidies.

> Does this include thermals on batteries?

Yes, not just batteries, but we collect cell level temperature for all cells in all batteries.

> And how much power is spent on keeping them at the optimals?

We run an AC unit on every container to keep them cool. (Its in the NV desert so never any need for a heater)

> What about SoC/SoD figures?

We do compute estimates of SoC but as you probably know charge state isn't always easy to estimate. All we really know is voltage, c rate and time.

> I can sort of name ballpark figures for the above, the thing I can't get is how this can even approach profitable w/o hype and subsidies.

There is certainly risk involved with any investment. But when you are buying batteries at the scale we do the price is probably much lower than you are thinking. And if we do properly gather all the warranty data then the risk of loss on battery failure is minimized.

well.. okay.

NV looks like similar enough to mid-to-southern-africa where we did stuff.

an AC unit (6KW? 24KW?) (per what, TEU or double-TEU) doesn't look like something sustainable. but we had much less dense installs, so I'm not really ready to argue that

SoC vs thermals vs load/charge profile is very not-a-single-number, but when the battery banks suddenly start demanding replacement (+ african logistics ) one develops models for the monitoring dashboards quite fast indeed.

I still believe that 20 yr warranty is bullshit on any serious load cycle. But if the manufacturers are willing to swap them, then no problem of course.

Reply on news.ycombinator.com