Of course, our business is still running on nightly reports. But the tech is cool!
So I wanna say you're wrong, but I've got man-years invested in a system that hasn't been utilized to its full potential yet :(
Time will tell. Somewhere, some competitor will be using real-time insights to out-compete us.
Why? Can a business mobilise in anything less than days? If a report is minutes out of data, is that any loss? Given that some largish proportion of reports are never used, perhaps better management is key.
not disagreeing but efficiency is not just a matter of quickness.
This generally has implications on data quality, because you aren't fixing data quality issues once they occur in prod.
This also makes it far easier to develop ETL pipelines. You don't need complex tooling to see whether your ETL pipeline works.
You can technically fix data quality and dev velocity issues without low data freshness, but a quick glance at the data engineering landscape tells you they aren't being solved enough.
Quite simply, the end goal is increased oil production and reduced power consumption. Realtime data flows are a big deal when reading from thousands of sensors for just one process system and there are hundreds of tunable parameters. All this has to be safe also.