Bespoke software for the energy market
how we revitalised Gas Industry Co’s D+1 allocation system
Like all exciting stories, this one starts with some procedural background context – bear with us or feel free to skip ahead if you’re familiar with the daily gas allocation process!
Gas Industry Co is New Zealand’s official gas industry body and co-regulates the industry with the government.
Energy Market Services (EMS) was awarded the service provider function to act as the Gas Allocation Agent on behalf of Gas Industry Co for the gas industry. The main function of this role is to allocate gas to retailers for the previous month, and to wash up other historic gas allocations, up to 13 months in the past. Allocating gas to retailers in a timely manner has always been a challenge because the majority of customer’ meters are not read until many weeks (and in some case months) after the trading day.
As a result, in 2015 Gas Industry Co developed a daily allocation system (D+1) as a proof of concept for the gas industry that successfully used regression analysis to produce two daily allocation runs to the transmission system owner (Firstgas) and its shippers. The D+1 system provided timely information using these regression estimates and has become embedded into critical industry processes.
Gas Industry Co designed the D+1 allocation process to provide retailers with a timely estimate of their usage.
D+1 uses known gas injection data (what came out of the transmission system) from the previous day, validated meter readings from large customers and modelling to forecast a retailer’s deliveries from the transmission system. This allows Firstgas to calculate each retailers’ mismatch position between their previous day’s nominations and their estimated usage.
This means that when retailers inform pipeline operators of how much gas they’ll consume, they can adjust the current day’s nominations to correct for over and under nominating on the previous day.
The D+1 system was a success as a proof of concept, but Gas Industry Co needed a robust, supportable and more secure solution to replace it – that’s where we come in.
After several years, the D+1 prototype had a proven track record and was widely supported by industry stakeholders.
But although the original D+1 system delivered accurate information for end-users, there were several backend inefficiencies that made it challenging and risky for operators to make necessary changes.
So, in June 2021, Gas Industry Co tasked the EMS team with replacing D+1 with a production-strength system.
Having worked with YouDo for many years, reaching out to our team was a no-brainer.
We got on the tools straight away and delivered a custom solution that provided the same accurate data to end-users, but with a backend framework that was:
- Robust
- Secure
- Automated
- Scalable
- Compliant
… put simply, it made life a lot easier for the operators!
“We analysed the existing prototype code base and its documentation and used these to develop the new D+1 system. We improved the quality of the allocation process by testing and implementing data checks throughout, introducing comprehensive data quality reports & alerts and integrating with the Gas Registry. We automated the system by integrating it with AWS Lambda functions, developed by Fronde and productionised the R models used for forecasting.”
- Greig Robertson, Director at YouDo
We caught up with Richard Rowell from EMS to discuss the D+1 project and the advantage of custom solutions in the energy sector.
EMS and YouDo have worked closely together for years, that must have made this project a pretty seamless process?
“There’s a lot of comfort in dealing with YouDo because of their knowledge in the energy sector and their track record in developing specific applications for us and our industry.
And with our long-standing relationship, we can just dive straight in!
There is a lot of trust from both parties which makes taking on a complex project like this work really well.
How did the teams work together on the D+1 system? Do you still have inhouse developers or is it purely outsourced?
“We don’t have software developers in house.
We’ve developed the EMS business so that we’re the subject matter experts – in terms of the logic, the way the system works, operating the markets or services and driving continuous improvement – we outsource software development and technical support to our trusted partners.
This means that our Business Analysts know exactly what the customer’s requirements are and can then translate that into the developer’s language so they can get to work.
We’ve been working like this for over 20 years – it’s now called DevOps.”
What did the testing and implementation process look like for the new D+1 system?
“A benefit of this project was that we could benchmark the new system against the prototype.
In developing this solution, YouDo and our team could test against the old system, identify issues that needed to be resolved but also to prove that the revitalised product is much better.
Implementing the new system was simple. Because it was cloud-hosted in AWS, the YouDo team could apply changes in a test environment, we could complete the testing then release to the production system very quickly.
The parallel runs meant we could prove day-by-day that the system was producing the desired outcomes and we could explain any differences when they occurred.”
How does the upgraded product compare to the D+1 prototype?
“The challenge we faced was that the prototype system grew over time, but the way it was put together wasn’t optimal.
It did the job but in terms of reliability, the client realised they needed something better.
The fundamental outcome was a robust, proven technology that was built fit for purpose.
And, of course, the wider market customers don’t notice a thing – it was a seamless switch.”
What would you say to decision-makers in the energy sector who are asking the bespoke vs. off-the-shelf question?
“The uniqueness of the NZ energy market means that there are no viable OTS products available, so they have to be custom-built.
Decision-makers may feel the temptation to just buy an OTS system, but once you factor in the cost of licensing, modifications, and ramp-time, it’s in many ways cheaper to build custom from the start.
For us, YouDo is a trusted pair of hands.
These guys have a deep knowledge of the energy sector and how to build a system to meet the needs of businesses operating within it.
That makes it so much easier for us because we have a trusted go-to provider whenever we’re looking to build robust applications.”
This project showcased our competitive skillset in the energy market.
It takes expert knowledge in software development AND the energy market to revitalise systems in this industry.
We’re passionate about fulfilling this need for businesses by capturing their knowledge and competitive advantage in their software.
YouDo has a proven track record of delivering exceptional solutions to energy clients, including:
- Building Contact Energy’s Dam Assurance Monitoring System
- Working with renewable energy leader, Neoen
- Developing emsTradepoint’s carbon trading platform
From scaling an MVP like D+1 to enhancing an institutional system, we’ll bring leading minds, the best tech, and a let’s-crack-on attitude to make sure you’re operating in a modern and secure environment.
