On the early evening of Wednesday, February 8, electricity supply to some 90,000 households and businesses in South Australia was cut off for up to an hour. Two days later, all electricity consumers in New South Wales were warned the same could happen to them. It didn’t, but apparently only because supply was cut to the Tomago aluminium smelter instead. In Queensland, it was suggested consumers might also be at risk over the two following days, even though it was a weekend, and again on Monday, February 13. What is going on?

The first point to note is that these were all very hot days. This meant that electricity demand for air conditioning and refrigeration was very high. On February 8, Adelaide recorded its highest February maximum temperature since 2014. On February 10, western Sydney recorded its highest ever February maximum, and then broke this record the very next day. Brisbane posted its highest ever February maximum on February 13.

That said, the peak electricity demand in both SA and NSW was some way below the historical maximum, which in both states occurred during a heatwave on January 31 and February 1, 2011. In Queensland it was below the record reached last month, on January 18.

Regardless of all this, shouldn’t the electricity industry be able to anticipate such extreme days, and have a plan to ensure that consumers’ needs are met at all times?

Much has already been said and written about the reasons for the industry’s failure, or near-failure, to do so on these days. But almost all of this has focused on minute-by-minute details of the events themselves, without considering the bigger picture.

The wider issue is that the electricity market’s rules, written two decades ago, are not flexible enough to build a reliable grid for the 21st century.

Vast machine

In an electricity supply system, such as Australia’s National Electricity Market (NEM), the amount of electricity supplied must precisely match the amount being consumed in every second of every year, and always at the right voltage and frequency. This is a big challenge – literally, considering that the NEM covers an area stretching from Cairns in the north, to Port Lincoln in the west and beyond Hobart in the south.

Continent-sized electricity grids like this are sometimes described as the world’s largest and most complex machines. They require not only constant maintenance but also regular and careful planning to ensure they can meet new demands and incorporate new technologies, while keeping overall costs as low as possible. All of this has to happen without ever interrupting the secure and reliable supply of electricity throughout the grid.

Until the 1990s, this was the responsibility of publicly owned state electricity commissions, answerable to their state governments. But since the industry was comprehensively restructured from the mid-1990s onwards, individual states now have almost no direct responsibility for any aspect of electricity supply.

Electricity is now generated mainly by private-sector companies, while the grid itself is managed by federally appointed regulators. State governments’ role is confined to one of shared oversight and high-level policy development, through the COAG Energy Council.

This market-driven, quasi-federal regime is underpinned by the National Electricity Rules, a highly detailed and prescriptive document that runs to well over 1,000 pages. This is necessary to ensure that the grid runs safely and reliably at all times, and to minimise opportunities for profiteering.

The downside is that these rules are inflexible, hard to amend, and unable to anticipate changes in technology or economic circumstances.

Besides governing the grid’s day-to-day operations, the rules specify processes aimed at ensuring that “the market” makes the most sensible investments in new generation and transmission capacity. These investments need to be optimal in terms of technical characteristics, timing and cost.

To borrow a phrase from the prime minister, the rules are not agile and innovative enough to keep up. When they were drawn up in the mid-1990s, electricity came almost exclusively from coal and gas. Today we have a changing mix of new supply technologies, and a much more uncertain investment environment.

Neither can the rules ensure that the closure of old, unreliable and increasingly expensive coal-fired power stations will occur in a way that is most efficient for the grid as a whole, rather than most expedient for individual owners. (About 3.6 gigawatts of capacity, spread across all four mainland NEM states and equalling more than 14% of current coal power capacity, has been closed since 2011; this will increase to 5.4GW and 22% when Hazelwood closes next month.)

Finally, one of the biggest drivers of change in the NEM over the past decade has been the construction of new wind and solar generation, driven by the Renewable Energy Target (RET) scheme. Yet this scheme stands completely outside the NEM rules.

The Australian Energy Markets Commission – effectively the custodian of the rules – has been adamant that climate policy, the reason for the RET, must be treated as an external perturbation, to which the NEM must adjust while making as few changes as possible to its basic architecture. On several occasions over recent years the commission has successfully blocked proposals to broaden the terms of the rules by amending the National Electricity Objective to include an environmental goal of boosting renewable energy and reducing greenhouse emissions.

Events in every state market over the past year have shown that the electricity market’s problems run much deeper than the environmental question. Indeed, they go right to the core of the NEM’s reason for existence, which is to keep the lights on. A fundamental review is surely long overdue.

The most urgent task will be identifying what needs to be done in the short term to ensure that next summer, with Hazelwood closed, peak demands can be met without more load shedding. Possible actions may include establishing firm contracts with major users, such as aluminium smelters, to make large but brief reductions in consumption, in exchange for appropriate compensation. Another option may be paying some gas generators to be available at short notice, if required; this would not be cheap, as it would presumably require contingency gas supply contracts to be in place.

The most important tasks will address the longer term. Ultimately we need a grid that can supply enough electricity throughout the year, including the highest peaks, while ensuring security and stability at all times, and that emissions fall fast enough to help meet Australia’s climate targets.