AEMO: Failed transmission and wind farms triggered SA blackout

From the AEMO:

The Australian Energy Market Operator (AEMO) has today published a preliminary operating incident report into the South Australian state-wide power outage (referred to as ‘SA region Black System’) that occurred at 16:18hrs Australian Eastern Standard Time (AEST) on Wednesday, 28 September 2016, during severe storm activity in the state.

AEMO has prepared this preliminary report as part of its review of power system events, under the National Electricity Rules (Rules). The observations noted in this initial report are based on early data and information up to 9.00 am, Monday, 3 October 2016.

The preliminary report explains how severe weather moved through South Australia on the afternoon of Wednesday 28 September 2016, with high winds, thunderstorms, lightning strikes, hail, and heavy rainfall. The weather resulted in multiple transmission system faults including, in the space of 12 seconds, the loss of three major 275 kV transmission lines north of Adelaide.

Generation initially rode through the faults, but at 16:18hrs, following multiple faults in a short period, 315 MW of wind generation disconnected, affecting the region north of Adelaide. The uncontrolled reduction in generation increased the flow on the main Victorian interconnector (Heywood) to make up the deficit and resulted in the interconnector overloading.

To avoid damage to the interconnector, the automatic-protection mechanism activated, tripping the interconnector and resulting in the remaining customer load and electricity generation in SA being lost. This automatic-protection operated in less than half a second at 16:18hrs and the event resulted in the SA regional electricity market being suspended.

While the event was triggered by extreme weather, AEMO will conduct a thorough investigation into how each component of the electricity system responded under these circumstances. In an ever-changing environment, AEMO’s ongoing responsibility continues to be to safely and securely manage the power system and wholesale electricity markets to deliver energy security for all Australians.

AEMO will continue to analyse the events of last Wednesday and will provide a further update on Wednesday 19 October. AEMO expects that a detailed report on the SA region Black System, including recommendations, may take up to six months to complete.

And the sequence:

qwert

It appears there were both wind farm and transmission failures:

WFD

Before this:

argwq

Full report.

David Llewellyn-Smith

Comments

  1. The wind farms have to trip out in high winds. The cut-out wind speed of a wind turbine is one of the design variables. It’s part of the generator protection mechanism.

    This could easily have been prevented by modeling the wind speed into the grid control strategy to bring additional thermal-generation capacity when high winds are expected.

    • From my reading it wasn’t wind speed that shut them down it was frequency drop caused by the supply dropping too quickly in relation to the load from the falling towers.

      I think that in the future load is going to be shed much quicker so less likely to be system blacks but more likely to be localised power outages. The Heywood interconnector also might need to be upgraded again, or a high capacity HVDC connection coming in from NSW

      • No, the protection systems on those transmission lines tripped. They were taken out of service and everything was initially fine.
        It was the reduction in wind generation which saw Heywood interconnector flows (from Victoria) increase to 800-950 MW tripping it off line and seeing a catastrophic gap between generation and demand = frequency separation and system black.

      • T-7s and you’re saying that it was the wind farms stoping due to excess wind rather than frequency drop?

    • So let me understand this from your perspective:
      Wind power is known to be hugely variable and likely to suddenly shut down when high winds occur. It is therefore the responsibility of the more reliable and controllable baseload / gas turbine power generation to stay on standby patiently waiting for a possible sudden shut down of the wind farms. Of course they (the thermal power generators) are excepted to provide this service out of their love for grid reliability.
      All that I can say is that You live in a strange world if this sounds like standard business practice.
      Maybe someone else looking at the same facts would conclude that Wind power operators should not have bid to provide any power if they couldn’t deliver it reliably, the wind farm operators should have voluntarily and systematically withdrawn their capacity in a planned manner when they realized that a forced shut-down was probable.
      Same facts slightly different conclusion.

      • Yeah, and the Fukushima nuclear plant shouldn’t have bid to provide any power to the Japanese grid if it was unable to withstand an earthquake.

        No generator should bid unless they can promise to survive a magnitude 12 earthquake, record-high solar activity, a bushfire, followed immediately by regional flooding, and a meteor strike. At night.

      • *Of course they (the thermal power generators) are excepted to provide this service out of their love for grid reliability.*

        They could be paid to be stay on standby to provide system security services. Ancillary services of this kind are a standard part of all electricity networks.

      • CB too many avoid the facts on this issue – Climate Institute is upset with is preliminary report – doesn’t fit their spin.

      • Wind power is known to be hugely variable and likely to suddenly shut down when high winds occur. It is therefore the responsibility of the more reliable and controllable baseload / gas turbine power generation to stay on standby patiently waiting for a possible sudden shut down of the wind farms. Of course they (the thermal power generators) are excepted to provide this service out of their love for grid reliability.

        Sounds to me like both need to be managed by one entity with grid reliability as its goal.

        (Assuming grid reliability *is* the objective, of course.)

      • @Peachy, Trust me I’m not against Wind power, I’m all for it BUT it’s the job of the Wind power providers to manage their resources in such a manner as to maintain grid reliability. If this means running at half capacity than that’s how they should operate, if it means providing additional backup through Batteries or by other means than that’s what they need to do. Maybe it just means that they should pay extra to insure that spinning reserves are kept ready to take over at a moments notice or maybe they should manage it through demand side contracts (lots of possible solutions, their job is to find solutions that match their needs)….In this case it appears that there was plenty of reserve capacity all spinning and synchronized and ready to fill the wind shortfall the problem was with the transmission failures.
        @Skippy The electricity grid is often considered to be the largest single machine ever constructed. It has no way to store electricity so every Watt generated must be consumed, in past decades most of this power was provided by base load power stations that exhibit extremely high levels of reliability AND fail in a known manner. This means the fail safe aspects of the Grid were developed side by side with the primary generation equipment. The Grid was developed as one large complex machine. At some point in time it was decided (Politically) that competitive Generation would replace collaborative generation and that distribution must carry anyone and everyones power whether they need it or not. Only a Politician would suggest this as a solution.
        Good we’re here now…looking forward we need to understand how the new systems (with high levels of renewables) can be operated to achieve the same reliability as the power systems that they replace. I know that Technically it’s doable…but at what cost..
        Unfortunately if grid operators fail to address growing reliability concerns than they’ll actually reduce the Value of the system that they’re indirectly selling. and thereby force their most valued customers to find alternate power solutions.

      • China-bob…

        The so called political decision was informed by the ideological economics at the time, its been a huge failure, Enron and on going similar cases – Chesapeake, and lets not forget the recent bush fires because of poor line maintenance, which imo is a result of incentives wrt buffing equities for bsd remuneration.

        I would also add their is no corner hardware shop for a lot of the medium to huge sized gear for these applications, its bespoke. I remember doing the massive transformer for the Sydney Olympics due to terrorism fears and no replacement.

        It gets a bit wonky when your engineering monkeys are in the Bahamas or India for some cattle call staffing start up and have zero applied knowlage or industry experience – see Port Waratah 3rd phase expansion, I was up to 30+ revisions just after half way and pulling up independent Q&A inspectors for incorrect procedures.

        Disheveled Marsupial…. lovely to the compliance staff slowly get the flick after the half way point, get some malleable grunt to sign the paper work so he can sit in aircon…

      • @Skip, don’t get me wrong I personally believe that the Grid as we know today is a dead concept these and other related costs will drive it out of business. The national grid will be replaced by local micro-grids each with their own generation, storage and demand management systems. The larger national grid will become an interconnected mass or micro-grids each separable, each independently operable. For this to happen our basic concepts of Electricity availability will need to change. consumers will need to adapt their behavior AND understand / accept the limitations of the new concept micro grid.
        Unfortunately with PV available at costs of 50c/W all forms of daytime baseload electricity generation are doomed PV will when available will always be cheaper so the baseload will get driven out of business. What’s interesting with very high levels of PV is that storage efficiency becomes irrelevant…this opens a whole world of energy storage solutions from compressed air to at home Methane / methanol production/storage.

      • “will be replaced with local micro grids” – sounds libertarian! ??

        Also, that’s what I’ve been saying about telecoms since I first played with mesh networks over a decade ago…

      • Sorry mig monopolies are a libertarian feature… what your suggesting almost sounds socialist

        Its a distinct possibility in some regions or locations china-bob, thou I would say it would take decades to implement on any significant scale, not to mention all the extra resource and energy expenditure to kit out. Considering currant global economic head winds one would easily imagine reduced demand compounded by the increases in product efficiency.

        I would also add in such weather conditions which occur over such large areas, any man mad system is going to have high fail rates [the cyclone that hit Qld for instance] . Australia’s issue has always been its unique features and weather compounded by distances and a population of only 22ish million or less back in the day.

        Disheveled Marsupial…. lol people better get used to weather disruptions increasing….

    • There is no additional thermal capacity in the SA pool which was available to bring on. Northern power station at Port Augusta shut down a few years ago because zero cost wind made it un-competitive. Now SA has a resource adequacy problem because the rest can’t meet demand without the interconnector.

      Not enough dispatchable generation.

  2. Great. To make matters worse, base load power generation remains on standby for when the wind is not blowing so as to be able to spin the very heavy wind turbine blades so that they do not flatten the bearings that help to keep them spinning. i.e. when the wind is not blowing, wind generation is a negative for the grid. Oh dear.

    • Ive seen the turbines in NSW being idle (not spinning) on calm days before. Is the requirement to keep spinning them a universal thing or peculiar to SA?

  3. This is AET time, in my world shit happened at 15.45ish (yes, my world is 24hr format based. :P)..

    Australia needs a national grid and a debate that starts with accepting that what SA is trying is to be commended but that there are weaknesses that need to be fixed. But no, debate in this country will always end in “See, change is bad!”.

    • “Change is bad!”

      Seems fair, given they’ve just proven that diversifying generation sources decreases reliability.

    • It is a sign of the times. Reasoned debate has given way to one-line slogans and screaming; emotion over logic; short-term gain over long-term stability. People are just slightly smarter apes.

    • Especially when a substantial part of our economy relies on the export of coal and gas. Good ol’ reliable coal and gas…

  4. That is a lot of transmission lines to go down at once. It is almost as if there was there a massive storm or something.

  5. Tassie TomMEMBER

    What bullshit that wind power had anything to do with this.

    SA wasn’t generating anything at all during the blackout. Not Torrens Island – nothing. If Torrens Island was generating something and sending it anywhere then there wouldn’t have been a statewide blackout, would there?

    Wind power has protected SA from things like this if anything by decentralising the generation and transmission network.

    If there was no wind power there would still be Port Augusta. Given that the high-voltage power lines connecting Port Augusta to Adelaide were bent over and on the ground, I’d say that (in the parallel universe with no wind power and Port Augusta still burning lignite) Port Augusta’s power output would have been instantly lost to the grid and an even bigger surge from Victoria would have occurred.

    The real question is: Why is there a rule that a power surge on the interconnector forces the whole state into a blackout even though there is still plenty of generating capacity (although not enough for everyone) in the state without the interconnector?

    • Umm are you sure Torrens Island wasn’t generating????

      Torrens Island, Pelican Point, Dry Creek etc all combined are not enough to meet demand. So when the wind cut out and the interconnector tripped and was a generation shortage which saw the whole system (including all generators) shut down to prevent damage to the physical power system core.

      generation must equal demand at all times or the system crashes. It is a pool.

      • Tassie TomMEMBER

        You’ve just answered your own question – yes – Torrens Island was not generating. That was on the AEMO site at the time, it was on the reneweconomy.com.au site – it was not generating. I was watching it closely.

        Have you ever heard of “rolling brownouts”? That’s what they term the really hot summers’ days when there is not enough supply to meet demand – so they turn a quarter of the city off for two hours or so, then turn them back on and turn another quarter off.

        Rolling brownouts would have been the obvious solution if there was some but not enough power because SA was cut off from Victorian power. This would not have spoiled supermarket food or frozen lead smelters or anything as damaging as that. That’s the big question here – why were there not rolling brownouts – why was there a total transmission failure for 10 or 12 hours?

        BTW – wind power has nothing to do with any of this.

      • We are talking about the loss of 500 MW capacity on the Heywood interconnector from victoria.
        This ain’t small Potatoes for a pool like SA which is small as it it.
        Rolling brown outs/black outs are no use when when you have such a mismatch.
        This is a different level event.

      • Tassie TomMEMBER

        That’s an unreferenced, ignorant, and plainly wrong statement. What capacity do you have (apart from being an LNP voter) to make blanket conversation-ending statements like that?

        Editing my comment – I see that in a tongue in cheek way – you are completely correct. Wind was indeed a major part of this – wind blew down the transmission lines which caused the computers to turn of SA’s power for almost half a day.

        I apologise for my above forceful rebuttal if your intention was a double-meaning all along.

      • This is actually my bag. I am a specialist in this area.

        Just read the article. it says it all.
        Wind generation cut out and there was insufficient capacity on the interconnector.
        There was frequency separation and the interconnector tripped.
        Because the wind cut out (due to the high wind speeds) there was insufficient generation to meet demand.

        Just read the article.

  6. It seems from what i can see written above that no-one seems to know much technically about this. I dont really either, but it all reads as due to interplay of surge protections of various parts of the circuit responding to transmission fails. The unfortunate endpoint of 900mw trying to go down a 600mw line is likely to be a problem, no matter what the technology.

    Of course those transmission towers didn’t fall over on their own it was indeed windpower that felled them 🙂

  7. Nuclear power!!

    SLX will singlehandedly solve this mess. The SA government just needs to award a $100b contract.

  8. Any reason we couldn’t stop using the Snowy and Tassie hydro systems for baseload? Keep the dams as full as possible and then switch them on when renewables dip below total demand?
    How much power can they produce at maximum? Could they be upgraded to produce more?

    • Huh, The interconnector from Victoria tripped.
      How are you going to get Snowy power into the SA pool?

      • Not so much as a way of preventing this particular event, but just as a way of countering the variability of renewables and thus reducing grid instability.
        Just having a few late night mental meanderings after a very long day at work.

  9. Comprehension skills are lacking chaps/chapettes

    “Generation initially rode through the faults, but at 16:18hrs, following multiple faults in a short period, 315 MW of wind generation disconnected, affecting the region north of Adelaide. The uncontrolled reduction in generation increased the flow on the main Victorian interconnector (Heywood) to make up the deficit and resulted in the interconnector overloading.

    To avoid damage to the interconnector, the automatic-protection mechanism activated, tripping the interconnector and resulting in the remaining customer load and electricity generation in SA being lost. This automatic-protection operated in less than half a second at 16:18hrs and the event resulted in the SA regional electricity market being suspended.

    Fun to see so many here try the spin – sorry boys leave that that to the experts 😉

    Initiated by a bit of wild early spring weather…renewable based system just can’t cope.

  10. Xo,

    Isn’t that saying “…….following multiple faults in a short period 315 MW of wind gen disc…..”, the wind gen disc BECAUSE of multiple faults?

  11. The loss of transmission was 3 times as much as the loss of wind farm power.
    Wind farm power during high winds is therefore worthy of analysis but it is clear that the system was brought undone by loss of infrastructure that couldn’t cope with the winds and changes in demand on the interconnector.
    It seems all 3 areas are worthy of considered, and there are a few others as well
    1. analysis wind power phase out in high winds using differing cut out wind speeds so it doesn’t all go at once.
    2. review of transmission infrastructure
    3. upgrading of connector.
    4. failure of standby generation to kick in at sufficient generation for emergency power is also worthy of analysis
    5. the risks of having to transmit huge power loads over long distance are also worthy of analysis compared to much more local generation

    It is worth noting that houses with solar power not damaged in the storm were among the first to gain some power and that batteries would have provided at least an orderly close down (as they do in most IT installations) or a long period of very reduced supply for critical purposes eg medical and maybe refrigeration and some LED lighting.

    Of course, the storm is also the perfect example of an effect forecast by climate change scientists for years – increasing frequency and intensity of major storms so a review of Australia’s commitment to fighting climate change is also required, and to some extent as a wealthy country with very high per capita emissions we ought be at the forefront of individual national efforts to reduce emissions.

    Big wake up call for a risk analysis by other states too.

  12. @ChinaBob.
    Windpower doesn’t provide standby thermal costs or storage costs. Those costs are external.
    Fossil fuel powered thermal power generators also have external costs they don’t pay. The main ones are air and water pollution and of course climate change. You could also go back through the fossil fuel supply chain to coal mines and fracking and find other huge external costs.
    External costs of the fossil fuel sector ought not be ignored when criticising the failure of the clean energy provider to pay what in the current system are their externalised costs.
    some external costs are most born by those immediately ajacent to coal fired power stations such as the effects of mercury emissions, others are diluted over time and space so they are barely noticeable at one point in time by most people, eg coal fired power stations in the 50’s before climate change and some health effects were obvious on any scale, but we now see a lot of those externalities coming home to roost and a lot more a re forecaast to have major economic and social effects including in Australia if climate change is not stopped.

    • I agree 100%, I’m not ideologically for or against any form of power generation, to be honest in the past it was a boring field of engineering that I didn’t concern myself with. In the last decade this boring industry has morphed and now presents some very interesting problems. I’m getting involved because I smell lots of opportunity for innovative solutions. I suspect that the solutions that are developed in Germany and the US will prove unsuitable for direct adoption in Australia and that also spells opportunity for anyone selling / providing / developing customized solutions that address some unique Australian problems. Personally I’m not bought into the longer term viability of the electricity power industry as we know it, but that doesn’t mean I can’t profit from the changes that will/must happen. To a large extent those that have grown up with the power industry have continuously underestimated the growth of the Renewable sector, put simply they don’t understand it, they see grid management problems that never existed in the past and so their naturally tendency is to wind the clock back, unfortunately life doesn’t work that way. This is precisely the thinking creates highly disruptive events which can completely displace the incumbent supply chains.

  13. If the initial cause of the black out was the system over loading and a switch getting tripped, how long would that have taken to fix? Is it a simple matter of resetting it? Would that have taken minutes, hours or days?

    The tripped switch may have been the initial cause but twenty two downed high tension power lines would be a damn good reason for the black out lasting days.

  14. occurred at 16:18hrs Australian Eastern Standard Time (AEST) on Wednesday, 28 September 2016

    Check out the weather just after 16:18hrs Australian Eastern Standard Time: http://www.theweatherchaser.com/radar-loop/IDR642-adelaide-buckland-park/2016-09-28-06/2016-09-28-07

    Great weather for transmission lines and wind farms!

    You would almost think the system was designed to fail by setting up wind farms dotted along the transmission lines linking Adelaide and Port Augusta which just happen to often line up accurately with intense cold front storm lines like the one on 28/9/16. Result: lots of electricity assets can get knocked out simultaneously by just one weather system.

    The message for wind system designers: don’t depend on generators all set out in a line that coincides with storm lines, especially if that line also coincides with transmission lines. It’s obvious now but the system suffered from a serious flaw in geographic diversity.