News•October 28, 2013

Experts Say Sandy Showed Limits of an Accurate Forecast

Thanks to a vast network of data-gathering instruments on the ground, attached to balloons launched twice daily across the U.S., and orbiting satellites higher in space, along with the sophisticated computer models that run on supercomputers of ever-increasing speed, weather forecasts are more accurate than they have ever been, as was demonstrated with Hurricane Sandy.

A week before the “superstorm” devastated the East Coast, the models available to the National Weather Service showed that Hurricane Sandy was going to do something that no storm had ever done — slam into the New Jersey coastline at a nearly 90-degree angle, going from east to west. That track would bring a massive storm surge into New York City, New Jersey, and Long Island, a nearly perfect storm or worst-case scenario.

In this 3-D map of potential temperature, relatively cool air wraps around Sandy's core near the surface (purple and blue colors), while air parcels gain heat from moisture condensing into clouds and precipitation as they ascend through the storm’s core.
Credit: UCAR.

Yet Sandy also demonstrated the frustrating limitations of an accurate forecast. Despite the advanced notice and the huge threat it posed everywhere from Massachusetts to Maryland, the storm still killed 159 people, including 44 within New York City alone. Most of the New York victims drowned from coastal flooding.

Even now, nearly a year after Sandy helped reshape the weather enterprise in the U.S., a gap is growing between the capabilities of weather forecasters and the state of risk communications and emergency preparedness in the country.

The growing disconnect between what forecasters can do — predict a storm’s path and intensity well in advance — and how the public and officials use that information was driven home by Gary Szatkowski, a meteorologist-in-charge of the NWS’ forecast office outside of Philadelphia, which handles forecasting for the Jersey Shore.

More than any other forecaster, Szatkowski sounded the alarm for New Jersey residents in particular, gaining national recognition for his clear, consistent warnings and use of social media to reach the public. At one point prior to the storm, Szatkowski took the extraordinary step of sharing his cell phone number in a public briefing, inviting people to call him directly to complain if Sandy failed to live up to its billing (as Tropical Storm Irene had a year earlier) and they had evacuated their homes, rather than ignoring evacuation orders and risking their lives.

“If you think the storm is overhyped and exaggerated, please err on the side of caution,” he wrote. “You can call me on Friday and yell at me all you want . . . I will be very happy that you are alive & well, no matter how much you yell at me.”

Szatkowski said Sandy proved that an accurate forecast is not enough to avoid loss of life and massive damage from an extreme weather event.

“I do think Sandy raised the bar regarding weather forecasts and warnings, and more importantly, opened the door to some new questions,” Szatkowski said in an email to Climate Central. “I’ve been drawing the contrast between how accurate the forecast was and how the information was utilized (in some cases, not so well).”

In two well-documented gaffes, New Jersey Transit parked dozens of train cars and expensive engines in a low-lying lot in Hoboken that was forecast to flood, ultimately causing more than $120 million in damage. And in New York, officials had to hastily evacuate patients from Coney Island Hospital, Bellevue Hospital, and NYU Langone Medical Center, among other facilities, including patients in critical condition and more than 15 infants in neo-natal intensive care. Bellevue Hospital and NYU Langone Medical Center lost power when floodwaters from the East River breached an electrical substation.

“The best technical forecast won’t achieve the desired results if the threat is not understood or not prioritized correctly,” Szatkowski said. “Everywhere it flooded during Sandy was forecast to flood.”

“I find our risk assessment as a society is quite interesting,” Szatkowski said. “As a society, we are comfortable with recalling large quantities of food if something inappropriate is found in a bottle or jar. As a society, we are comfortable with closing down a bus or train terminal if an unattended bag or parcel is found. Yet as a society, with a killer hurricane coming up the East Coast, we struggle with evacuation decisions, and protection of expensive infrastructure.”

Post-Sandy Focus Is On New Warnings, Communications

Hurricane Sandy prompted a number of changes throughout the different corners of the weather community, including the public and private sectors. New emphasis has been placed on improving how severe weather warnings are communicated, and rolling out new forecast products to help the public better understand threats. In addition, more funding has been secured to help shore up weaknesses in U.S. computer modeling and satellite development.

If ever there were a weather event designed to be a forecasting success but a communications failure, it was Hurricane Sandy. The storm was complex, owing part of its DNA to a hurricane and part to an extra-tropical nor’easter, and its track had no previous analog that could be compared. It produced up to 3 feet of snow in West Virginia at the same time as it was flooding Lower Manhattan. Plus, it struck just 1 year after the underwhelming Tropical Storm Irene passed over Manhattan, leading people to be more skeptical than usual about the dire forecasts.

New York City Mayor Michael Bloomberg briefs the press about Hurricane Sandy on Oct. 27, 2012. On that day, he downplayed the threat posed by the potential storm surge, and did not order large-scale mandatory evacuations.
Credit: NYC Mayor's Office.

Take the storm surge forecasts, for example. They were accurate, yet they were delivered in a text-based format that made it nearly impossible for most people to understand their implications. The format of the surge forecasts is suspected as being one of the reasons why so many New Yorkers and Jersey Shore residents were caught off guard by the height and force of the storm surge.

At The Battery in Lower Manhattan, Hurricane Sandy's storm surge of 9.4 feet contributed to a 13.88 foot storm tide above the normal low tide level. That resulted in an inundation of between 4 to 9 feet above ground level, enough to flood all of the subway tunnels between Manhattan, Brooklyn, and Queens, wipe out homes in Staten Island, and flood neighborhoods like Red Hook in Brooklyn and the Rockaways in Queens.

Although the NWS accurately forecast the storm surge magnitude and timing, the agency provided little in the way of graphical guidance for people to envision the impact of storm surge on their neighborhoods. Sans that visual aid, most coastal residents had no way to understand what a particular storm surge height would mean for their homes.

“There are ways of getting this mapped more explicitly,” Louis Uccellini, director of the NWS, said in an interview. “We’ve got to get it down to the street level.”

Despite the communications shortcomings, Uccellini said the agency's forecasts and warnings saved countless lives. “We also feel that our forecasts and our messaging saved lives. That there were evacuations in areas that if they weren’t evacuated… there would have been many more lives lost.”

A post-Sandy federal review of the NWS’ performance found that the forecasts the agency issued, including the surge forecasts, were remarkably accurate, but they were not communicated in ways that made it easy for officials and the public to understand. The confusing surge forecasts may have been partly to blame for why New York Mayor Michael Bloomberg downplayed the storm’s threat on Oct. 27, with less than 48 hours to go until landfall.

“It is not expected to be a tropical storm or hurricane-type surge,” Bloomberg said at a press conference. “With this storm we're likely to see a slow pileup of water rather than a sudden surge, which is what you would expect from a hurricane and which we saw with Irene 14 months ago, so it will be less dangerous, but make no mistake about it there will be a lot of water and low-lying areas will experience flooding.”

Spurred by Sandy as well as Hurricane Isaac, which struck Louisiana in 2012, the NWS is developing storm surge warnings as well as mapping tools that will allow forecasters to bring the threat closer to home for the public.

A prototype of NOAA's new storm surge depth maps, which will be in use starting in 2014.
Credit: NOAA/NWS.

Jamie Rhome, the storm surge lead for the NWS, said that Congress appropriated funds for storm surge forecasting improvements in the wake of Sandy, but that much of the work had already begun before the storm. The $10 million in post-Sandy funds devoted to storm surge modeling will accelerate the development of these warnings and enable the NWS to deliver this nationally earlier than expected.

The NWS is also working to roll out experimental inundation graphics in 2014 and storm surge warnings in 2015. These warnings, and the maps used to convey them, are being designed collaboratively between forecasters and risk communication experts.

While the NWS and weather community more broadly work to develop better computer models and satellite systems, along with more effective storm warnings, the central lesson from Sandy may not concern any one particular technology or forecast product, or even a single satellite.

Rather, as David Bernard, chief meteorologist at CBS’s Miami affiliate, told Climate Central, the problem that forecasters face in future storms is that more progress needs to be made in improving risk communication and threat response.

“The meteorology is moving along but our ability to react and prepare is not,” Bernard said.

In other words, forecasters are already bumping up against diminishing returns, with improvements failing to translate directly into lives and property saved.

Before Sandy, the NWS was already beginning to work with social scientists to help redesign its forecast products and warnings. Post-Sandy, the agency has accelerated such efforts. “(It’s) not just physical science anymore,” Uccellini said of the job of severe weather forecasting at the NWS. “It’s physical science plus social science, if in fact we want people to take the response that we expect when we make the forecast.”

“Sandy has gotten academia, the private sector, and federal government to move beyond ‘lip service’ on social science, communications, and related research,” Marshall Shepherd, a professor at the University of Georgia and the president of the American Meteorological Society, said.

So far, though, the NWS has not hired any social scientists to work full-time alongside its meteorologists. Instead, the agency is training certain field experts, known as warning coordination meteorologists, to make them aware of the insights that social science research can provide regarding the weather warnings process. “This is going to become an increasingly important part of their job description,” Uccellini said.

In addition, storm surge specialists like Rhome are working closely with communications experts to design the storm surge warning products.

Bryan Norcross, the senior hurricane specialist at The Weather Channel who was on air throughout Hurricane Sandy, told Climate Central that the meteorological community needs to come to grips with what he says is the sorry state of risk management in the U.S.

“It has become frighteningly predictable what the research is going to say after every big storm: most people did not understand the threat. They knew the storm was coming, but they could not translate the avalanche of bulletins, advisories, watches, warnings, and opinions into a clear understanding of the threat to their home and their family,” he said.

Shoring Up Critical Weather Infrastructure Before the Next Sandy

By ripping apart pieces of the East Coast and causing billions of dollars of damage, Sandy moved the needle on the National Oceanic and Atmospheric Administration (NOAA) funding requests for both computer modeling improvements and satellite development.

The storm hit the U.S. at a critical time for the public and private weather sector. A successful weather forecast relies on products from observational systems, like satellites and weather balloons, as well as tools that scoop up that data and make sense of it, feeding it into sophisticated computer models that assist human forecasters in making a prediction.

Life cycle of Hurricane Sandy: Surface Wind Speeds. Credit: UCAR.

If any one of these three critical areas — observations, data assimilation, and computing capacity — are lagging, then forecasts could suffer, Uccellini said. Before Sandy hit, NOAA was growing increasingly concerned about its declining computing capacity relative to other nations, as well as its troubled satellite development program.

The agency faced a political climate that was toxic to any requests for funding increases, especially for programs that could be spun as “climate-related,” which earned the suspicion of many Members of Congress who are skeptical of the existence of manmade global warming.

Sandy provided NOAA with a powerful argument to use on Capitol Hill, where lawmakers had been harshly critical of the agency’s mismanagement of the satellite program, in particular, and largely unaware of the supercomputing lead other nations were opening over the U.S.

As they had done after previous major storms, NOAA officials went before lawmakers and showed them how much less accurate the forecast would have been had the computer models been deprived of satellite data. Kathryn Sullivan, the NOAA administrator, touted a study from meteorologists at the European Center for Medium-Range Weather Forecasts in Reading, England, in which they re-ran their computer model after depriving it of the data that comes from the current fleet of polar-orbiting satellites, and found the forecast would have been hundreds of miles off course.

The Hurricane Sandy forecast from ECMWF with and without the polar-orbiting satellite data.
Click to enlarge the image. Credit: NOAA

The test was done because cost overruns, launch setbacks, and bureaucratic delays in the U.S. satellite program threaten to cause a gap of a year or more between when one of the current polar-orbiting satellites — known as the Suomi NPP — reaches the end of its design lifetime in 2016, and when the next polar-orbiting satellite is ready for launch in 2017, at the earliest.

Partly as a result of that lobbying push, the continuing resolutions that provided funding for reopening the government following the 16-day shutdown included a provision for keeping satellite funding at levels that would prevent any launch dates from slipping further behind, and there was a similar provision in a bill that funded the government through Fiscal Year 2013.

The storm also publicly demonstrated the shortcomings of the premier U.S. forecasting model, known as the Global Forecast System, or GFS, compared to the European’s medium-range model. Meteorologists had known for years that the Euro model often outperformed the GFS, particularly several days in advance of a weather event, but lawmakers and the public were largely unaware.

The Euro model picked up the scent of Sandy’s track, including its bizarre left hook into New Jersey, sooner than the GFS model did, providing earlier warning to public officials.

It is no accident that Congress has since provided critical support for boosting the agency’s computing capabilities. The NWS is using $25.2 of the $48 million allocated to the NWS in the Sandy relief legislation, along with funds called for in President Obama’s fiscal year 2014 budget proposal, to bring about unprecedented computing upgrades.

“We just tripled the size of our operational computer,” Uccellini said, adding that in a year and a half, the agency will add another 10 times the current amount of capacity. “The Sandy supplemental allowed us to break through the glass ceiling,” Uccellini said of the NWS’ computing capabilities. “I thank the Hill every time I get the chance for supporting that.”

The new money will allow the NWS to go from an operational computing capacity of 213 peak teraflops at the end of the current fiscal year, to 1,950 peak teraflops by the end of fiscal year 2015. A teraflop is a measure of how many calculations a computer can make per second, and indicates that a computer can make one trillion “floating point calculations” in just one second.

But as Sandy sadly showed, all that computing power doesn’t mean anything if the message isn’t communicated well and heeded by public officials and the people in harms’ way.