September 1, 2015

Perhaps They Should Have Tested More - SunGard



Perhaps SunGard and/or BNY Mellon should have tested more


If you've been watching the Market the past few weeks, you may have been concerned by all the volatility and heavy activity occurring.

And if your funds' NAV (Net Asset Value) was reported through BNY Mellon, you may have been even more concerned to learn that the NAVs at the end of the day's trading were delayed.

That's apparently because an operating system upgrade by SunGard (which hosts the fund account platform of BNY Mellon) caused a malfunction - on both the production platform, and the backup platform!

  • SunGard hosts the fund account platform of BNY Mellon
  • A malfunction in this platform disrupted the net asset values (NAVs) of BNY Mellon mutual funds and exchange-traded funds
  • Impacted 66 of BNY Mellon's US fund accounting clients
  • For up to 1200 funds valued at over $400 billion, the reporting of the NAVs at the end of the day's trading was delayed.
  • The timing was especially poor, due to the heavy market activity in the days that followed.
  • The problems may have led BNY Mellon to calculate inaccurate prices during the market's volatility
  • BNY Mellon's clients were likely receiving imprecise information
  • Per SunGard "issue appears to have been caused by an unforeseen complication resulting from an operating system change."
  • SunGard said it followed standard testing procedure for the implementation of this change, but the update still corrupted both BNY Mellon's primary and backup environments
  • BNY Mellon's chairman and CEO Gerald Hassell said the full restoration of SunGard's platform "has taken far longer than any of us would have expected."
  • Dozens of companies rely on Bank of New York data to determine the correct value for their funds
  • Bank officials say the problem began on Aug. 24, just as U.S. markets suffered their worst single-day loss since 2011

Perhaps it was just bad timing. 

It's nice that SunGard followed their standard testing procedures, but perhaps they weren't adequate. And perhaps it's not a good idea to make changes to the Backup environment, until you are completely sure the Production environment is actually working correctly.

Perhaps SunGard (and perhaps even BNY Mellon) should have tested more.

See also:


This article originally appeared in my blog: All Things Quality
My name is Joe Strazzere and I'm an experienced Quality Assurance professional.
I like to lead, to test, and occasionally to write about leading and testing.
Find me at http://AllThingsQuality.com/.