Would More I.T. Governance Helped McAfee’s Global Software Glitch?

The news of McAfee's global security update gone awry has been the target this week of what is being widely dubbed as a computer glitch.  Apparently, its antivirus update program froze computers by misidentifying a common Microsoft Windows file as a virus causing the computers to keep rebooting.
 
The impact is being felt in businesses of nearly every type.  Over 1,000 Coles Supermarkets throughout South Australia, which is about 10 percent of the chain, were shut down due to the glitch that shut down the cash registers.  In Rhode Island, several hospitals had to turn away non-trauma patients and reschedule elective surgeries.  In Lexington, Kentucky more than 1,000 government and school computers, including those at police stations, were shut down due to the glitch.
 
Not surprisingly, McAfee addressed the technical issue within a few hours and their public relations team was in full force to clarify the issues behind the glitch that solely affected computers running Microsoft Windows XP with Service Pack 3 installed.
 
While I suspect distributing the global patch to subscribers is more than a routine task at McAfee, I have to wonder what went wrong this time.  It's an interesting question from the standpoint of I.T. governance.  How is it that a glitch can appear seemingly out of nowhere in a process that's done on a regular basis?  The folks at McAfee outline the details here.  However, the end users are still up in arms about the glitch.
 
It appears that McAfee did all the right things to recover from the public flogging associated with this glitch.  Behind the scenes, I'm going to venture a guess that there's a very detailed analysis and review of how exactly that glitch made its way out to the global customer base.  And I'm going to wonder just how much IT governance was in place and how it may have potentially avoided the issue altogether.

Happy (Glitch-Free) Tax Day

There's nothing like a tax filing deadline to make us all aware of our dependence on technology.  After all, the IRS reports that online tax filing is on the rise with 95 million people having used their e-file system in 2009.  According to an article in USA Today, the total number of electronically filed, self-prepared returns was up 6.7 percent from the same period a year ago. 

So what does the rush to the tax filing finish line have to do with a blog about technology?

Well, along with reminding us about the onslaught of network traffic, there are several software glitches to be aware of.  The folks at Intuit's TurboTax were hit with a software glitch that recently double counted a tax payer's medical insurance premiums as deductions resulting in a claim for a refund that was $600 too large.  I have to think this glitch is part of the software code that resulted from a lack of IT governance and that this individual tax payer -- who, by the way, returned the money -- is not the only one who will be affected.

TurboTax is not the only one dealing with a glitch.  H&R Block's system has not been modified to address the filing extension deadline of May 10 for those New Englanders affected by the torrential rain storms and subsequent flooding last month.  Seems like a harmless glitch at first but as BostInnovation reported, for taxpayers that owe money and are seeking payment plans, the difference between an April 15 filing and one on May 10 can be significant in terms of interest and fees required to take advantage of those payment plans.

Of course these glitches add up in terms of financial losses from taxpayers as well as the time and money that's required to fix them.  Just look at the major glitch occurred in March 2007 at the Canada Revenue Agency when a software patch that was supposed to prevent computer failures had the opposite effect and prevented thousands of taxpayers from filing electronic returns.  After some heavy pressure from the Canadian press to get at the truth of the matter, it was discovered just this past February that the more than 16,515 tax returns that were filed during a 43 hour 'corruption window' weren't filed properly. The cost to repair the glitch was $2.4 million.

As we become more and more dependent on technology, let's not lose sight of the fundamentals that make it easy and convenient to file taxes electronically.  One of the most effective ways to avoid many of these glitches is through a sound IT strategy that prioritizes IT governance so that IT departments aren't spending the rest of the year cleaning up the errors that could have been caught long before taxpayers alert vendors and government agencies to glitches in their systems.

Healthcare Bill To Challenge IT Health?

Whether you’re for or against healthcare reform, you have to admit it’s going to cause major shifts that will directly impact IT.  Of course, this doesn’t come as a surprise to anybody who’s even remotely involved in managing the infrastructure of a healthcare or health-related organization.  

For years, the healthcare industry has been dealing with a somewhat late arrival to IT adoption with regard to building and sustaining infrastructures.  And this certainly isn't a knock on the healthcare industry as a whole. It's just that  in order to get a full view of all activities in a healthcare organization requires a complex IT infrastructure that needs to support and connect all of the various departments and people in the patient life cycle.  

For example, a patient who enters the emergency room for a short visit can easily interact with admissions, the HMO, x-rays, doctors, nurses, and the pharmacist before the information is aggregated and linked to billing and insurance. Now multiply this complexity by the fact that the average emergency room in the United States sees 82 patients per day.

Now when you add in new government regulations -- like we saw with HIPAA a few years ago -- the massive IT undertaking required to adhere to the new healthcare reform can't be understated.

Even for those organizations like Kaiser Permanente and Independence Blue Cross that have built sound IT infrastructures that are based on a proactive approach to IT governance, the new regulations are going to require significant investments of time from both the business and IT groups.

On the upside, however, is the opportunity for the healthcare industry to leapfrog other industries and show how to simplify these complex back-end systems so that the end user -- in this case, the patients -- do not have their care interrupted due to shoddy IT practices.

Healthcare is certainly one area where preventative measures, including IT governance, can go a long way, especially as new government mandates kick in.  

Flying High with IT Governance?

There's nothing like a massive software upgrade mandated by the Federal Aviation Administration (FAA) to get my attention, along with a lot of other frequent flying IT folks.

In case you haven't heard or read in the Wall Street Journal, the FAA has mandated that airlines install new autopilot software in Boeing 777's within the next 90 days. The bottom line is that there were two incidents in January regarding autopilot software. It turns out that when crews inadvertently engaged the autopilot before take off, there was unusually strong resistance from the autopilot as they preparing for takeoff. This caused the pilots of both planes to remain on the runway for the safety of the passengers. Along with the autopilot software, this new "airworthiness directive" calls for additional software changes in the Boeing 777. 

Now I'm all for whatever makes flying as safe as possible. Though given the fact that there have only been nine of these types of incidents among the 4.2 million flights since 1995, I have to scratch my head and wonder if this overhaul is going to cause unnecessary glitches due to the integration and introduction of new software into a system that, for the most part, is working just fine. What's even more baffling is that these changes could affect more than 800 planes if these regulations are accepted overseas.

Given what we know about the propensity of glitches when new software is introduced without proper IT governance around it, I have to wonder if this new federal mandate will cause more problems than it will potentially solve.

What do you think? Does the new FAA airworthiness mandate inspire confidence in flying? Post your comments below or send an email to jeff@weblayers.com

Software Glitches Will Take Down Your Company

Now I know that headline may sound a bit dramatic at first but when you hear about the on-going issues at Toyota and you read a rather frightening New York Times article, "When Radiation Treatment Turns Deadly," about software glitches leading to accidental radiation poisoning, it's not a far fetched hypothesis after all.

Every day we keep learning about these software glitches that are not only affecting businesses but are also disrupting our personal lives and putting our health and safety at risk. Just this week, Alan Paller, director of research at SANS Institute was one of many pushing for dramatic change. He wrote in an email to the team at nextgov.com, "The only way programming errors can be eradicated is by making software development organizations legally liable for the errors. And that can only be done if there is a safe harbor." 

There is an industry wide movement currently underway to protect software buyers from being held responsible for faulty code. This news coincides with the annual "Top 25 Most Dangerous Programming Errors" list of the most widespread and critical programming errors that can lead to serious software vulnerability. The list is the result of collaboration among SANS Institute, MITRE and top software security experts in the US and Europe. In addition to the latest rankings, acquisition experts announced new standards for contract language aimed at protecting software buyers from being held responsible for faulty code.

According to Common Weakness Enumeration (CWE), a community-developed dictionary of software weakness types, these vulnerabilities are often easy to find and easy to exploit. Much like a most wanted list, the vulnerabilities on this year's Top 25 are cited as dangerous because they will frequently allow attackers to completely take over the software, steal data or prevent software from working at all. 

If ever there was a time to play closer attention to the role of IT governance, it is now.  With this in mind, I'm currently working on a webinar presentation with my good friend Joe McKendrick to specifically address how glitches can take down a company.  The registrations are starting to pile up and we'd love for you to join us on Tuesday, February 23 at 2:00 eastern to discuss and debate software vulnerabilities in the real world.  Here's the link.

Even the Governator Needs Governance

 

Toyota's Software Glitches: 2005 and 2010. Lack of IT Governance?

 

I can't imagine being in Toyota's shoes right now as journalists, customers and consumer advocates are demanding more information as to how exactly they built and shipped so many cars with faulty brakes.

This PR nightmare will continue for quite a while and Toyota's ability to recapture what was once a very strong brand is questionable.  There have already been some discussions as to whether they should change their name.

While the details continue to be rolled out, I have to wonder how that software glitch got into the system in the first place.

According to Toyota, the company changed its braking system software in January as part of what it called "constant quality improvements."  Company officials are describing the problem as a 'disconnect' in the vehicle's complex anti-lock brake system (ABS) that causes less than a one-second lag.  Now one second may not seem like a lot of time to you.  However, if you're driving 60 miles per hour, it will be about 90 feet or so before the brakes take hold.

Now you know where I stand on the importance of IT governance.  I'm also not going to pretend to know the specifics of the Toyota situation beyond what you've likely read.  However, the issue does call into question the broader issue of IT governance and the level that is applied in the software development process at Toyota. 

This is not the first time that Toyota has been hung up by a software glitch.  In 2005, CNet reported that a software glitch was affecting the performance of the Prius.  At that time, Toyota asked 75,000 owners of Prius hybrids to have the vehicle's software checked.  Apparently, a software glitch causes the warning light to come on for no reason and in some cases shut down the gas engine.

This story will continue to unravel in the coming days and weeks though perhaps we shouldn't isolate Toyota as glitches are ubiquitous.  It's the lack of IT governance that may make the difference in the way that the public views and consumes your product.

Could IT governance have prevented Google’s Issues with China?

 

By now, I assume that everybody in the industry is aware of Google potentially ceasing operations in China.  Given the cyber attacks on gmail accounts that were supposedly initiated by the Chinese government, it's clear that this story will continue to play out over the next several months. We've already heard from Secretary of State Hilary Clinton stating that the issue has "raised very serious concerns"  along with other daily reports on the topic.

While there are many different ways to view this evolving story, the piece that I find most troubling is today's news that indicates that Google insiders may have aided the Chinese government in hacking the gmail accounts of human rights activists.

The tech crowd is pointing to vulnerabilities in Internet Explorer and there is a solid argument about the ability to use IE to hack into the gmail accounts.  However, blaming Microsoft technology for the security compromises is about as productive as blaming the gasoline after the arsonist has set the fire.The issue that this whole Google China incident raises in my mind is the role of the government in the Internet and where IT governance intersects the two.

Now I don't expect nor do I want any government to step in and start overseeing the usage of the Internet and search engine results.  Yet with the latest news that the cyber attacks are being traced back to insiders, it calls into question the amount of IT governance that was in place. 

Could governance have prevented the attacks?  No, as that's a far-fetched and unrealistic claim.  However, with the right amount of governance in place, rogue applications and activities may not have been allowed to infiltrate the infrastructure.  In this instance, it may have been able to alert Google China's managers to potentially compromising behavior before it put the entire operation at risk including the 700 employees in the country. 

If Google does cease operations in China, the implications will be pretty far reaching given the search engine giant's presence in the country and its recent expansion into music and the mobile device market.  It would be a shame for the company to lose the momentum it's been building in China over since 2005 yet it would be an even bigger shame if it was forced to compromise it's "Do No Evil" mission statement.

I'm going to keep watching this story, as there's likely to be more that unfolds.  Meanwhile, I can't help but wonder how much of a difference governance would have been able to make in minimizing the impact of the situation.

2010: The Year for Governance Beyond IT

Like everybody else at this time of the year, we look backwith the hope that perhaps we've learned something in the past 12 months.  And if you're like me, we're alsolooking forward in anticipation to a new decade.

 The past year was certainly an interesting one and we havehopefully gotten through the worst of one of the worst recessions I canrecall.  We were also witness tosome major industry shifts including Oracle's purchase of Sun,Microsoft's launch of Windows 7 and Twitter's ability to bea viable source for breaking news.

When I first started blogging last February, the talk around the water cooler was whether or not SOA was dead and when the economy would fullyrecover.  The more things changethe more they stay the same.

As I thought about some New Year's resolutions, I started tothink about how we could all use a little more governance across theboard.  Not just for ourinformation technology infrastructures but across all other aspects of ourindustry such as media, the blogosphere, the analyst community, and our overallapproach to mitigating all the potential risks in our business. 

So here are three resolutions I've decided to adopt.  All are relatively easy and are probablynot too far from what you're already thinking about as next year approaches. In 2010 I resolve to do the following:

1.    In a recent conversation with industry analyst Dana Gardner from Interarbor Solutions, he aptlypointed out that you really can't successfully engage in cloud computing unlessyou have a sound SOA infrastructure so the notion that SOA is dead, is, well,dead.

2.    Tocarefully evaluate every trend that seems to catch fire in the blogosphere andassess it on its own merits with regard to industry relevance and what actuallyconstitutes news.  It's far tooeasy to read headlines without the actual story or take so-called news items atface value simply because they appear online.  It seems to me that the blogosphere could use a set ofgovernance policies to better mitigate the risks of inconclusive reporting.

3.    Along those lines, I resolve to also pay closer attention to critical issues andtrends that will have far reaching effects on our IT infrastructures.  Infoworldactually did a great round up on "The Top Underreported Tech Stories of 2009" citing the issues around the wireless spectrum and broadband availability as well as the dark side of cloud computing and its legal ramifications along with eight other under reported stories that will most certainly be part of our conversations in 2010.

Ah, three simple resolutions and none of themrequire that I restrict calories. If you have some resolutions to share, drop me a line at:jeff@weblayers.com        

Computer glitch delay your unemployment check?

 

The latest computer glitch comes from the California Employment Development Department. Due to the system's inability to record extended unemployment benefits, over 100,000 residents have not received their checks and may not until after December 25.

Recognizing back in 2002 that the 30-year old computer systems were in need of a serious upgrade, the federal government gave the state of California $60 million to address the current and anticipated issues. While some progress has been made, it's clear that the project is not complete.

Of course, this isn't what unemployed residents want to hear when their bills are mounting and the holiday season is upon us.

So how do you explain to little Suzy or Bobby that Christmas may be delayed? Articulating the complexity behind the computer glitch may be a bit heady for some parents so here are five excuses to tide you over until the unemployment checks clear.

1. Santa decided that this year that he would split up his trip into two nights and you're on the list for the second delivery that is expected after the 25th, or somewhere thereabouts.

2. There was a layoff in Santa's workshop and with less elves working, they weren't able to make as many toys as they used to.

3. The reindeer caught the H1N1 virus and won't be able to fly until later this month when they're feeling better.

4. Santa's credit card was frozen in the North Pole and it won't thaw until the end of the month.

5. Your email to Santa got stuck in a spam filter.

While the California residents patiently wait to see if the glitch does in fact delay their checks for another few weeks, the issue calls into question the flexibility of the IT infrastructure. You have to wonder how much governance has been applied to the system design and why such a slight change such as the extension of benefits could trip up so many.