Posted by Jeff Papows on Mon, Mar 26, 2012 @ 08:30 AM
It takes a highly unusual event for a company to cancel its IPO on the first trading day. During the past thirty years, only the loss of a CEO or accusations of wrongdoing have led to the withdrawal of an IPO. But for BATS Global Markets, all it took was a software bug, which have become all too common in today’s world.
Adding insult to injury to the BATS withdrawal was the fact that it was the company’s own exchange that failed. “This wasn’t just any IPO. It was an IPO of a listing and trading venue,” said Diego Perfumo, Equity Research Desk analyst. The glitch and its consequences crushed investors’ confidence in the company, leading BATS Chairman Joe Ratterman to confess, “We’ve built our reputation on building world-class technology that delivers results. But in this case, we failed.”
The damage caused by one computer glitch impact not only BATS’ reputation but its market share, which dropped several percentage points on Friday, and ultimately, its financial stability. As company founder David Cummings states, “The software bug is probably the easiest thing to correct.” If only repairing the damage to the company’s brand and reputation was just as simple.
Should governance, code compliance, code validation and other forms of governance be a mandate within IT organizations? Let me know your thoughts.
Posted by Jeff Papows on Wed, Nov 30, 2011 @ 04:15 PM
Just in the last two days, computer glitches from coast to coast have caused havoc, concern and even a little humor for many:
Have software glitches become so pervasive that even Hollywood is getting in on the act?
Posted by Jeff Papows on Tue, Nov 29, 2011 @ 09:30 AM
For most people, a computer glitch such as the one that recently hit BlackBerry Bold 9900 phones may cause users a lot of frustration and companies to lose productivity. Annoying , yes; life-threatening , not really. But for some members of the California Public Employees’ Retirement System (CalPERS), a software glitch has created a bureaucratic and emotional nightmare that goes far beyond inconvenience.
CalPERS hired a leading technology consulting firm to consolidate nearly 50 archaic IT systems and create one new online system for its members. But rather than simplifying things, the new “my CalPERS” has caused nothing but fear and stress. A glitch in the new system resulted in insurance policy cancellation letters being incorrectly sent users to some members, and delayed the mailing of death benefit checks to widowed spouses. While the company stresses that members will not go without medical coverage due to the erroneous correspondence, some members are doing just that until each case can be resolved.
And it is also causing CalPERS major headaches in terms of both customer support and IT fixes. Instead of decreasing the amount of time it takes to administer death benefits, the new system has doubled the processing time. Customer service on-hold times have also increased significantly since the switchover. Launched just two months ago in September, programmers already have implemented 16 system-wide updates and 1,340 fixes/enhancements. CalPERS member services staff estimate that the problem will be fixed in 60-90 days – a long time to go without health insurance,
The impact of a software glitch on users and companies goes way beyond aggravation. For CalPERS, the consequences are loss of time, money and productivity as well as damage to their corporate reputation. For its members, the glitch has resulted in the unnecessary payment of medical costs, delayed availability of prescription medicines and the anxiety that comes with losing one’s safety net.
Posted by Jeff Papows on Wed, Oct 26, 2011 @ 09:42 AM
On this blog and in the real world, we often tackle problems that (can) lead to malfunctioning software and products. However there is a category of problems that rears its head ever more frequently and, unfortunately, most business fail to address it in a meaningful way until it’s too late: Information Security.
Last week, it was noted that a glitch resulted in oft-troubled bank Wells Fargo mailing –physically mailing – account information to customers that belonged to other members. This information included account numbers, balances, transactions and contact information.
While Wells Fargo naturally says that the risk of identity theft is “very low,” they also have admitted that they do not know the extent of the problem and branch managers have been left to their own devices in how to deal with customers.
For its part, the bank plans to offer customers one free year of identity theft protection, and is encouraging customers to use online banking, which it deems “safer.” While that may be an implicit acknowledgement of a problem, it’s difficult to see how it would reassure affected customers.
Were the Wells Fargo problems not enough of a scare; right on its heels comes word of two additional information security issues.
This week, it was revealed that private information belonging to 5,000 college students was made accessible on a student loan website. Deemed the result of a “computer glitch,” users who logged-in to the site were able to view information, including social security numbers, of other student borrowers.
While the response was notable (the website was shut down and users were immediately notified), it highlights the government’s challenge in handling information relating to an already-cumbersome process of managing and disbursing government-backed loans for millions of U.S. college students.
Another glitch that has come to light this week involves G.E., and its electronic health records (EHR) systems. It was reported that glitches in its report writing systems have caused problems for health providers seeking incentive payments for using the EHR system. It is currently unknown how many providers experienced the issues; not surprising given the complexity of interactions between physicians, health care providers, state governments and the federal government in the U.S. health system.
Until the problem is fixed, customers are being told to “run the reports again.” And while the problem almost certainly will be corrected, the fact that it exists to begin with, and the response of “try it again,” won’t do much to convince any naysayers of the benefits of EHR programs.
Posted by Jeff Papows on Tue, Oct 25, 2011 @ 03:08 PM
We don’t always expound upon every major glitch that makes the news affecting businesses and their customers (though we do point them out here), but a few have happened in succession over the past several days that warranted examination.
According to The Telegraph, Jaguar has found that once the cruise control was engaged in certain car models, it became difficult to disengage it in what the company calls a “normal manner.” As a result, Jaguar is recalling 18,000 cars in the U.K. The problem (fortunately discovered by an employee) results from a glitch in the cruise control software.
This is yet another example of what can happen when a mismatch occurs between software vendors and manufacturers that may have little expertise with implementing software controls. While we’re annoyed when a navigation system doesn’t work correctly, we often overlook the fact that the same problems (software code and process) can also lead to more severe results.
Additionally, the New Zealand-based telecommunications company, Telecom, has refunded more than $2.7 million dollars to customers after a software flaw caused inaccurate data usage readings for more than 95,000 customers, causing many to be charged overuse fees for exceeding their data caps between November 2010 and June 2011.
Unlike Jaguar, the company did not discover the problem. Instead, customer complaints finally lead Telecom to take action. And while this problem is minor in comparison to the Jaguar recall, it serves as an illustrative example of the pervasiveness of software problems in our lives.
Posted by Jeff Papows on Thu, Sep 01, 2011 @ 02:25 PM
When speaking to prospects about what we do, or to customers about additional functionality, we get a lot of questions. There are some, however, that we hear more frequently. Below is a sample.
“A big component of governance automation is automating the review process. Do you have tools to manage review/inspect contents of the review package (Word documents, pdf, etc.) and how do you track action items?”
We have a component called Review Center within WebLayers. It works like this: you complete a review template, and whenever a service gets created, a review is created for that service at various lifecycle stages. Automated test results also are included in a dashboard where you can track submitted items and comments from users. It streamlines the process and eliminates having to sit around a conference room debating changes.
“What is the most common policy you see companies implement?”
We get this question all the time. It’s like asking which rivet on an airplane is the most important – they’re all important. What’s critical is having the best set of policies for your particular environment.
“Are policies enforced to absolutely prevent violations, or is there a concept of policy deviation approvals?”
WebLayers uses a waiver process; when a deviation occurs causing a prohibitive event, you can then request a waiver.
“When should I establish SOA governance?”
The simple answer is the sooner the better. Customers feel that they can buy a tool that can automatically give you governance. There needs to be some forethought put into looking at existing IT processes and establish a simplified governance process that you can translate into something actionable and automated with the tool.
It’s much easier to get SOA best practice and policies in place for managing your services when you have a small number of them. Then, as the number of services increases, you’ll find it easy to apply company-wide standards and policies to them and manage consumer provider relationships.
Posted by Jeff Papows on Fri, Aug 05, 2011 @ 02:03 PM
An article from CNN’s John D. Sutter
appeared Wednesday discussing the evolution of hacking from prank-like internet amusement to attacks that damage real-world infrastructure at this year’s Black Hat conference.
The article centered on a presentation given by Cofer Black, former director of the CIA’s Counterterrorism Center, describing what he sees as an impending “Code War.” Black is famous for, among other things, warning (in August 2001) the U.S. government of an impending al Qaeda terrorist attack. It goes without saying that Black’s warnings should be heeded.
It’s something we’re also hearing about first-hand in customer interactions and more general software discussions with attendees at industry events – namely, protection against intrusions that may go beyond disrupting websites and web services. Instead, concern is forming around using these entry points to damage something on a more fundamental/infrastructure level that cannot be so easily remedied by shutting down a particular service.
As an extreme example, Black mentions the
Stuxnet worm discovered last year that is designed to infiltrate Siemen’s industrial software. While theories abound as to who could have had the resources and knowledge required in creating such an expensive, complicated attack (and why the attack seemed designed to target Iran and its nuclear programs in particular), Black’s larger point is that these types of attacks will become easier, cheaper and more common.
Perhaps because of the multitude of high-profile attacks that have been taking place over the last 12 months, we’ve been hearing about this as an increased concern from companies in a variety of industries. It’s not that the attacks have just now become possible, but that their proliferation has transformed them from unlikely possibility to a reality. What was once an “if,” is now a “when.”
Black says that, in a way, the 9/11 attacks were a validation of sorts for his team. A controversial statement if it were to be misinterpreted, but it is one that we can relate to here. While we’re not a tech security company, our experience has given us great insight into the often-flawed software that makes its way from development into production, and the multitude of risks that companies are exposed to as a result. Hopefully it will not take another attack on the scale of Stuxnet for this message to sink in.
Posted by Jeff Papows on Fri, Jul 29, 2011 @ 02:49 PM
Our final look into some common misconceptions regarding software governance.
I use registries, so I don’t need governancePerhaps the easiest one to explain. Registries are not governance, and it’s really that simple. In this case the old adage applies very well: "garbage in, garbage out" and, unless you are convinced your checked-in code is perfect, then your registry/repository is a landfill for software glitches and flaws.
Defining policies is riskyAn understandable fear. In a worst-case scenario, a poorly thought-out policy can have serious negative consequences. Therefore, understanding the impact of a policy before making it operational should itself be a policy.
Run a business impact simulation against the existing artifacts, services, projects, systems, etc. before making a decision on policy changes and you’ll find out it’s much less risky than you think.
Architects and tech managers are too busy for another processWith such large responsibilities, there is little time for manual execution of policy enforcement. Automated governance allows your best tech people to focus on strategic initiatives, rather than eyeballing code.
Posted by Jeff Papows on Fri, Jul 22, 2011 @ 09:34 AM
Our third look in as many weeks into some of the most common misconceptions about governance that we run into.
Governance is only needed for mature SOA initiatives
A truly effective SOA paradigm requires governance from Day One. Assets will stay with companies for years to come (assuming, of course, that the related applications perform as intended), so get it right the first time. Additionally, it is critical to apply governance processes in your pilot or other early stage projects.
SOA governance can be done without policy enforcement.
Policies are the cornerstone of governance. Control is critical as SOA projects evolve and governance needs to happen in all stages of the SDLC, as well as all systems in your enterprise. This promotes effective adoption and educates developers on industry/corporate standards, best practices and design specs for compliance in your organization.
Automation is something to fear
Notebooks of policies are almost never implemented consistently…even if you continue to throw bodies at the problem. If a policy is written, but nobody reads it, is it really a policy?
Policies must be translated into actions and enforced. To do this effectively, the process must be automated and the results must be measured. Proactive, automated policy enforcement ensures quality, increases productivity and reduces risk.
I have testing tools, so I don’t need governance
Another common one. Testing tools are not governance, despite what you may hear elsewhere (existing vendors). You need to enforce policies throughout every step in the lifecycle - from design to development, through testing and into deployment. Testing tools are necessary, but alone are insufficient for achieving anything approximating governance.
Posted by Jeff Papows on Fri, Jul 15, 2011 @ 10:24 AM
Building off of last week’s post mentioning various misconceptions surrounding software governance, below are a few more that we see in the field.
“We only work in X language”
It shouldn’t matter, and this is especially so for SOA initiatives. A comprehensive governance system, like a SOA project, is not dependent on languages or platforms, but rather on initiatives that result in quality, trust and control among various participants.
“Governance takes work out of the hands of developers”
Actually, yes, but only the work they don’t want to do. Developers consistently face deadlines and milestones, so time is always a source of pressure. Developers can check from within an IDE to get iterative guidance, which means not having to wait for the QA phase to see if the service is compliant. Simply put, this means less rework.
“Developers are resistant to the idea”
On the contrary, governance, especially automated governance, makes their lives easier. It also refocuses development priorities on end results, rather than minutia involving code.
Additionally, the reporting functionality offers a way of displaying the value of developers to non-tech employees without having to explain the details of work that is so esoteric to most.
More to come next week.