software testing services
 Software testing training, consulting, and outsourcing from the experts: Rex Black Consulting Services (RBCS)
CALL US TODAY
(866) 438-4830
ISTQB certification testingISTQB certification testing ISTQB certification testing
PMI

Posts Tagged ‘software quality’

Bug of the Week: Possibly Ours

As those of you who receive our newsletter know, we had a little glitch this week.  The newsletter greeted readers with “Dear (Contact First Name)”, when of course it should have given the reader’s first name rather than the parenthesized stuff. 

Immediately–given the 14,000+ good testers who read our newsletter–we received many reports of the defect.  A massive self-head-smack and “Doh!” action (a la Homer Simpson) ensued at the RBCS mothership.  We promptly issued a 20% discount code to our readers as a way of apologizing for–well, someone forgetting their names. 

So, who forgot their names?  We use ConstantContact for our e-mail, so one logical explanation is a bug in ConstantContact’s mail merge code.  We suspect that is the cause, because a test e-mail sent to Laurel Becker, RBCS VP, included the “Dear Laurel” string, but in the wrong place. 

That said, there’s the “never attribute to malice that which can be explained by human error” angle to this.  Did we accidentally mangle the newsletter template in a way that overrode the mail merge function?  It’s certainly a possibility, and we can’t tell from the source newsletter whether that is the case.  This is a bug in ConstantContact’s editing tool, because it’s possible to damage these mail merge keywords in such a way that you can’t see the mistake in either the source letter or the test copies.

So, the conclusion at this end is: It’s a mystery.  We apologize to our readers again for appearing to have forgotten their names.  We’ll keep an eye on this for future newsletters.

In the meantime, this is…uh, yeah, Rex, that’s right…Rex here reporting a possible bug…or just an inattentive copy-paste action somewhere in the process.

Software Bugs of the Week

This post is part of my all-too-regular series of posts on bugs that have vexed, frustrated, or inconvenienced me, my clients, or the world in general.  It is a series that shows no sign of abating, sadly.

This week we had two bugs, one fairly nasty, one less so but still irritating.

To start with the nasty, as some readers will know, we use Articulate to author and host our e-learning.  I’ve used other authoring and hosting tools in the past, and found them all relatively immature and buggy.  Articulate is better than the others I’ve used, but by no means good enough, as this latest incident illustrates. 

Articulate did a “scheduled update” on Saturday.  They were nice enough to let us know about it a week beforehand, so we could warn our e-learning customers about downtime.  In the past, we’ve had “stealth updates” where they didn’t tell us and then all of a sudden something stopped working; they would then deny that they had done an update and insist that we must have changed something.

However, Articulate apparently wasn’t nice enough to thoroughly regression test the update before they put it into production.  As a result, all of our podcast attachments (MP3 audios of the e-learning courses) are corrupted.  Re-attaching them doesn’t work, so we had to file a bug report with Articulate. 

Did I say they didn’t regression test the update?  Wait, I’m wrong, they are regression testing it, using their paying customers to do so.  Why bother with equivalence partitioning the support attachments and testing that they all work when you can just promote untested code into production and make your paying customers do free work for you as testers?  What’s most frustrating about this incident is that this is certainly not the first time it’s happened, and Articulate is by no means alone in employing this infamous tactic. 

Now, on to the irritating.  I did a webinar today.  Over 400 people showed up, which was great.  What wasn’t so great was that a few of them had a weird bug affect them.  They reported not seeing the slides, but only the white welcome page with a greeting from me and my picture.  As attendee James Heeren, who was affected by the bug, put it:

A funny thing happened when I closed the Webinar Window, a second window appeared with the picture of the graph of defects reported and defects closed.  [This was a slide in the webinar presentation.]  The real thing was hidden behind the white page somehow.

Was not a big problem for me as I followed on the deck from the website.

If it were just one person, I chalk it up to someone moving the focus to another tab in their browser by accident.  Since it was quite a few people, it sounds like what was happening was that the GoToWebinar software was not properly getting the focus on the presentation window in some cases. 

I have some sympathy for Citrix in this case.  This sounds like a configuration-specific bug, which is really hard to find.  That said, even though we will report it, I’m not expecting a lot of help from Citrix, based on past results of reporting bugs to them.

Until there is some implied warranty rule for software, as there is for every other product and services sold in the US, this kind of nonsense will continue.

Product Quality Includes the Whole Product

Fresh off my diatribe just a couple weeks ago about how McAfee had a bug that serendipitously made a credit card charge in their favor, here’s another “bug from the trenches.”  Now, Cisco enjoys a good reputation for the quality of their products, but the packaging doesn’t exactly inspire confidence.  Can you spot the error?

Does Packaging Quality Matter?  Yes

Does Packaging Quality Matter? Yes

The point of this post is not to yank Cisco’s chain–though I am surprised to see such a large, serious organization making such an obvious and silly mistake–but to reinforce an important point. Product quality includes the whole product, and that includes the package.  I’ve done some consulting for systems and consumer electronics makers, and most of them include testing of the “out of box experience”. 

Cisco, did your OBE testers miss this one?  I’d be happy to post a comment from Cisco explaining how this kind of obvious bug snuck past.

Testing and Quality Metrics: How to Stay “Lean and Agile” while Maintaining Visibility?

As part of the continuing series of video blog entries, I’m throwing out to the wider software testing and quality community a question:  How do we balance the desire–especially in Agile and Lean/Agile projects–to enjoy the efficiency of lightweight record-keeping, while at the same time having enough visibility into project, process, and product status, including testing and quality metrics?

For a little video context on my question, check out: How to Balance Metrics, Lean, and Agile when Measuring Software Testing and Quality

I’d be interested in hearing from people as to how their projects are achieving balance here, and also in anecdotes about when they are not achieving balance.

What Do Test Stakeholders Want?

Next week, I’ll be in Germany for the Testing and Finance conference, giving a keynote speech on how testing professionals and teams can satisfy their stakeholders.  One of the key themes of that presentation is the following:

There are a wide variety of groups with an interest in testing and quality on each project; these are testing stakeholders.  Each stakeholder group has objectives and expectations for the testing work that will occur. 

When we do test assessments for our clients, we often find that test teams are not satisfying their stakeholders. 

Why?  Well, many times, what the testers think the stakeholders need and expect from testing differs from what the stakeholders actually need and expect.  In order to understand the stakeholder’s true objectives and expectations, testers need to talk to each stakeholder group.  Since in many cases the stakeholders have not thought about this issue before, these talks often take the form of an iterative, brainstorming discussion between testers and stakeholders to articulate and define these objectives and expectations. 

To truly satisfy these stakeholders, the test team needs to achieve these objectives effectively, efficiently, and elegantly. 

  • Effectiveness: satisfying objectives and expectations to some reasonable degree.
  • Efficiency: maximizing the value delivered for the resources invested.
  • Elegance: achieving effectiveness and efficiency in a graceful, well-executed fashion.

The next step of defining these objectives, and what it means to achieve them effectively, efficiently, and elegantly, is often to define a set of metrics, along with goals for those metrics.  These metrics and their goals allow the test team to demonstrate the value they are delivering.  With goals achieved, testers and stakeholders can be confident that testing is delivering satisfying services to the organization.

Are you satisfying your stakeholders?  Catch me in Bad Homborg, Germany, on June 8, to discuss the topic with me directly.  Or, your can e-mail info@rbcs-us.com to find out when we will post the recorded webinar on the RBCS Digital Library.

What Is Software Quality?

Since software testing is an assessment of quality, this question is not a theoretical one.  I suggest using a definition from J. M. Juran, one of the quality gurus who helped Japan achieve its astounding progress in the last 60 years.  Juran wrote that “quality is fitness for use. Features [that] are decisive as to product performance and as to ‘product satisfaction’… The word ‘quality’ also refers to freedom from deficiencies…[that] result in complaints, claims, returns, rework and other damage. Those collectively are forms of ‘product dissatisfaction.’”  Since satisfaction revolves around the question of who is (or isn’t) satisfied, it’s clear that Juran is referring to the satisfaction of key stakeholders.

Okay, that’s pretty straightforward.  So, how do we think about quality?  I suggest there are three ways to approach this question:

  • Outcomes: What outcomes do we enjoy if we deliver a quality product or service?  Clearly, we would have customer satisfaction, conformance to requirements, etc.
  • Attributes: What attributes must the product or service have to deliver quality? These would include functionality, performance, security, etc.
  • Means: What must we do to build those attributes into the product?  These activities would include good requirements, good design, good testing, etc.

So, if we think about quality properly, and think about the approach to thinking about quality properly, then we can approach the assessment of quality properly.



 
`