Wednesday, August 13, 2014

Anti-pattern for August: Walking in a Minefield


The title of the anti-pattern is: Walking in a minefield. The sub title is “the application’s ready for production… you go first."

The elaboration is read as:  When a product (known to be incomplete) is released prematurely, this forces users to stumble through the system finding bugs while the development team scrambles to resolve them.




This link provide some further reading on the anti-pattern (including interesting photo). For your convenience, I copied some here:

When software is released before it is ready, and users of the software are made to find all of its bugs and shortcomings, they're made to feel as though they're walking in a minefield. It's important to release software as quickly as possible, in order to minimize feedback loops, but what is released should work so that users do not lose confidence. Each new release should be more stable than the last, and should incrementally add new functionality. If the system is constantly shifting such that things that work one day fail the next, eventually users will demand a different system altogether, and the project will fail.

 Quotes

"Software undergoes beta testing shortly before it's released. Beta is Latin for 'still doesn't work.'" - Anonymous

There is a related anti-pattern, it says “let’s test it in production.”
 

The commonality of them is releasing a untested product to the end user, with the hope that the product is not buggy or if it is buggy, the users will help to identify the bugs for the development team to fix so that the software product can be useable.


From principle point of view, this is not a good practices, when the product is released for production, it need to have some kind of quality standard. Release an untested product to production is not fair for the users or client. It may very well negatively impact client’s business operation and could damage their reputation or expose client to other tangibles damage.

From technical point of view, some bugs could cause loss of data, and the lost data may not recoverable even if the bugs are fixed afterwards.

The biggest risk is that when an untested product exposed to its user, the users might very well discouraged by the buggy product and hence reject the product totally. 


However, in my decade’s application development experience, I have seen many incidences of the anti-pattern. I remember a few years ago, I attended a high level project strategy meeting. The meeting is to establish a strategy to address the issues of lacking of resource and time for developing and conducting test cases during the stabilization phase of the project. To my surprise, the decision was cutting down the number of test cases  significantly. The justification was: there is no enough time.  When I heard that, I could not believe my ear, this is exactly, “let’s test in production”

My take on it is simple, “Never”, never do that! An experienced project manager should be able to project such project risk and mitigate such risk so that it will never become an issue to be addresses in the project.  From technical resource point of view, when you are brought into a project to handle the crisis, you should take positive attitude do your best to recover from the crisis. I personally have been in such situation many times. The advice I could offer are the following:

 
1.    Backup database often, make sure the raw data entered by user is stored in database correctly. In production system, nothing is more important than data.

2.    When patching data error generated buggy logic, make sure the patching logic does not get extended beyond its intended scope.

3.    Fixing bug and patching data need to be in sync.

4.    Apply pair programming to reduce human error.

5.    When fixing bug, do your best to respect the original design and minimize the changes to be made. It is always risky to change design when product is in production.
 
Finally, remember there is no bug is small in production.  You could never be more careful. As the title said, it is a minefield, you do not want to walk in the minefield. When you are forced to walk in the minefield, be absolute careful.   

No comments:

Post a Comment