How much details Test Cases should have...

This is a very crucial aspect for designing the test cases. Even though it vary from situation to situation, but I'm giving some of the guidelines that may help you decide the optimal level of test case you should keep:

1. Test cases should cover positive, negative, boundary conditions, If it is a new requirement then we should try to cover full functional testing and if it is a patch or hot fix then we should focus more on Impact Assessment to the existing functions.


2. Wherever possible database testing should be planned have SQL related tests

3. We should keep number of steps to minimal and provide most of the tests data driven

4. Keep test data combinations in tabular format and try to make it compressed from easier maintenance and consistency

5. Use test techniques like DOE and Orthogonal array (also pair testing) for the scenario where we have large combination and data parameters

6. Identify and keep screenshots for test execution for future references and audits

7. Try to keep test cases modular and keep cross reference between different scripts for easier maintenance

8. In case you have related specifications available then you should try to keep references instead of repeating the scenarios e.g. For UI testing you can keep Style guides, for configuration testing, you can keep Configuration Baseline Document (CBD), for report validation, you can keep Report Mapping document etc for consistency purpose 

9. Have different sets (and levels) of test cases focusing on Functional testing (Validation of functional requirement), Platform testing (Validity of different environment combination), Installation Qualifications for Upgrades (Upgrade from older version to new version) as per applicability to your situations

10. For Regression testing you should try to focus more on positive and business scenario level details

11. Following criteria should also be considered while deciding the level of test cases needed:
         a. Test case detail dependency on the kind of lifecycle you are following e.g. Iterative vs. V-Model vs. Agile
         b. Details of test cases depending on critically and priority of modules
         c. Type of work is being planned - If you have your internal experts working vs if the testing work is outsourced
        d. Level of available requirements and design specs
        e. Level of testing performed functional, security, automation, performance

How you can grow in your testing career...

It is imperative for everyone to grow in their career with learning and understanding new aspects and improving knowledge and technical skills. Just like anything else we need to plan and follow what we want to achieve. I'm here by providing my inputs on how you can grow in your career and at the same time enjoy what you do the best. (Yes, I mean software testing):

1. Identify your interest areas - This is very important aspect to consider and you should do what you want to do and not just do something which is imposed on you. Identify your strengths and identify what are the areas that you likes working most. From a career point of you identify whether do you want to work as a Individual Contributor stream or you want to Managerial stream. Both have their own pros and cons and you are the best judge of what is most suitable to you. Similarly from technical side, you should think about what kind of testing you want to perform e.g. Functional, Penetration, Performance, Security, Automation etc. All of these are equally good and you can follow similar growth levels in any of these fields. Select a career choice and work towards building relevant skills.
2. Identify skills that you want to improve - Identify what are the skills you want to develop. Make short term or long term goals and follow them. e.g. for a short term you may want to learn something related to your work, domain or technology that may help you do your job better on a project or an assignment. For long term you may want to learn new automation tool or new language say Perl. It is important that you focus on few things rather selecting too many things at a same time and then failing to focus on something. You can take help from your friends, team members, leads, or other resources like internet, books, training course etc. This will help you in learning these skills better and faster.   
3. Learn new Technology, Tools, Techniques and Approach - Spend some time i learning the technology in which the product is built upon. Even though you may not needed to knwo this directly but rather it will give you knwoledge of how application works and will help breaking the same in more effectively. Always be on lookout for learning new techniques and approaches. This will help you improvise when needed as you will have spectrum of choices to solve one problem and you will become more effective. Believe that 'things can be better if tried differently'. Similarly learn new tools that helps that may help you more productive, some of the tools that you may want to focus might be, Automation tools, Performance testing tools, Security testing tools, Test data generator tools etc. You can some some of the tools that best fit your need and learn them. 
4. Challenge yourself - No challenge is better and fruitful than challenging yourself. It is the most important and enjoying than anything else. Identify few areas and try to make yourself sweat with the challenges. Try to do things faster in lesser time, try to learn something in short duration, try to minimize test cases (still keeping the effectiveness) etc. Also try to spend some time for physical activities to keep yourself fit by involving your-self to some sort of activity jogging, gyming or playing some sports. 
5. Read books and Read blogs of industry thought leaders - Try to spend some time of the day (may be an hour atleast) reading blogs on the net. It will give you new perspective to various situations that other people faces and it's a very good mechanism to get different perspective. Also read books related to your interest areas. This will help you clear any doubts and help you learn things comprehensively and in details.
6. Share your thoughts with other and learn - Share your ideas, thoughts, knowledge with others within your teams, blogs, forums, groups. Discuss your point of view with others and see how people think about it. This will help you get lot of insights to certain areas and thoughts and in the process you will find new, interesting and useful skills, techniques, approaches and methods.

Remember - Knowledge is what separates success with luck...

How to conduct effective Root Cause Analysis...

Root Cause Analysis is a very effective tool to conduct Causal Analysis. This process helps understand the problems which are being report and take corrective and preventive actions so that similar problems are fixed permanently and do not repeat in future.

Following are the steps to conduct effective Root Cause Analysis. Given below are the process steps with few guidelines to help complete RCA in a quick time and with effective and productive manner. These guidelines can be extended as needed for any specific scenario. The steps for conducting RCA are following:

A. Modularize Issue - Identify and categorize the issue, so that it can be assigned to a individual or a team
B. Identify Root Cause of Issue - Identify what exactly caused the issue to occur
C. Identify Corrective Actions - Identify what we need to do to provide immediate fix
D. Identify Preventive Actions - Identify what we need to do to prevent similar problem from re-occurring
E. Review with the team and Action items tracking - Identify and track actions items for the team

Let's understand each of these steps in details on what activities we should do in each of these steps:

A. Modularize Issue –

   #1. Identify Modules and Sub-modules in the product to categorize the product
   #2. Identify Modules wise SME in project team
   #3. Identify issue to respective Modules and Sub Modules
   #4. Based on respective Modules, assign to the respective SME or team

B. Identify Root Cause

   #1. Review issue details
      a. In Bug database
      b. Service request id from Consulting/Customer Support
      c. Customer Issue details e.g. QC system of customer
      d. Any relevant email and other available information

  #2. Define the problem
     a. Understand the problem that is being reported based on available information
     b. Understand the area of problem and steps to reproduce the problem
     c. Identify any dependencies that may help in understanding the problem
     d. If there is any gap in understanding talk to respective support, consulting or Dev resources

   #3. Reproduce issue in test environment
     a. Identify a respective QC environment on the release issue found e.g. if issue is found on version 5.1 then check issue in version 5.1 test environment
     b. Reproduce in QC test env
     c. Try to reproduce issue with the exact steps provided in the issue

     d. If issue do not get reproduce then seek more information rather than giving up!
       i. Send email to respective consulting or CS representative
       ii. Talk to respective Dev lead/Developer or peer testers
       iii. Other relevant sources that may help…

     e. Smart Thought (If QC environment is not available)
       i. Check any possible env with Dev team
       ii. Check any available env with CS or Consulting
       iii. If above options not available then prepare quick env with bare minimum components

(Note: It is highly advisable to keep latest release up-and-running to expedite issue reproduction process)

   #4. Analyze issue on following parameters
      a. Behavior of Design
        i. Is this a known issue
        ii. Do we have workaround conveyed to customer in release notes
        iii. Do we have design document covering this scenario
        iv. Have we prepared Impact assessment document covering this scenario (for patches)
        v. Problem in logic
        vi. Problem in algorithm
        vii. Problem in coverage
        viii. Problem in test strategy
        ix. Problem in performance

      b. Behavior of Coding
        i. Code segment not correct
        ii. Code segment not as per requirement
        iii. Code segment has logical error
        iv. Unit test strategy not covers this scenario
        v. Unit testing not conducted for this scenario
        vi. Unit testing not challenging the business scenario
        vii. Unit testing not have adequate test data and test coverage
        viii. Code review not done properly

     c. Behavior of Testing
       i. Scenario not part of test strategy
       ii. Scenario not part of test scenario
       iii. Scenario not part of test steps
       iv. Was testing done for this issue
       v. Was coverage enough for the issue
       vi. Was Adequate regression testing done for the issue

C. Identify Corrective Actions

     #1. Is there any workaround available
     #2. Can workaround be suggested to customer
     #3. Can a small and quick fix be provided to customer
     #4. Does this require quick suggestion on IG and IQ steps that may help prevent the issue in near future
     #5. How many customers are affected by this issue
     #6. What is the Risk Priority Number for this issue (Severity x Occurrence-ability x Detect-ability)
     #7. What it will take to fix the problem in the code branch
     #8. What it will take to fix the problem in Main line code

D. Identify Preventive Actions

     #1. Requirements
        i. Was requirement was clearly stated
        ii. Was requirement reviewed completely by team
        iii. Was requirement review budgeted for this area
        iv. How we will ensure that this problem do not occur in future

     #2. Planning
        i. Was this task planned in test strategy
        ii. Was this task planned in mpp
        iii. Was estimation correct (Have we missed some scenarios)
        iv. Was enough time provided for the task in mpp
        v. Was the plan followed with team and status updated correctly
        vi. How planning can be made better for similar tasks

     #3. Design
        i. How similar logic/parameter/algorithm will be designed correctly in future
        ii. How similar logic/parameter/algorithm will be implemented fully in future
        iii. How can we prepare better test strategy during design
        iv. How will these gaps can be identified during reviews
        v. How can we plan and conduct good design practices

     #4. Construction
        i. How and why this scenario will be implemented correctly while coding
        ii. How will we follow coding guidelines and checklist
        iii. How will we add scenario part of Unit test strategy
        iv. How will we ensure unit testing strategy is covers similar scenario
        v. How will we ensure scenario missed in Unit testing
        vi. How we will ensure all boundary scenarios are covered in Unit testing
        vii. How Code Review done properly for the code to highlight the issue beforehand

     #5. Test Script
        i. How will we identify similar scenario part of test strategy
        ii. Was this scenario present in Test Script
          1. If Yes
             a. Was this step executed properly
             b. Why execution of step did not found the issue
             c. Identify any missing test data
            d. Identify any missing negative boundary condition
          2. If No
             a. Identify why this step not present in script
             b. Identify why this was not caught in script review
             c. Add this missing scenario to the script

        iii. How will we ensure similar scenarios are not missed in review
        iv. Are we considering past issues and test assets while designing test scripts
        v. Are we considering learning from past (CAPA sheet) in next projects
        vi. Is this scenario covered in Regression scripts
        vii. Are existing regression scripts can discover the issue as-is
        viii. What changes are required in Regression script

     #6. Test Execution
        i. Why this scenario was not planned in mpp
        ii. Why this scenario was not executed
        iii. Why we do not have complete proofs and available in VSS
        iv. Why we do not have execution logs in VSS
        v. Do this scenario need to be covered in Functional test script for next project
        vi. Do this scenario need to be covered in Smoke test script for next project
        vii. Do this scenario need to be covered in Regression test script for next project

E. Review with the team and Action Items tracking

     #1. Once RCA is completed it should be reviewed with the team
     #2. Any additional points and information should be updated in RCA
     #3. A concrete action item should be identifed and planned for tracking
     #4. The action item should have an owner who is responsible for completing the action items
     #5. Action items should be reviewed by team on frequent basis (at least once a week/fortnight)
     #6. Once action item is completed it should be marked as closed with details of artifacts which are updated

Do Software companies deliberately ship products with bugs?

Almost all of the software companies ships their products with a known bug list. It is important to understand what forces these companies are forced to follow this practice on regular basis. Also most of the companies have their own defined criteria to determine which bugs they can defer and deliver with the product release. Some of the possible criteria can be:

1. Bugs which are low severity and low priority

2. Bugs that may not cause any business impact

3. Bugs which have some level of workaround possible

4. Bugs which are existing (since old release) in the product

5. Bugs which are not high Priority and requires high level of efforts to fix them

These are some of the reasons that may allow some of the bugs to be deferred for a release. For any organization it is very important to determine good enough testing. Testing is good enough (This is also referred as Test Stop Criteria) if:

1. Requirements are correct and each of them are covered with at least one test scenario

2. All required functional and non-functional test scenarios are identified and are executed

3. All logical paths and conditions are exercised

4. All failed requirements are revalidated with respective regression test

5. All Identified issues planned for fix are fixed and validated

It is important to think on how as a tester you can help, contribute and voice over these areas and help improve quality within your Team and line of business:

1. Determine appropriate test strategy that maximizes the probability of finding defect

2. Try to build a Bug Deferral Criteria with your LOB, BU or product suite teams that is agreed upon by all stakeholders

3. If a defect is logged then try your best that the defect is fixed. you may not be able to do alone yourself, so use mechanisms like status reports, escalation and proper communication involving your superiors

4. If a bug is marked as deferred analyze if the bug not severe and has possible workaround. If you suspect that this bug can raise concerns from customers or decreases competitiveness in the market then raise your voice and escalate immediately. I'm sure if the alarm is genuine then it will be heard out

5. Ensure that the possible workaround is published in appropriate customer facing documents e.g. release notes, user guides, online help etc

6. Identify the means on how you can minimize the number of backlog issues. You will certainly need support from your superior and management for this.

7. Become game changers and identify measures on how you can contribute your best.

How to reduce number of tests using test techniques

Many a times we are required to perform validation with a large number of test scenarios that will require significant time and effort to cover the complete coverage. Most of us face this challlenge on a day to day basis due to following reasons:
1. Large number of Permutation and Combination of tests
2. Lack of resource and time available for testing
3. History data is not available that will help identify prioritization of test 

To tackle this problem we may use test techniques that may help in reducing the number of tests and at the same time provide high level of coverage. This will ensure that we are using our resources effectively and optimally. one of the techniques that was used extensively in manufacturing domain and now used in software testing is Design Of Experiment (DOE).

DOE allows statistical analysis by experimenting behavior of different combinations of data. When you have large number of combinations then you can design your test data with Orthogonal Array (All Pair) techniques to reduce your number of tests. In this case instead of testing all n * n-1 combinations we create all possible pairs of data and test them.  Let's take an example:

Suppose you have following parameters to test:

O/S
Win 2008
Red Hat Linux
Mac

Browser
IE
Mozilla

Print size
A4
A5

In this case if you take all combinations then you will get a total of 3*2*2 = 12 combinations, with all pair technique you can get this validated in only 6 combinations:

TEST CASES
Case	O/S	Browser	Print
1	Win	IE	A4
2	Win	Mozilla	A5
3	Linux	IE	A5
4	Linux	Mozilla	A4
5	Mac	IE	A4
6	Mac	Mozilla	A5

As the number of test combinations increased you will see lot of difference between all nth combination test and all pair test.

There are tools also available but I have found tool provided by James Bach site as very effective and easy to use. There is a document inside the zip file that illustrates on how to execute the tool.

http://www.satisfice.com/tools.shtml

I would be more than happy to answer any related question on this.

Qualities you should evaluate while taking Interview for a tester position.

Interviewing is an art and it just like other arts comes by practice. While interviewing you are required to judge a candidate, based on only few questions, whether he/she is fit enough to work in your team and your organization. I'm outlining some of the important aspects that will help you in evaluating a candidate for testing role. These are not the only aspects you should consider rather you should observe a candidate based on your specific requirements for the open position: 

1. Plan the interview - Just like anything other activity you should plan your interview. You should go through the resume and understand different level of information e.g. Education, Type of exp, Type of project done, Domains candidate has worked, Knowledge of tools etc. Identify what are the key skills you need for specific role and position. Each position may require different level of skills and should be evaluated accordingly.
2. Type of questions - are very important and should depend on the candidates total experience. E.g. for the junior position you should ask objective questions (What is BVA?) and should expect objective answers. For senior positions in addition you should also ask subjective questions (like explain test plan and test strategy), this will help you understand the comprehensive knowledge of the candidate. Similarly you would like to ask different set of questions to a database tester, a functional tester and a performance tester. If you have doubts about the answer that is no harm in asking a counter or related question to check the depth of candidate skills.
3. Communication skills - This is one of the important aspect as the candidate will require to communicate a lot with different groups e.g. testers, developers, managers, customers etc. Candidate should have good written and speaking skills.
4. Faking the skills - Many a time you will observe a resume that mentions lot of tools and project around them. At a first look you will find the resume very impressive, but when you ask few questions you will realize that lot of information is not real and fake. You should spend first 5-10 minutes to understand if someone is faking the skill, if it is true there is no point in proceeding with interview.
5. Attention to Details - Try to judge candidate based on his attention during the interview. Observe how he/she is able to understand the questions in details and giving appropriate answers to the questions you have asked. If you have to explain same question multiple times then it may create trouble for you later.
6. Technical Skills - This is the most important aspect of the evaluation. You need someone in your team who has good technical skills on the specific areas you are looking. Evaluate candidate skills based on the requirement for your project. Also evaluate candidate skills for the specific role you are looking into. e.g. if you are looking for a security tester and he/she doesn't have a clue on buffer overflow or SQL injection then that may not be the worth.
7. Ability to learn new things - Identify the aptitude of candidates towards new aspects. If a candidate doesn't know the specific area you are planning to work, then evaluate if the candidate is curious, quick learner, shows interest in learning and exploring new things he/she doesn't know.
8. Knowledge of Test methodologies, techniques and approaches - If the candidate possess good knowledge on various test methods and techniques then he might prove effective when he/she is assigned tasks on the project. You may ask few questions to start with and see how effectively candidate defines test data for the problems or defines test approach for a particular case study.
9. Giving respect to the candidate - We should give respect to the candidate even if the candidate is rejected. If possible explain the kind of work and profile you are looking for. Also try to see if the person suits some other open requirements in your organization. A rejected candidate for security testing may be a prospective candidate for functional testing if he/she posses related skills.
10. Collation of feedback - Collate all the inputs you receive and make a call on where the candidate stands as per your expectations. You can prepare a skill assessment sheet where you have defined skills and rating against each of the skill area. This will help you make a go/no-go decision. Also based on your organizational setup try to analyze the candidate skills in respect to the organizational vision. Many a time technical teams pass the candidates but they are rejected in later stages of interviews for senior management and HR. You should also keep these things in mind. 

Myths and Facts about Software Testing...

Even though software testing is a complete software engineering role. There is lack of information available at acedamic level about software testing (at least in India). Due to this not many of the computer science graduates think about Software testing as a full fledged career and rather more attracted towards a programmer role. With this blog I'm trying to clear some of the myths that exists in people minds:

Myth 1 – Testers are non technical bunch of people who can test applications without using any technical skills

Fact – Testing is a Software Engineering job and Testers are software engineers who are equally qualified for any software engineering role

Myth 2 – Testing is a routine job and do not require techniques and thinking

Fact – Testing is a very challenging, innovative and creative skill that requires significant level of analytical thinking, system level thinking and lateral thinking

Myth 3 – Testing is a relative easy job and can be done very quickly as comparison to coding

Fact – Testing is equally (and more) tough and strategic as coding and also requires significant time in planning, designing and execution of tests

Myth 4 – There are not many and better career opportunities available in software testing field as compared to programming
Fact - Today testing is one of the best career available with career levels equals to any other software engineering role. Most of the organizations recognizes testing as a responsible and innovative role. 

I believe that being tester everyone should educate other people about these facts and make software testing as a respectful career choice.

Feel Pride in you and you being a TESTER...

Qualities a Tester must Strive for...

Software testing is a profession and like any profession that existed, software testing also requires tremendous level of abilities, skills and qualities. I'm trying to list some of the qualities that a Tester must strive for...

1. Passion for Quality – If you have this… then well…Rest will follow
2. Five Senses for Quality - Speak, See, Hear, Smell and Feel Quality  
3. Understand Technology – Learn & Earn Technological Advances 
4. Continuously Improve Technical Skills – Focused and Strategic learning 
5. Understand Product Domain - You can Break ONLY IF you know How It Works 
6. Think, Think and Think – Its all about thinking 
1. System Level Thinking 
2. Analytical Problem Solving  
3. Problem Decomposition Skills 
4. Lateral Thinking 
5. Survival mechanism – The fittest to survive 
6. Don’t restrict your thinking 
7. Think Smart, Do Smart, Be Smart – Curiosity and Improvisation 
8. Innovate with the ideas – Believe ‘Things can be better if done differently’ 
9. Be a Team player - Collaborate with others...Share and Learn
10. Be a Differentiator - Develop competencies and differentiate Outstanding work from Typical work
11. Be a Motivator - Motivate yourself and motivate others
12. Self Management and Discipline – ‘Don’t lose to your inner self’

How to Solve customer problems by identifying Critical to Quality (CTQ) approach.

Many a times we are faced with situations with problems being reports from customers. If we do not look into the problems quickly and handle them carefully then we may end up with unwanted escalations and loss of business opportunities.

To manage customer problems there are many frameworks available that helps understand the problem areas and then work towards correcting them. One of such technique is Voice of Customer (VOC) to Critical to Quality (CTQ) mapping techniques in Six Sigma. This is a very effective methodology to understand and solve customer problems very quickly and effectively. Some of the elements that we need to consider for this are listed below:

1. Identify Voice of Customer (VOC) - It is very important to understand the customer problems clearly. This may include carefully examine feedback provided by customer, talking directly to customers or getting details from customer issue base etc.
2. Service/Quality Affected - Once you identify the problem customer is reporting then try to map this to a specific Service/Quality Area. This will help in alignment of identifying team and other resources needed to resolve the problem.
3. Customer Expectations - Try to understand what exactly customer needs. If the need is understood correctly then you can target the corrective measures effectively.
4. Critical to Quality (CTQ) measurement - Identify the CTQ data measurement that will help you evaluate whether the exact problem is solved or not with the suggested corrective actions. This will also define benchmark for tracking and identification of improvement in future.
5. Action Items - This will help tracking of various action items that are identified while solving the problem. 

I'm giving an example for your easier understanding of this methodology:

Voice of Customer (VOC)	Service/Quality Area Affected	Customer Expectations	Critical To Quality (CTQ)
Customers complains that bills are either not provided to them or not delivered timely	Inconsistency of Billing cycle	Bill needs to be delivered consistently  and on time	Bill needs to be delivered on 10th of every month with a exception of +/- 1 day

Risk Based Regression Testing approach

One of the goals of Regression testing is 'To ensure defects in existing functionalities are not introduced while making changes or adding new features in a module'.


Regression testing is a strategic mechanism of selection of tests that needs to be exercised and performed in a release to uncover any defect which is introduced while fixing something or adding a new feature in the product or an application. One of the very effective and optimized mechanism to meet this goal is Risk Based Regression Testing approach.

Following are the steps to plan the approach:

1. First of all your need to organize your product into various modules and sub modules

2. Once the modularization is done then do Risk assessment for each module/sub-module from business perspective

3. Next step is to identify Impact on design and level of code changes done on each module/sub-module

4. You may also consider past Stability of each modules based on assessment from some of the metrics like ‘Total Issues reported against each Module’, ‘Severity of issues reported for each modules’ etc

5. Once you complete this assessment then you can identify easily the level of regression you need to perform.

Module	Sub-Module	Risk	Impact	Stability	Regression Needed
M1	S1M1	High	High	Fragile	Extensive
M1	S2M1	Medium	Medium	Stable	Medium
M2	S1M2	Low	Low	Stable	Low
M3	S1M3	Low	None	Stable	None

After each release you can review the data again and then you can redefine some of the parameters to make it more realistic and accurate for future references. This approach can help you define your regression strategy easily and ensure you focus exactly where you need to focus on impacted areas.