Epilogue
Being a software developer I were not worried much about an issue’s status after finally clicking on the so desired “Resolve” button in an issue tracker software. Let’s skip all pathetic words about “personal responsibility” for the delivering production ready and tested functionality – at some point the developer’s responsibility is over and either an end-user will work with it or, that it’s much better, a QA team start verifying the just developed of fixed functionality.
However, becoming a Project Manager I were forced to look deeper and be aware of the status of each issue (story, task, bug etc.) until the moment of deployment it on all existing environments: testing, staging and production and it is the place where the problems start appearing.
Introduction
Atlassian Jira has already become the de facto standard for many projects, although appliance of it in real projects causes difficulties and attempts to map real project's processes to simple workflows.
If we remember the standard issue workflow in Jira:
It looks good, isn’t it? Yes, but for very simple project: functionality specified, developed, verified and closed. However, the real use of this workflow in real projects, in my mind, is limited at least due to one simple reason:
- Huge amount of projects requires several phases of verification before the production deployment: testing, staging, production. In case of mobile applications that are different OS versions and resolutions etc.
As the result, the whole QA team should balance on the edge between the Resolved and Closed statuses.
If for the development team it's possible to use different filters (Open, In Progress, Recently updated or Resolved) and/or Agile board, for the QA team the standard functionality that is suggested by Jira is extremely limited.
Many special web and desktop applications were created for test cases management, but here I want to focus how to use and configure Jira for needs of a project. Probably, it is the reinvention of a wheel, but nevertheless.
What typical questions are arise in mind of a project manager?
- What is the tracebility matrix for a release?
- What are the current tasks for all members of the QA team?
- What tests cases have been written, been approved or rejected, been completed verified or have failed test runs?
- If it is the post-deployment testing, how many test cases have been already passed and how many still awaiting execution?
Let’s start with the defining new issues types and corresponding workflows in Jira.
Test Cases management in Jira
New types of issues should be defined in order to structure the hierarchy of issues that will help with describing the QA workflow in Jira.
Figure 1 – The hierarchy of QA types of issue in Jira
New issue linking types
In order to manage relations of Test Cases to Test Suites and to User Stories a couple of linking types can be created in Jira through the "System - Issue Linking" menu item.
New custom fields
Actually, I'd like to highlight only one custom field that is essential for the functionality described further: "Test Environment". This custom field is the standard single choice list with all available environment for you and should be applicable at least to "Test Case" and "Test Run" issue types.
New issue types
Test Case - A formal written text that allows to test that requested functionality was correctly developed. Positive or negative. Several custom fields might be created specifically to shown on the default screen for this type of issue: "Pre-conditions", "Steps", "Post-conditions" etc. The required field is "Fix version/s", only by using this field it's possible to create a filter that will show all test cases for a specific release.
Figure 2 – Test Case issue example
Test Suite - A complete replica of a test case issue type, is used mainly for grouping test cases by means of Jira's linking feature.
Figure 3 – Test Suite Content linking type
Figure 4 – Test Suite issue example
Test Task - A sub-task issue type that reflects a QA member activity for running the specified test case on a specific environment. The required fields are "Fix version/s" and “Test Environment”. “Test Environment” field defines on what test environment the issue should be verified.
Figure 5 – Test Task issue example
Test Run - A result of passing test case's steps on a specific environment. The required custom fields: "Test Environment", "Test Run Result".
Figure 6 – Tests Runs linking type
Figure 7 – Test Task with the linked Test Run issue
Figure 8 – Test Run issue example.
New workflows
The most interesting aspect in the test cases management is the workflows for the test case, test task and bug issues.
Test Case workflow
Features of this workflow
- A test case should be assigned to a release. "Fix version/s" reflects the releases where this test case should be passed.
- After assigning a test case to a release for each environment will be created a separate test task for reflecting QA activities for verifying this test cases on a specific environment (will be done by a script and be discussed later).
- Only a QA lead can change a test case status to the “Reviewed”.
- After passing all checks by a project lead or verifying all child test tasks for a test case it returns to the “Ready to run” status.
Test Task workflow
An extremely simple workflow. A test task initially has “Open” status.
A QA engineer can start working on verifying this task by transferring an issue to the “In Progress” through the “Start Process” action or to the “Resolved” through the “Testing is not required” action if by some reason this test case is not applicable for this environment. Only QA lead can invoke the “Testing is not required” action.
After starting working on a test task a QA engineer can resolve it through the “Passed” action or mark it as failed through the “Failed” action or revert the test task to the “Open” status through the “Cannot complete” action if a test task cannot be completed.
Bug workflow
This workflow is pretty close to the standard Jira’s issue workflow, but with the minor changes:
- The standard “Resolved” status was renamed to the “In testing”. Simple action, but significantly increases the understanding what should be done further and doesn’t hide implicit “work is done” under the common “Resolved” status.
- It is possible to pass a bug to a QA team if no any development activities are required (e.g. some configuration changes in files, databases or portals settings) through invoking the “Development is not required” action.
- From the “In Testing” status a bug either can be verified and closed by passing to the “Closed” or be reopened by moving to the “Reopened” status.
Summary
We have abilities:
- to define a “Test Case”
- to group and link together test cases in a “Test Suite”
- to assign a test case to a release
- to track execution of these test cases on a specific environment through a specific "Test Task" issue
- to identify who was working on a specific “Test Task”
- to understand what is the current status of each “Test Case” and/or “Test Task”
Automating Test Cases management in Jira
Let’s think about actions in the afore-mentioned workflows that can be automated for each issue type.
“Test Case” workflow automating
- On changing an issue from the “Ready to run” status to the “In Progress” through the “Assign to a release” action for each combination of the “Environment” and “Fix Version/s” will be created a new child “Test Task”.
- If a “Test Case” already has a “Test Task” for a specific environment and version, this task will be reopened.
- A parent “Test Case” will automatically change its status from the “In Progress” to the “Ready to run” if all child “Test Task” have the “Resolved” status.
“Test Task” workflow automating
- Each running of a “Test Task” will be reflected by creating a child “Test Run” issue for each run.
- Workflow’s actions “Passed” and “Failed” will set the “Status” value.
“Bug” workflow automating
- On changing an issue from the “In Progress” to the “In Testing” through the “Development is done” action for each combination of the “Environment” and “Fix Version/s” will be created a new child “Test Task”.
- A workflow’s action “Close an issue” will be executed if all child “Test Task”s have the “Passed” status.
- If at least one “Test Task” was marked as “Failed” a bug will be reopened.
- If a bug is being automatically reopened the last person who changed its status from the “In Progress” to the “In Testing” will be selected as the value for the “Assignee” field.
Comments
Post a Comment